[Orca-users] too many open files issue -> found it

[Blair Zajac]

Hi Adrian,

Rather late reply, but I'm going through my Inbox at the beginning
of the year.

Are you still running into this issue with the number of open
file descriptors?

Happy New Year!
Blair

Adrian Bhatt wrote:
> 
> the version of perl i mentioned before that I had
> compiled myself was on an AIX box so it wasn't a fair
> test, because when I compiled perl on the sun box
> myself I still had the 256 open file limit. I started
> poking through perl documentation and found this in
> README.solaris
> 
> The stdio(3C) manpage notes that only 255 files may be
> opened using
> fopen(), and only file descriptors 0 through 255 can
> be used in a
> stream.  Since perl calls open() and then fdopen(3C)
> with the
> resulting file descriptor, perl is limited to 255
> simultaneous open
> files.
> 
> Where should I go from here?
> 
> Bob
> 
> --- Blair Zajac <blair at gps.caltech.edu> wrote:
> > Yes, Orca finding the number of file descriptors
> > could be done
> > pretty easily.  The problem is managing this number.
> >  Orca manages
> > this when opening RRD files, but it'll take more
> > modifications to
> > do this with the HTML files.  Probably the easiest
> > way is to have
> > Orca::HTMLFile store the HTML to write in memory and
> > then write it
> > to disk when the subroutine calling the object has
> > DESTROYED the
> > object.  This wouldn't be too hard to do, but it
> > would increase the
> > memory usage.
> >
> > Maybe a smart thing would be to have Orca::HTMLFile
> > see how many
> > open fds there are and then decide in the moment if
> > it should cache
> > the HTML and write it later in one go or open the
> > file now and
> > write to it continuously as the caller writes to it.
> >
> > Blair
> >
> > Adrian Bhatt wrote:
> > >
> > > I am using perl 5.005_03 that comes with solaris8.
> > It
> > > turns out that the perl binary I am using will
> > only
> > > open up 256 files. I determined this with
> > >
> > > #!/usr/bin/perl
> > >
> > > use FileHandle;
> > >
> > > for ($i=0;$i<2048;$i++){
> > >         $fd[$i] = new FileHandle;
> > >         $fd[$i] -> open(">$i") or die "stopped at:
> > $i:
> > > $!\n";
> > > }
> > >
> > > When I ran this same script on a version of 5.6.1
> > that
> > > I compiled it would run all the way up to the
> > ulimit.
> > >
> > > Would there be an easy way to check this limit in
> > perl
> > > and manage file descripters appropiatly?
> > >
> > > Bob
> > >
> > > --- Blair Zajac <blair at gps.caltech.edu> wrote:
> > > > Hello,
> > > >
> > > > The way to limit the number of open files that
> > Orca
> > > > uses is to edit
> > > > lib/Orca/OpenFileHash.pm and find the 100 and
> > lower
> > > > this.  Try this
> > > > and see what happens.  Which version of Orca are
> > you
> > > > using?
> > > >
> > > > If this doesn't work, then can you try running
> > Orca
> > > > with just one
> > > > host instead of 6 and put the HTML pages up for
> > > > viewing so I can
> > > > take a look at them.
> > > >
> > > > At this point, put the scripts up on your site
> > with
> > > > some example
> > > > plots, that's the best way.
> > > >
> > > > Regards,
> > > > Blair
> > > >
> > > > Adrian Bhatt wrote:
> > > > >
> > > > > I searched the archives and some some
> > discussion
> > > > to
> > > > > having issues of too many open files. Most of
> > > > those
> > > > > issues were solved by increase the file
> > descripter
> > > > > limit from something like 64 to 128 or 256. We
> > > > started
> > > > > our file descripter limit at 512. All was
> > working
> > > > > really well. Last night at the request of
> > certain
> > > > data
> > > > > from the business we added some plots
> > detailing
> > > > some
> > > > > more information about different file system
> > > > usage. So
> > > > > I set up the plots at let orca munge through
> > all
> > > > the
> > > > > data, to my dismay I came back an hour later
> > to
> > > > find
> > > > > error messages like the ones below. So I did
> > some
> > > > math
> > > > > and figured that at 6 servers, the normal
> > plots,
> > > > as
> > > > > well as a plot for each file system (40 per
> > box)
> > > > that
> > > > > the number of files (html/png/meta) that orca
> > > > would be
> > > > > creating was reaching into the thousands. So I
> > > > > increased the file descripter limit to 1024.
> > Still
> > > > no
> > > > > good. Then to 2048. Still no good. Then to
> > 4096.
> > > > When
> > > > > it failed at this point I began thinking
> > something
> > > > is
> > > > > wrong. I started looking through the code
> > mostly
> > > > at
> > > > > HTMLFile.pm then how it was called. I am a
> > little
> > > > > confused about how you open files then how
> > they
> > > > get
> > > > > closed, but that is probally because I only
> > spent
> > > > > about 20 minutes looking through it. Is it
> > > > possible
> > > > > there is some sort of leak that under X
> > condition
> > > > it
> > > > > doesn't close some file handle right away? If
> > so
> > > > were
> > > > > should I start looking for such a leak?
> > > > >
> > > > > We have also created some scripts which create
> > > > data
> > > > > files for various peices of equipment to feed
> > into
> > > > > orca. Is there a place to publish such scripts
> > or
> > > > > should I publish them on my site and send a
> > link
> > > > to
> > > > > the user list? Just wondering how to give back
> > to
> > > > the
> > > > > community.
> > > > >
> > > > > Thanks again for any help. It would be greatly
> > > > > appreciated.
> > > > >
> > > > > Bob
> > > > >
> > > > > /opt/orca/bin/orca: warning: cannot create
> > > > > Orca::HTMLFile object: cannot open
> > > > >
> > > >
> > >
> >
> `/opt/orca/data/e10k/html/o_mntC_per_mkt_per_app_per_dqmgr_p,o_mntU_per_mkt_per_app_per_dqmgr_p,o_mntA_per_mkt_per_app_per_dqmgr_p-monthly.html.htm'
> > > > > for writing: Too many open files.
> > > > >
> > > > > /opt/orca/bin/orca: warning: cannot create
> > > > > Orca::HTMLFile object: cannot open
> > > > >
> > > >
> > >
> >
> `/opt/orca/data/e10k/html/o_mntC_per_mkt_per_app_per_dqmgr_p,o_mntU_per_mkt_per_app_per_dqmgr_p,o_mntA_per_mkt_per_app_per_dqmgr_p-quarterly.html.htm'
> > > > > for writing: Too many open files.
> > > > >
> > > > >