[Orca-users] orcallator data stops at 18:35 each day and then starts a new percol file at midnight

[Charles R. Dennett]

Mark Bruen wrote:
> I'm experiencing a new strange problem where the orcallator.se data stops at 
> 18:35 every evening then a new percol file is created for the next day at 
> midnight as expected. The graphs are blank between 18:35 and midnight. I've 
> got the interval in orcallator.se set to 300 and in the 
> orcallator_<hostname>.cfg set to 300. Any ideas?
> Thanks.
> 	-Mark

Is the orcallator data collection process actually stopping at 18:35 or
is it still running but the orca process stops graphing new data at 18:35?

If the orcallator process stops, does it leave behind any message?  If
you are starting it from a boot script in /etc/init.d, try starting it
manually from a terminal session so that you can see any message it
produces when it aborts.

I always run the orca process with the -v option (verbose) and redirect
output to a file in /tmp so I can see what it is doing.  If the orca
process is not too big a load on the system, I run it from a boot script
in a continuous mode.  If the orca process tends to be a hog on a system
and the machine has other work to do, I usually run it in one-shot mode
(use the -o option).  That I do from a cron job so I can control how
often it runs.  In either case I use the -v option so I can "see" what
it is doing.  You may wish to try this too as it might give a hint for
what follows.

Assuming the orcallator never stops (and I'll bet this may be the case),
when this has happened to me it usually means that the data collector
has magically found new additional data to record.  Because the first
line of the data file, which is really the column headings for the rest
of the lines in the data file, no longer has a one to one correspondence
with the data lines, orca will stop graphing that data.  Using the -v
option as I describe above will produce an error message from orca when
it comes across this column mismatch.

Take a look at a data file and see if the data line at 18:35 has one or
more additional data items.   Yes, it will be tedious to compare it with
the line from 18:30 or the first line.

Now, why might this happen?  What new data could it find to record? My
first guess would be a filesystem that gets mounted at 18:30 or just
after that.  This would be a local filesystem, not an NFS one.  Do you
have a job that might do this?  A backup job perhaps?  Some other job
that might want to record data on a CD?  The mounting of a CD would
appear to be a local filesystem, I believe.  It may be something else,
but I think change in the number of mounted filesystems is the most
likely candidate.

The reason it starts again at midnight is that the data collector starts
a new file with a new column heading line as the first line in the file
and this keeps orca happy.

Hope this all helps.

Charlie Dennett