[Orca-users] working with mixed-column data sources...

[David Michaels]

Hi, Steve --

Steven McPhillips wrote:
> I've got what I suppose could be called a normalised file format - 
> instead of having every log entry stored on the same line (as with the 
> orcallator source), my log entries have a recType column, for 
> instance. This column defines what type of data is being stored for 
> that record - could be JVM memory allocation, website user sessions, 
> current active threads etc. All up, my logfile has about 3 of these 
> "type" columns, along with a timestamp and "value" column.

Do all the recType entries share the same timestamp, or is there a 
different timestamp for each line?

If each 'set' of recTypes share the same timestamp for a given reading, 
then massaging the data into Orca format should only involve a bit of 
perl or awk scripting.  They could even be made 'live' by making a .. um 
.. I guess a "steveallator" ;)

However, if each recType has a new timestamp, it gets a bit more 
complicated.  Orca (and, ultimately, RRD?) wants a full set of data for 
each timestamp.  It will complain if some data is missing (unless you 
substitute a default/baseline value for such gaps).  If some of your 
timestamps are 'missing' data, or in other words, if your data sampling 
for one period is scattered across multiple timestamps, the converter 
would have to account for this somehow, such as by consolidating data 
from neighboring timestamps into one until a full set is obtained, 
throwing an exception if the timestamp discrepancy gets too big (e.g., 
greater than the period specified).

In a worst case scenario, you would have to resample the data using 
statistical methods, if you wanted an accurate graph.

My $0.02.
--Dragon nemo omnibus horis sapit


>
> How easy / difficult would it be to use Orca to plot my data? After 
> looking at the orca man page, I get the feeling it might be quite 
> difficult, although perhaps do-able using some judicious regexps for 
> the plot data entries. If someone on the list can give a simple yes/no 
> answer that would be great - I don't mind having to refactor the log 
> format via a pipe for orca to read in if I have to, but naturally I 
> would like to be neater if there is a better way!
>
> thanks in advance for any assistance,
>
> Steve
>
> *-----*
> Steven McPhillips
> http://www.nla.gov.au
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/orca-users/attachments/20060206/e079d4a9/attachment.html>