[Orca-users] orca bug: multiple data statements cause

[Danny Howard]

Hello,

This took me a LONG time to track down.

Orca 0.28.0 (dev 422)

I have recently gotten Orca to graph the memory patterns on my FreeBSD
systems.  The long and the short of it is:

>From FreeBSD:
0-14:45 djh at amun ~> /sbin/sysctl vm.stats.vm.v_free_count vm.stats.vm.v_wire_count vm.stats.vm.v_active_count vm.stats.vm.v_inactive_count vm.stats.vm.v_cache_count | awk -F : '{print $1": \t"$2 * 4096}'
vm.stats.vm.v_free_count:       125083648
vm.stats.vm.v_wire_count:       111083520
vm.stats.vm.v_active_count:     24334336
vm.stats.vm.v_inactive_count:   234483712
vm.stats.vm.v_cache_count:      22061056

To orca data files:
0-14:46 djh at ra /usr/local/orca# tail -1 var/orca/bigsister/in/amun:cpu.mem.*
==> var/orca/bigsister/in/amun:cpu.mem.active <==
1114033499 24322048

==> var/orca/bigsister/in/amun:cpu.mem.cache <==
1114033499 22061056

==> var/orca/bigsister/in/amun:cpu.mem.free <==
1114033499 125071360

==> var/orca/bigsister/in/amun:cpu.mem.inactive <==
1114033499 234516480

==> var/orca/bigsister/in/amun:cpu.mem.wire <==
1114033499 111067136

Over in orca.cfg:
base_dir   /usr/local/orca

state_file var/orca/state_file
html_dir   /usr/local/www/data/orca
rrd_dir    var/orca/rrd/orca

group Nextrials {
find_files         var/orca/bigsister/in/(\w+?):.*
interval           30
column_description first_line
date_source        column_name date
reopen             1
}

plot {
source    Nextrials
title     %g Memory Use
data      cpu.mem.wire
data      cpu.mem.cache
data      cpu.mem.active
data      cpu.mem.inactive
data      cpu.mem.free
legend    wire     bytes
legend    cache    bytes
legend    active   bytes
legend    inactive bytes
legend    free     bytes
data_type GAUGE
plot_min  0
line_type AREA
line_type STACK
line_type STACK
line_type STACK
line_type STACK
y_legend  memory in use
base      1024
}

So far, so good, right?  AH, but I was puzzling over system performance
... why do I see an increase in INACTIVE memory and a drop in FREE
memory when a certain rsync process runs on the box?  You would think it
would be consuming memory, not freeing it.  Well, it is because the
GRAPH I am looking at is WRONG!

In the hourly graph generated by the data and config above, I'll see the
following in Orca:
wire:      24M
cache:     21M
active:   120M
inactive: 224M
free:     105M

Basically, wire and active are getting swapped around.

Further, the graph on the amun hourly page then links to:
https://internal.nextrials.com/orca/_cpu.mem.active.html

Which shows data for a different system.  (The last in the alphabet.)

Originally, I simply tried doing "data cpu.mem.(\w+)" but then the
AREA-STACK thing would not work out ... I would have a bunch of
overlapping AREA graphs.

So, as a workaround, I try:
plot {
source    Nextrials
title     %g Memory Use
data      cpu.mem.(\w+)
legend    %1 bytes
data_type GAUGE
plot_min  0
line_type LINE1
y_legend  memory in use
base      1024
}

Get a bunch of line graphs, right?  Nope, I get errors:
1-14:55 djh at ra /usr/local/orca# /usr/local/orca/bin/bigsister-perf && /usr/local/orca/bin/orca -o /usr/local/orca/etc/orca.cfg |& tail -10
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/shiva/Nextrials_shiva_gauge_volatile_cpu.mem.(\w+)-monthly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/shiva/Nextrials_shiva_gauge_volatile_cpu.mem.(\w+)-quarterly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/shiva/Nextrials_shiva_gauge_volatile_cpu.mem.(\w+)-yearly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/sobek/Nextrials_sobek_gauge_volatile_cpu.mem.(\w+)-hourly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/sobek/Nextrials_sobek_gauge_volatile_cpu.mem.(\w+)-daily.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/sobek/Nextrials_sobek_gauge_volatile_cpu.mem.(\w+)-weekly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/sobek/Nextrials_sobek_gauge_volatile_cpu.mem.(\w+)-monthly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/sobek/Nextrials_sobek_gauge_volatile_cpu.mem.(\w+)-quarterly.png': can't parse ':average1#:'
/usr/local/orca/bin/orca: warning: cannot create '/usr/local/www/data/orca/sobek/Nextrials_sobek_gauge_volatile_cpu.mem.(\w+)-yearly.png': can't parse ':average1#:'
Orca has completed.

This is a bug in the dev version?  Is it fixed?  Is there a patch?  New
snapshot to try?  Workaround?  Thanks for any and all assistance.

Sincerely,
-danny

-- 
http://dannyman.toldme.com/