[Orca-dev] problem with mutex and high system time

[Tommy Henriksen]

Hi Orcausers and developers,
 
I thought I share this information with all you orca users and dev., - this is how orca is used in a production enviroment toghetter SE-Toolkit on a Solaris 5.8, - and what we have discovered sofar with both these tools.
 
It shows the strength of both products, - and how it can be used to solve "performance" related problems, - pinpoint where the problem really exist.  

Does anyone have similar experience related to mutex and high system time, - or just want to comment, - I would really appreciate.
 
I have attached two grahps, the graph usr-system time indicates that we have a large amount of system time compared to usr time, - at times even as high as usr time.
 
We can see from the other graph that we at same time have a lot of mutex spins. With lockstat -D 10 sleep 5 (see below) we found out that most of these locks came from enfk_malloc which is apparently a call one of our driver make intensively use of (a GNU grep reveals this, can grep binary files). More specific a caienf 64(CA) driver. I have also added the output from lockstat at bottom of mail. I gues if we stop this driver our problem is solved ??
 
Any comments ??
 
Best regards Tommy Henriksen
 
Adaptive mutex spin: 7237 events in 5.029 seconds (1439 events/sec)
 
Count indv cuml rcnt     spin Lock                   Caller                  
-------------------------------------------------------------------------------
 1427  20%  20% 1.00       15 enfk_mem_lock          enfk_malloc+0x2d0       
  733  10%  30% 1.00       10 enfk_mem_lock          enfk_free+0x1c0         
  683   9%  39% 1.00       25 0x30000da5038          callout_execute+0x98    
  294   4%  43% 1.00       24 0x30000da8048          callout_execute+0x98    
  282   4%  47% 1.00       10 0x30000da5038          timeout_common+0x4


 
 
 
 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: /pipermail/orca-dev/attachments/20041104/ed271d4c/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: usr-system time.png
Type: image/png
Size: 9863 bytes
Desc: not available
Url : /pipermail/orca-dev/attachments/20041104/ed271d4c/attachment.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mutex.png
Type: image/png
Size: 6334 bytes
Desc: not available
Url : /pipermail/orca-dev/attachments/20041104/ed271d4c/attachment-0001.png