[Orca-users] Increase threads to process bz2 orcallator files ?

[Hudes, Dana]

The bunzip2 itself is single-threaded. There is some limit on how many such children it will launch but on the 4-cpu v490 I have orca running on the bunzip2 processes are all i/o bound (which is why I have the data on SAN via dual 4-bit FC paths to the Hitachi 9990) 

I am more interested in parallel-processing the unbzip'd data: the orca I have is running cpu-bound pretty much single-threaded. All the processing for each host is independent so a perfect parallel-processing task (though there is that "site-wide" summary rrd et al). 

I have not had a chance to rebuild Perl with the auto-parallelization option in Sun Studio 12 (it IS built threaded 64 bit optimized for ultraIII+/native)
Supposedly that gives Perl 40% more throughput 


________________________________

From: orca-users-bounces+hudesd=hra.nyc.gov at orcaware.com 
To: orca-users at orcaware.com 
Sent: Wed May 27 20:59:45 2009
Subject: [Orca-users] Increase threads to process bz2 orcallator files ? 


I want to increase the number of threads used to bunzip the bz2 orcallator files so I can utilise more CPU and process them quicker ?? Any ideas ?
 

________________________________

This email, including any attachments, may be confidential or privileged, and is sent for the personal attention of the intended recipient.  If you have received this email in error, please delete it immediately.  The views expressed are not necessarily those of the Rabobank Group.  The Group is not liable for the effects of any virus which may be contained in this email. 

If this email contains marketing material and you do not wish to receive such material by email in future, please reply to this email and place the words "Remove My Details - Electronic Messages" in the Subject Header.

The Rabobank Group

Australia: 1800 025 484
New Zealand: 0800 500 933 

________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/orca-users/attachments/20090527/433b90bd/attachment.html>