Difference: HowToMergeRootFiles (1 vs. 3)

Revision 32010-09-07 - ThomasBird

 
META TOPICPARENT name="HowTos"

How Merge root files

Ganga

The easiest way is to let ganga handle it with with the RootMerger()

j=Job(t)
j.merger = RootMerger()
j.merger.files = ['out-file.root']
Changed:
<
<
This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually...
>
>
Note, this only works with non grid backends. This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually, see below. Apparently its caused by root not being backwards compatible, https://twiki.cern.ch/twiki/bin/view/LHCb/FAQ/GangaLHCbFAQ#Root_merging_fails_segfaults_rec suggests that changing your version of root could fix this.
 

Merging manually

The program hadd is the one that the root merger calls to merge the files

hadd ~/outputfile.root ~/input1.root ~/input2.root [...]
Changed:
<
<
When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning, by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted.
>
>
When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning, by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted, they probably pull in different versions of root, which are not compatible.
 

Merging many jobs

If you have a few jobs each with hundreds of subjobs, this command will find all the output files and merge them. You will need to change 101 102 103 104 to your jobs numbers, output.root to the file that your jobs output and /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root to where you want your file to be stored. I suggest storing big files in /opt/ppd/lhcb somewhere.

cd ~/gangadir/workspace/uoh35620/LocalXML/
find 101 102 103 104 -name output.root | xargs hadd /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root 

-- ThomasBird - 2010-08-04

Revision 22010-09-06 - ThomasBird

 
META TOPICPARENT name="HowTos"

How Merge root files

Ganga

The easiest way is to let ganga handle it with with the RootMerger()

j=Job(t)
j.merger = RootMerger()
j.merger.files = ['out-file.root']

This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually...

Merging manually

The program hadd is the one that the root merger calls to merge the files

hadd ~/outputfile.root ~/input1.root ~/input2.root [...]
Changed:
<
<
When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted.
>
>
When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning, by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted.
Added:
>
>

Merging many jobs

If you have a few jobs each with hundreds of subjobs, this command will find all the output files and merge them. You will need to change 101 102 103 104 to your jobs numbers, output.root to the file that your jobs output and /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root to where you want your file to be stored. I suggest storing big files in /opt/ppd/lhcb somewhere.

cd ~/gangadir/workspace/uoh35620/LocalXML/
find 101 102 103 104 -name output.root | xargs hadd /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root 
  -- ThomasBird - 2010-08-04

Revision 12010-08-04 - ThomasBird

 
META TOPICPARENT name="HowTos"

How Merge root files

Ganga

The easiest way is to let ganga handle it with with the RootMerger()

j=Job(t)
j.merger = RootMerger()
j.merger.files = ['out-file.root']

This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually...

Merging manually

The program hadd is the one that the root merger calls to merge the files

hadd ~/outputfile.root ~/input1.root ~/input2.root [...]

When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted.

-- ThomasBird - 2010-08-04

 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback