How Merge root filesGangaThe easiest way is to let ganga handle it with with the RootMerger()j=Job(t) j.merger = RootMerger() j.merger.files = ['out-file.root'] | ||||||||
Changed: | ||||||||
< < | This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually... | |||||||
> > | Note, this only works with non grid backends. This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually, see below. Apparently its caused by root not being backwards compatible, https://twiki.cern.ch/twiki/bin/view/LHCb/FAQ/GangaLHCbFAQ#Root_merging_fails_segfaults_rec suggests that changing your version of root could fix this. | |||||||
Merging manuallyThe program hadd is the one that the root merger calls to merge the fileshadd ~/outputfile.root ~/input1.root ~/input2.root [...] | ||||||||
Changed: | ||||||||
< < | When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning, by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted. | |||||||
> > | When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning, by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted, they probably pull in different versions of root, which are not compatible. | |||||||
Merging many jobsIf you have a few jobs each with hundreds of subjobs, this command will find all the output files and merge them. You will need to change 101 102 103 104 to your jobs numbers, output.root to the file that your jobs output and /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root to where you want your file to be stored. I suggest storing big files in /opt/ppd/lhcb somewhere.cd ~/gangadir/workspace/uoh35620/LocalXML/ find 101 102 103 104 -name output.root | xargs hadd /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root-- ThomasBird - 2010-08-04 |
How Merge root filesGangaThe easiest way is to let ganga handle it with with the RootMerger()j=Job(t) j.merger = RootMerger() j.merger.files = ['out-file.root']This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually... Merging manuallyThe program hadd is the one that the root merger calls to merge the fileshadd ~/outputfile.root ~/input1.root ~/input2.root [...] | ||||||||
Changed: | ||||||||
< < | When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted. | |||||||
> > | When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning, by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted. | |||||||
Added: | ||||||||
> > |
Merging many jobsIf you have a few jobs each with hundreds of subjobs, this command will find all the output files and merge them. You will need to change 101 102 103 104 to your jobs numbers, output.root to the file that your jobs output and /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root to where you want your file to be stored. I suggest storing big files in /opt/ppd/lhcb somewhere.cd ~/gangadir/workspace/uoh35620/LocalXML/ find 101 102 103 104 -name output.root | xargs hadd /opt/ppd/lhcb/user/place/to/save/merged-outputfile.root | |||||||
-- ThomasBird - 2010-08-04 |
How Merge root filesGangaThe easiest way is to let ganga handle it with with the RootMerger()j=Job(t) j.merger = RootMerger() j.merger.files = ['out-file.root']This worked fine for me for ages until the merging program this calls stayed at 100% forever and the python memory usage consumed everything available. So I had to merge the files manually... Merging manuallyThe program hadd is the one that the root merger calls to merge the fileshadd ~/outputfile.root ~/input1.root ~/input2.root [...]When running manually this segfaulted on my root file, it turned out to be because I had run SetupProject Ganga at the beginning by starting a new session and doing SetupProject DaVinci the hadd program no longer segfaulted. -- ThomasBird - 2010-08-04 |