(r2) ThomasBirdSandbox < Sandbox

Tags: view all tags
-- Main.ThomasBird - 2010-07-19

---+ Data Analysis

---++ Creating an ntuple

---+++ Real Data

to create an ntuple with real data you need
   * the DST files you wish to run on
   * a python file pointing to the data
   * the davinci python file

My python file pointing to my data looks like:
<verbatim>from Gaudi.Configuration import *

EventSelector().Input   = [
#  for i in /home/hep/uoh35620/stuff/data/data_2010_up/* ; do echo "\"   DATAFILE='pfn:${i}' TYP='POOL_ROOTTREE' OPT='READ'\"," ; done
"   DATAFILE='pfn:/home/hep/uoh35620/stuff/data/data_2010_up/00007054_00000001_1.bhadron.dst' TYP='POOL_ROOTTREE' OPT='READ'",
"   DATAFILE='pfn:/home/hep/uoh35620/stuff/data/data_2010_up/00007054_00000002_1.bhadron.dst' TYP='POOL_ROOTTREE' OPT='READ'",
# ...
]</verbatim>

<br /> and can be mostly generated with the for command commented in the file.

<strong>Be careful </strong>when creating this file as <strong>symlinks </strong>have caused me issues and prevented DaVinci from reading the data. Since the files existed it caused no errors and it seemed to just skip the file it was looking at resulting in<strong> 0 events processed.</strong>

the davinci file needed to create the ntuple is as follows
<verbatim>
import GaudiPython
from Gaudi.Configuration import *
from GaudiPython.Bindings import gbl, AppMgr, Helper
from Configurables import GaudiSequencer, HltSelReportsMaker, OutputStream, DaVinci
from Configurables import PrintDecayTree
from os import environ
from Configurables import DecayTreeTuple, EventTuple
from Configurables import CombineParticles, FilterDesktop
from Configurables import TupleToolMCBackgroundInfo, TupleToolMCTruth, TupleToolTrigger, TupleToolTISTOS, TupleToolDecay, BackgroundCategory
from StrippingConf.StrippingLine import StrippingLine



DV = DaVinci()
DV.EvtMax = -1
DV.PrintFreq = 100
DV.SkipEvents = 0
DV.DataType = "2010"
DV.Simulation = True

DV.L0 = False #want this to be False for data; True will overwrite decisions. the same applies to the next few settings
DV.Hlt = False
DV.ReplaceL0BanksWithEmulated = False #reruns the L0. need more info
#DV.HltThresholdSettings = 'Physics_25Vis_25L0_2Hlt1_2Hlt2_May10' #only makes sense if re-running trigger

ApplicationMgr().ExtSvc +=  [ "NTupleSvc" ]
NTupleSvc().Output =  [ "FILE1 DATAFILE='/home/hep/uoh35620/stuff/batch-generate/data/up/all/B2Dpi_Up_6_7_2010.root' TYP='ROOT' OPT='NEW'" ]
#NTupleSvc().Output =  [ "FILE1 DATAFILE='castor:/castor/cern.ch/user/j/jnardull/RealDataRootFiles/B2DX_UP_FromHadron_6_7_2010.root' TYP='ROOT' OPT='NEW'" ]

from Configurables import CondDB
CondDB().IgnoreHeartBeat = True

#stripping stream location
location = '/Event/Bhadron/Phys/B2DXStrippingSelChi2Loose'

from PhysSelPython.Wrappers import AutomaticData, Selection, SelectionSequence
# Get the Candidates from the DST.
# Treat particles already on the DST as data-on-demand, but give the full path.
B2DXSel = AutomaticData(Location = location)
# Filter the Candidate. Replace 'ALL' by some cuts.

B2DXFilter = FilterDesktop('B2DXFilter')
B2DXFilter.Code = "ALL" #replace this if wanted

# make a Selection
B2DXFilterSel = Selection(name = 'B2DXFilterSel',
                          Algorithm = B2DXFilter,
                          RequiredSelections = [ B2DXSel])

# build the SelectionSequence
B2DXSeq = SelectionSequence('B2DXSeq', TopSelection = B2DXFilterSel)
DV.appendToMainSequence( [ B2DXSeq.sequence() ] )

########################################################################
# DecayTreeTuples
########################################################################

#
# Initialisation
#

TupleSequence = GaudiSequencer ("TupleSequencer")

AnalysisDecayTreeTuple = DecayTreeTuple("AnalysisDecayTreeTuple")
BachelorDecayTreeTuple = DecayTreeTuple("BachelorDecayTreeTuple")
KstarDecayTreeTuple = DecayTreeTuple("KstarDecayTreeTuple")

BachelorOrKstarDecayTreeTuple = GaudiSequencer("BachelorOrKstarDecayTreeTuple")
BachelorOrKstarDecayTreeTuple.ModeOR = 1
BachelorOrKstarDecayTreeTuple.Members = [BachelorDecayTreeTuple,KstarDecayTreeTuple]

TupleSequence.Members += [AnalysisDecayTreeTuple]
TupleSequence.Members += [BachelorOrKstarDecayTreeTuple]


#Plots
AnalysisDecayTreeTuple.InputLocations = ["B2DXFilterSel"]
AnalysisDecayTreeTuple.Decay = "[[B0]cc ->  (^D~0 => {^K+ ^pi-, ^K- ^pi+,^K+ ^K-,^pi+ ^pi-}) ? ]cc"
AnalysisDecayTreeTuple.Branches = {
    "D" : "[[B0]cc ->  (^D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-}) {K+, pi+}]cc"
    ,"Positive_D_daughter" : "[[B0]cc ->  (D~0 => {^K+ pi-, K- ^pi+, ^K+ K-, ^pi+ pi-, ^K+ ^pi+,K- pi-}) ? ]cc"
    ,"Negative_D_daughter" : "[[B0]cc ->  (D~0 => {K+ ^pi-, ^K- pi+, K+ ^K-, pi+ ^pi-, K+ pi+,^K- ^pi-}) ? ]cc"
    ,"B" : "[B0]cc : [[B0]cc ->  (D~0 => {K+ pi-, K- pi+,K+ K-,pi+ pi-, K+ pi+, K- pi-}) ? ]cc"
    }

BachelorDecayTreeTuple = DecayTreeTuple("BachelorDecayTreeTuple")
BachelorDecayTreeTuple.InputLocations = ["B2DXFilterSel"]
BachelorDecayTreeTuple.Decay = "[[B+]cc ->  (^D~0 => {^K+ ^pi-, ^K- ^pi+,^K+ ^K-,^pi+ ^pi-,^K+ ^pi+,^K- ^pi-}) {^K+, ^pi+}]cc"
BachelorDecayTreeTuple.Branches = {
    "D" : "[[B+]cc ->  (^D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) {K+, pi+}]cc"
    ,"Positive_D_Daughter" : "[[B+]cc ->  (D~0 => {^K+ pi-, K- ^pi+, ^K+ K-, ^pi+ pi-,^K+ ^pi+,K- pi-}) {K+, pi+}]cc"
    ,"Negative_D_Daughter" : "[[B+]cc ->  (D~0 => {K+ ^pi-, ^K- pi+, K+ ^K-, pi+ ^pi-,K+ pi+,^K- ^pi-}) {K+, pi+}]cc"
    ,"Bachelor" : "[[B+]cc ->  (D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) {^K+, ^pi+}]cc"
    ,"B" : "[B+]cc : [[B+]cc ->  (D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) {K+, pi+}]cc"
    }

KstarDecayTreeTuple = DecayTreeTuple("KstarDecayTreeTuple")
KstarDecayTreeTuple.InputLocations = ["B2DXFilterSel"]
KstarDecayTreeTuple.Decay = "[[B0]cc -> (^D~0 => {^K+ ^pi-, ^K- ^pi+,^K+ ^K-,^pi+ ^pi-,^K+ ^pi+,^K- ^pi-}) (^K*(892)0 => ^K+ ^pi-) ]cc"
KstarDecayTreeTuple.Branches = {
    "KStar" : "[[B0]cc ->  (D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) (^K*(892)0 => K+ pi-) ]cc"
    ,"D" : "[[B0]cc ->  (^D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) (K*(892)0 => K+ pi-) ]cc"
    ,"Positive_D_Daughter" : "[[B0]cc ->  (D~0 => {^K+ pi-, K- ^pi+, ^K+ K-, ^pi+ pi-,^K+ ^pi+,K- pi-}) (K*(892)0 => K+ pi-) ]cc"
    ,"Negative_D_Daughter" : "[[B0]cc ->  (D~0 => {K+ ^pi-, ^K- pi+, K+ ^K-, pi+ ^pi-,K+ pi+,^K- ^pi-}) (K*(892)0 => K+ pi-) ]cc"
    ,"K_from_kstar" : "[[B0]cc ->  (D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) (K*(892)0 => ^K+ pi-) ]cc"
    ,"Pi_from_kstar" : "[[B0]cc ->  (D~0 => {K+ pi-, K- pi+, K+ K-, pi+ pi-,K+ pi+,K- pi-}) (K*(892)0 => K+ ^pi-) ]cc"
    ,"B" : "[B0]cc : [[B0]cc ->  (D~0 => {K+ pi-, K- pi+,K+ K-,pi+ pi-}) (K*(892)0 => K+ pi-) ]cc"
    }


KstarDecayTreeTuple.ToolList +=  [
    "TupleToolGeometry"
    , "TupleToolAngles"
    , "TupleToolKinematic"
    , "TupleToolPrimaries"
    , "TupleToolPid"
    , "TupleToolEventInfo"
    , "TupleToolTrackInfo"
     ]


BachelorDecayTreeTuple.ToolList +=  [
    "TupleToolGeometry"
    , "TupleToolAngles"
    , "TupleToolKinematic"
    , "TupleToolPrimaries"
    , "TupleToolPid"
    , "TupleToolEventInfo"
    , "TupleToolTrackInfo"
    ]

AnalysisDecayTreeTuple.ToolList +=  [
    "TupleToolGeometry"
    , "TupleToolAngles"
    , "TupleToolKinematic"
    , "TupleToolPrimaries"
    , "TupleToolPid"
    , "TupleToolEventInfo"
    , "TupleToolTrackInfo"
    , "LoKi::Hybrid::TupleTool/LoKi_All" 
    ]

########################################################################
# Add sequence to DaVinci
########################################################################

DV.UserAlgorithms += [TupleSequence]</verbatim>

Then all this can be run with the following command.
<verbatim>SetupProject DaVinci
gaudirun.py DaVinci.py data.py | tee output.txt</verbatim>

the output will be copied to the output.txt file.

---++++ Running on ppd batch

you can run these taks on the ppd batch system, since doing a large number of events can take many hours. below is a script I use to run these taks.

<verbatim>#!/bin/bash

stda="`date`"
echo $stda

. /afs/rl.ac.uk/lhcb/lhcb/LBSCRIPTS/LBSCRIPTS_v5r2/InstallArea/scripts/SetupProject.sh DaVinci

dir=/home/hep/uoh35620/stuff/batch-generate/data/up/all

gaudirun.py ${dir}/DaVinciAndCandidatesUp.py ${dir}/6_7_2010_up.py  | tee ${dir}/output.txt | tee /home/hep/uoh35620/stuff/batch-generate/test.txt

echo
echo $stda
date
echo
echo Done.</verbatim>

The dot at the beginning of the line ending in SetupProject.sh DaVinci is very important. If this is not there the paths will not be set up properly for your script.

Placing this in a file called say script and running the following will make it run on the batch system.

<verbatim>qsub -q prod -S /bin/bash -j oe -o /home/hep/uoh35620/job-out/ script</verbatim>

-q chooses the queue you wish to submit the job to while -j oe joins the stderr and stdout together into one file. -o puts the output file into a given directory and script is the above script to run the job.
---+++++ Monitoring

The following command will list your currently running jobs. you will need to replace uoh35620 with your user name.

<verbatim>qstat -u uoh35620</verbatim>

For looking at jobs in real time I like to use the following, where the file is the output of one of the tee's in the script and uoh35620 will need to be changed to your user name.

<verbatim>watch -n 10 'tail -n 20 /home/hep/uoh35620/stuff/batch-generate/test.txt ; echo ; echo ; qstat -u uoh35620'</verbatim>

you might want to make alias of these commands in your ~/.bashrc file so the following in ~/.bashrc will make a new command called qstatme which will be like typing the full command.

<verbatim>alias qstatme="qstat -u uoh35620"</verbatim>

---+++++ Errors

I have gotten this email a few times and I'm not really sure of the cause, maybe an error on the bathch macine, but anyway, if you get this just keep re-submitting exactly the same job until it works. The most I've ever had to do it is about 5 times.

<verbatim>PBS Job Id: 4121274.heplnx208.pp.rl.ac.uk
Job Name:   script
Exec host:  heplnc308.pp.rl.ac.uk/0
An error has occurred processing your job, see below.
Post job file processing error; job 4121274.heplnx208.pp.rl.ac.uk on host heplnc308.pp.rl.ac.uk/0

Unable to copy file /var/spool/pbs/spool/4121274.heplnx208.pp.rl.ac.uk.OU to /home/hep/uoh35620/job-out/
>>> error from copy
/bin/cp: cannot stat `/var/spool/pbs/spool/4121274.heplnx208.pp.rl.ac.uk.OU': No such file or directory
>>> end error output</verbatim>

---+++ MC Data

Creating the ntuple with monte carlo data is harder than with the actual data since you must first strip it before creating the ntuple, but can be done with a simalar set up as before. Again you will need
   * the DST files you wish to run on
   * a python file pointing to the data
   * the davinci python file

---++ Analysing an ntuple
Topic revision: r2 - 2010-07-19 - ThomasBird