Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

• SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
• Log in from RALPPD Linux with:
  • gsissh -p 1975 cmssgm@heplnv144.pp.rl.ac.uk
• Installation is in /opt/ppd/cms/phedex
  • Agent configuration is in <PhEDExInstance>_<SiteName> under that directory
• Stop all agents for a site:
  • From /opt/ppd/cms/phedex
  • ./stop_phedex <PhEDExInstance>_<SiteName>
• Start all agents for a site:
  • From /opt/ppd/cms/phedex
  • ./start_phedex <PhEDExInstance>_<SiteName>
• Start/Stop a single agent
  • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/<PhEDExInstance>_<SiteName>/Config <AgentName>

Proxy Management

• On a RALPP Linux UI:
  • Upload long lived proxy to RAL MyProxy
    • myproxy-init -n -d -c 672
  • Get cmsphedex role Proxy
    • voms-proxy-init --voms cms:/cms/Role=cmsphedex
• Login on the PhEDEx box
  • gsissh -p 1975 cmssgm@heplnv144.pp.rl.ac.uk
  • Register a new proxy
    • vobox-proxy --vo cms --proxy-safe 3600 --myproxy-safe 86400 --email l.kreczko@bristol.ac.uk register
  • List available proxies
    • vobox-proxy --vo cms -dn all query

PhEDEx Checks

Daily Checks

• Prod Agents up
• Debug Agents up
• Errors on Debug Transfers
• Errors on Prod Transfers
• Debug Quality Maps RALPP Bristol
• Prod Quality Maps RALPP Bristol
• Routing Page
• Prod Transfers
• Debug Transfers

Occasional Checks

Block Download Verify

• Log onto the PhEDEx box
• cd /opt/ppd/cms/phedex
• . PHEDEX/etc/profile.d/init.sh
• PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --test size
• PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
• grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

• cd /pnfs/pp.rl.ac.uk/data/cms/store/
• find mc data generator results hidata himc lumi relval -type f -mtime +30 -print > /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
• cd /opt/ppd/cms/phedex
• . PHEDEX/etc/profile.d/init.sh
• sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
• PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out

Other Checks

• SAM Test Latest History
• Job Robot
• GSTAT Bristol
• RALLCG2 FTS Monitoring Pages
• FTS Monitor

-- ChrisBrew - 2010-10-27