SouthGridPhEDExNotes < CMS

---+ Notes on runnning PhEDEx for SouthGrid Sites

Main [[https://twiki.cern.ch/twiki/bin/view/CMS/PhEDEx][CMS PhEDEx TWiki]] Pages
   * SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
   * Log in from RALPPD Linux with: 
      * gsissh -p 1975 cmssgm@heplnv144.pp.rl.ac.uk
   * Installation is in /opt/ppd/cms/phedex 
      * Agent configuration is in &lt;PhEDExInstance&gt;_&lt;SiteName&gt; under that directory
   * Stop all agents for a site: 
      * From /opt/ppd/cms/phedex
      * ./stop_phedex &lt;PhEDExInstance&gt;_&lt;SiteName&gt;
   * Start all agents for a site: 
      * From /opt/ppd/cms/phedex
      * ./start_phedex &lt;PhEDExInstance&gt;_&lt;SiteName&gt;
   * Start/Stop a single agent 
      * /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/&lt;PhEDExInstance&gt;_&lt;SiteName&gt;/Config &lt;AgentName&gt;

---++ Proxy Management
   * On a RALPP Linux UI: 
      * Upload long lived proxy to RAL MyProxy 
         * myproxy-init -n -d -c 672
      * Get cmsphedex role Proxy 
         * voms-proxy-init --voms cms:/cms/Role=cmsphedex
   * Login on the PhEDEx box 
      * gsissh -p 1975 cmssgm@heplnv144.pp.rl.ac.uk
      * Register a new proxy 
         * vobox-proxy --vo cms --proxy-safe 3600 --myproxy-safe 86400 --email l.kreczko@bristol.ac.uk register
      * List available proxies 
         * vobox-proxy --vo cms -dn all query

---++ PhEDEx Checks
---+++ Daily Checks
   * [[https://cmsweb.cern.ch/phedex/prod/Components::Status][Prod Agents]] up
   * [[https://cmsweb.cern.ch/phedex/debug/Components::Status][Debug Agents]] up
   * Errors on [[https://cmsweb.cern.ch/phedex/debug/Activity::ErrorInfo?tofilter=T*_UK_SGrid*&fromfilter=T*_UK_SGrid*&report_code=.*&xfer_code=.*&to_pfn=.*&from_pfn=.*&log_detail=.*&log_validate=.*&.submit=Update#][Debug Transfers]]
   * Errors on [[https://cmsweb.cern.ch/phedex/prod/Activity::ErrorInfo?tofilter=T*_UK_SGrid*&fromfilter=T*_UK_SGrid*&report_code=.*&xfer_code=.*&to_pfn=.*&from_pfn=.*&log_detail=.*&log_validate=.*&.submit=Update#][Prod Transfers]]
   * Debug Quality Maps [[https://cmsweb.cern.ch/phedex/debug/Activity::QualityPlots?graph=quality_all&entity=src&src_filter=&dest_filter=T2_UK_SGrid_RALPP&no_mss=true&period=l96h&upto=][RALPP]] [[https://cmsweb.cern.ch/phedex/debug/Activity::QualityPlots?graph=quality_all&entity=src&src_filter=&dest_filter=T2_UK_SGrid_Bristol&no_mss=true&period=l96h&upto=&.submit=Update][Bristol]]
   * Prod Quality Maps [[https://cmsweb.cern.ch/phedex/prod/Activity::QualityPlots?graph=quality_all&entity=src&src_filter=&dest_filter=T2_UK_SGrid_RALPP&no_mss=true&period=l96h&upto=][RALPP]] [[https://cmsweb.cern.ch/phedex/prod/Activity::QualityPlots?graph=quality_all&entity=src&src_filter=&dest_filter=T2_UK_SGrid_Bristol&no_mss=true&period=l96h&upto=&.submit=Update][Bristol]]
   * [[https://cmsweb.cern.ch/phedex/prod/Activity::Routing][Routing]] Page
   * [[https://cmsweb.cern.ch/phedex/prod/Activity::TransferDetails?tofilter=T*_UK_SGrid&andor=or&fromfilter=T*_UK_SGrid][Prod Transfers]]
   * [[https://cmsweb.cern.ch/phedex/debug/Activity::TransferDetails?tofilter=T*_UK_SGrid&andor=or&fromfilter=T*_UK_SGrid][Debug Transfers]]

---+++ Occasional Checks
---++++ Block Download Verify
   * Log onto the PhEDEx box
   * <code>cd /opt/ppd/cms/phedex</code>
   * <code>. PHEDEX/etc/profile.d/init.sh</code>
   * <code>PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP  --test size</code>
   * <code>PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --node T2_UK_SGrid_RALPP --detail --days 7 &gt; Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt</code>
   * <code>grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort</code>

---++++ Storage Consistency Check
   * <code>cd /pnfs/pp.rl.ac.uk/data/cms/store/</code>
   * <code>find mc data generator results hidata himc lumi relval -type f -mtime +30 -print &gt; /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt</code>
   * <code> cd /opt/ppd/cms/phedex</code>
   * <code>. PHEDEX/etc/profile.d/init.sh</code>
   * <code> sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt</code>
   * <code>PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out</code>

---++ Other Checks

   * SAM Test [[http://lxarda16.cern.ch/dashboard/request.py/latestresultssmry?siteSelect3=All Sites&serviceTypeSelect3=vo&sites=T2_UK_SGrid_Bristol&sites=T2_UK_SGrid_RALPP&sites=T3_UK_SGrid_Oxford&services=CE&services=SRMv2&tests=1301&tests=133&tests=111&tests=6&tests=1261&tests=76&tests=64&tests=20&tests=281&tests=882&tests=1321&exitStatus=all][Latest]] [[http://lxarda16.cern.ch/dashboard/request.py/historicalsiteavailability?siteSelect3=T3T2T1&sites=T2_UK_SGrid_Bristol&sites=T2_UK_SGrid_RALPP&sites=T3_UK_SGrid_Oxford&timeRange=last48][History]]
   * [[http://jobrobot.web.cern.ch/JobRobot/][Job Robot]]
   * [[http://gstat-prod.cern.ch/gstat/site/UKI-SOUTHGRID-BRIS-HEP/][GSTAT Bristol]]
   * [[http://lcgwww.gridpp.rl.ac.uk/cgi-bin/fts-mon/fts-mon.pl][RALLCG2 FTS Monitoring Pages]]
   * [[http://lcgwww.gridpp.rl.ac.uk/ftsmonitor/ftsmonitor.php][FTS Monitor]]

-- Main.ChrisBrew - 2010-10-27
This topic: CMS > WebHome > Documentation > SouthGridPhEDExNotes
Topic revision: r18 - 2018-01-26 - ChrisBrew