Difference: SouthGridPhEDExNotes (1 vs. 18)

Revision 182018-01-26 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

  • SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
  • Log in from RALPPD Linux with:
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in <PhEDExInstance>_<SiteName> under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex <PhEDExInstance>_<SiteName>
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex <PhEDExInstance>_<SiteName>
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/<PhEDExInstance>_<SiteName>/Config <AgentName>

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

PhEDEx Checks

Daily Checks

Occasional Checks

Block Download Verify

  • Log onto the PhEDEx box
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
Changed:
<
<
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/Prod/T2UKSGRIDRALPP --test size
>
>
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --test size
 
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results hidata himc lumi relval -type f -mtime +30 -print > /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out

Other Checks

-- ChrisBrew - 2010-10-27

Revision 172017-03-27 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Changed:
<
<
    • Agent configuration is in _ under that directory
>
>
    • Agent configuration is in <PhEDExInstance>_<SiteName> under that directory
 
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
Changed:
<
<
    • ./stop_phedex _
>
>
    • ./stop_phedex <PhEDExInstance>_<SiteName>
 
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
Changed:
<
<
    • ./start_phedex _
>
>
    • ./start_phedex <PhEDExInstance>_<SiteName>
 
  • Start/Stop a single agent
Changed:
<
<
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config
>
>
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/<PhEDExInstance>_<SiteName>/Config <AgentName>
 

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

PhEDEx Checks

Daily Checks

Occasional Checks

Block Download Verify

  • Log onto the PhEDEx box
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/Prod/T2UKSGRIDRALPP --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results hidata himc lumi relval -type f -mtime +30 -print > /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out

Other Checks

-- ChrisBrew - 2010-10-27

Revision 162014-02-25 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

  • SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
  • Log in from RALPPD Linux with:
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in _ under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

PhEDEx Checks

Daily Checks

Occasional Checks

Block Download Verify

  • Log onto the PhEDEx box
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/Prod/T2UKSGRIDRALPP --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results hidata himc lumi relval -type f -mtime +30 -print > /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • cd /opt/ppd/cms/phedex
Added:
>
>
  • . PHEDEX/etc/profile.d/init.sh
 
  • sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out

Other Checks

-- ChrisBrew - 2010-10-27

Revision 152014-02-25 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

  • SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
  • Log in from RALPPD Linux with:
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in _ under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

PhEDEx Checks

Daily Checks

Occasional Checks

Block Download Verify

Changed:
<
<
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort
>
>
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/Prod/T2UKSGRIDRALPP --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort
Added:
>
>
 

Storage Consistency Check

Changed:
<
<
  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results RelVal -type f -mtime +30 -print > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
>
>
  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results hidata himc lumi relval -type f -mtime +30 -print > /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
Added:
>
>
  • cd /opt/ppd/cms/phedex
 
  • sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
Changed:
<
<
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out
>
>
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/T2UKSGRIDRALPP -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out
 

Other Checks

-- ChrisBrew - 2010-10-27

Revision 142012-01-26 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

  • SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
  • Log in from RALPPD Linux with:
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in _ under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

PhEDEx Checks

Daily Checks

Occasional Checks

Block Download Verify

  • Log onto the PhEDEx box
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results RelVal -type f -mtime +30 -print > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
Changed:
<
<
  • sed -i -e 's#^\(.*\)#/store/\1#' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
>
>
  • sed -i -e 's|^\(.*\)|/store/\1|' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
 
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out

Other Checks

-- ChrisBrew - 2010-10-27

Revision 132012-01-26 - EmmanuelOlaiya

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

  • SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
  • Log in from RALPPD Linux with:
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in _ under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

PhEDEx Checks

Daily Checks

Occasional Checks

Block Download Verify

Changed:
<
<
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results RelVal -type f -mtime +30 -print > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • sed -i -e 's|^\(.*\)|/store/\1/' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out
>
>
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results RelVal -type f -mtime +30 -print > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • sed -i -e 's#^\(.*\)#/store/\1#' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out
 

Other Checks

-- ChrisBrew - 2010-10-27

Revision 122012-01-26 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

  • SouthGrid PhEDEx Agents run on heplnv144.pp.rl.ac.uk
  • Log in from RALPPD Linux with:
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in _ under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box
Changed:
<
<

Daily Checks

PhEDEx Checks

>
>

PhEDEx Checks

Daily Checks

 
Changed:
<
<

Other Checks

>
>
Added:
>
>

Occasional Checks

Block Download Verify

  • Log onto the PhEDEx box
  • cd /opt/ppd/cms/phedex
  • . PHEDEX/etc/profile.d/init.sh
  • PHEDEX/Utilities/BlockDownloadVerify-injector.pl --node T2_UK_SGrid_RALPP --block '%' --dbconfig /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --test size
  • PHEDEX/Utilities/BlockDownloadVerify-report.pl --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD --node T2_UK_SGrid_RALPP --detail --days 7 > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt
  • grep Fail Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/DownloadVerify-$(date +%Y-%m-%d)-report.txt | grep LFN= | cut -d ' ' -f 3 | sort

Storage Consistency Check

  • cd /pnfs/pp.rl.ac.uk/data/cms/store/
  • find mc data generator results RelVal -type f -mtime +30 -print > Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • sed -i -e 's|^\(.*\)|/store/\1/' Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt
  • PHEDEX/Utilities/StorageConsistencyCheck --db /opt/ppd/cms/phedex/Prod_T2_UK_SGrid_RALPP/DBParam:Prod/RUTHERFORDPPD -se_name heplnx204.pp.rl.ac.uk -lfnlist Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/file_list-$(date +%Y-%m-%d).txt | tee Prod_T2_UK_SGrid_RALPP/ConsistencyCheck/StorageConsistencyCheck-$(date +%Y-%m-%d).out

Other Checks

 

-- ChrisBrew - 2010-10-27

Revision 112012-01-19 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Changed:
<
<
  • Installation is in /opt/ppd/cms/phedex
>
>
  • Installation is in /opt/ppd/cms/phedex
 
    • Agent configuration is in _ under that directory
Changed:
<
<
  • Stop all agents for a site:
>
>
  • Stop all agents for a site:
 
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
Changed:
<
<
  • Start all agents for a site:
>
>
  • Start all agents for a site:
 
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
Changed:
<
<
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config <start|stop>
>
>
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config
 

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

Daily Checks

PhEDEx Checks

Other Checks

-- ChrisBrew - 2010-10-27

Revision 102012-01-18 - ChrisBrew

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Added:
>
>
  • Installation is in /opt/ppd/cms/phedex
    • Agent configuration is in _ under that directory
  • Stop all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./stop_phedex _
  • Start all agents for a site:
    • From /opt/ppd/cms/phedex
    • ./start_phedex _
  • Start/Stop a single agent
    • /opt/ppd/cms/phedex/PHEDEX/Utilities/Master -config /opt/ppd/cms/phedex/_/Config <start|stop>
 

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

Daily Checks

PhEDEx Checks

Other Checks

-- ChrisBrew - 2010-10-27

Revision 92011-04-26 - LukaszKrezcko

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

Daily Checks

PhEDEx Checks

Changed:
<
<
>
>
 
Changed:
<
<
>
>
Added:
>
>
 

Other Checks

-- ChrisBrew - 2010-10-27

Revision 82011-04-05 - LukaszKrezcko

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

Daily Checks

PhEDEx Checks

Changed:
<
<
>
>
 

Other Checks

-- ChrisBrew - 2010-10-27

Revision 72011-03-01 - LukaszKrezcko

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

Daily Checks

PhEDEx Checks

Other Checks

Added:
>
>
  -- ChrisBrew - 2010-10-27

Revision 62010-11-15 - LukaszKrezcko

 
META TOPICPARENT name="Documentation"

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box

Daily Checks

PhEDEx Checks

Changed:
<
<
  • Debug Quality Maps RALPP
  • Prod Quality Maps RALPP
>
>
 

Other Checks

-- ChrisBrew - 2010-10-27

Revision 52010-10-29 - ChrisBrew

 
META TOPICPARENT name="Documentation"
Added:
>
>

Notes on runnning PhEDEx for SouthGrid Sites

Main CMS PhEDEx TWiki Pages

 
Changed:
<
<
>
>
  • Log in from RALPPD Linux with:
Added:
>
>
 
Changed:
<
<		+--- Proxy Management
  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box
vobox-proxy --vo cms --proxy-safe 3600 --myproxy-safe 86400 --email l.kreczko@bristol.ac.uk register
>
>

Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box
Deleted:
<
<
    • List available proxies
vobox-proxy --vo cms -dn all query
 
Changed:
<
<		+--- Daily Checks http://lxarda16.cern.ch/dashboard/request.py/latestresultssmry?siteSelect3=All%20Sites&serviceTypeSelect3=vo&sites=T1_UK_RAL&sites=T2_UK_London_Brunel&sites=T2_UK_London_IC&sites=T2_UK_SGrid_Bristol&sites=T2_UK_SGrid_RALPP&sites=T3_UK_London_QMUL&sites=T3_UK_London_RHUL&services=CE&services=SRMv2&tests=1301&tests=133&tests=111&tests=6&tests=1261&tests=76&tests=64&tests=20&tests=281&tests=882&exitStatus=all http://lxarda16.cern.ch/dashboard/request.py/historicalsiteavailability?siteSelect3=T3T2T1&sites=T1_UK_RAL&sites=T2_UK_London_Brunel&sites=T2_UK_London_IC&sites=T2_UK_SGrid_Bristol&sites=T2_UK_SGrid_RALPP&sites=T3_UK_London_QMUL&sites=T3_UK_London_RHUL&sites=T3_UK_London_UCL&sites=T3_UK_ScotGrid_ECDF&sites=T3_UK_ScotGrid_GLA&sites=T3_UK_SGrid_Oxford&timeRange=last48 http://jobrobot.web.cern.ch/JobRobot/
>
>

Daily Checks

PhEDEx Checks

Added:
>
>

Other Checks

  -- ChrisBrew - 2010-10-27

Revision 42010-10-27 - ChrisBrew

 
META TOPICPARENT name="Documentation"

+--- Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box
vobox-proxy --vo cms --proxy-safe 3600 --myproxy-safe 86400 --email l.kreczko@bristol.ac.uk register
    • List available proxies
vobox-proxy --vo cms -dn all query

+--- Daily Checks

Added:
>
>		http://lxarda16.cern.ch/dashboard/request.py/latestresultssmry?siteSelect3=All%20Sites&serviceTypeSelect3=vo&sites=T1_UK_RAL&sites=T2_UK_London_Brunel&sites=T2_UK_London_IC&sites=T2_UK_SGrid_Bristol&sites=T2_UK_SGrid_RALPP&sites=T3_UK_London_QMUL&sites=T3_UK_London_RHUL&services=CE&services=SRMv2&tests=1301&tests=133&tests=111&tests=6&tests=1261&tests=76&tests=64&tests=20&tests=281&tests=882&exitStatus=all http://lxarda16.cern.ch/dashboard/request.py/historicalsiteavailability?siteSelect3=T3T2T1&sites=T1_UK_RAL&sites=T2_UK_London_Brunel&sites=T2_UK_London_IC&sites=T2_UK_SGrid_Bristol&sites=T2_UK_SGrid_RALPP&sites=T3_UK_London_QMUL&sites=T3_UK_London_RHUL&sites=T3_UK_London_UCL&sites=T3_UK_ScotGrid_ECDF&sites=T3_UK_ScotGrid_GLA&sites=T3_UK_SGrid_Oxford&timeRange=last48 http://jobrobot.web.cern.ch/JobRobot/
  -- ChrisBrew - 2010-10-27

Revision 32010-10-27 - LukaszKrezcko

 
META TOPICPARENT name="Documentation"

+--- Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
  • Login on the PhEDEx box
vobox-proxy --vo cms --proxy-safe 3600 --myproxy-safe 86400 --email l.kreczko@bristol.ac.uk register
    • List available proxies
vobox-proxy --vo cms -dn all query
Added:
>
>		 +--- Daily Checks
  -- ChrisBrew - 2010-10-27

Revision 22010-10-27 - LukaszKrezcko

 
META TOPICPARENT name="Documentation"

+--- Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex
Added:
>
>		 vobox-proxy --vo cms --proxy-safe 3600 --myproxy-safe 86400 --email l.kreczko@bristol.ac.uk register
    • List available proxies
vobox-proxy --vo cms -dn all query
  -- ChrisBrew - 2010-10-27

Revision 12010-10-27 - ChrisBrew

 
META TOPICPARENT name="Documentation"

+--- Proxy Management

  • On a RALPP Linux UI:
    • Upload long lived proxy to RAL MyProxy
      • myproxy-init -n -d -c 672
    • Get cmsphedex role Proxy
      • voms-proxy-init --voms cms:/cms/Role=cmsphedex

-- ChrisBrew - 2010-10-27

View topic | History: r18 < r17 < r16 < r15 | More topic actions...
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback