Tags:
view all tags
---+ Data Storage This page describes where to store your data in order to run over it. In general all data files should be stored on dCache. A small amount of data can be stored on the local cluster for development purposes. afs space can be used to put output files/code that you may wish to share with others. ---+ On the local cluster For small to medium amounts of data it is recommended that you store it in either: <verbatim> /opt/ppd/scratch /opt/ppd/month </verbatim> Each has a capacity of 1.6 TB. ---+ On AFS There is an /afs cell at RAL which you can find at: <verbatim> /afs/rl.ac.uk/ </verbatim> ---+ On the dCache storage element locally Detail instructions on using the dCache element can be found on the [[https://twiki.pp.rl.ac.uk/twiki/bin/view/Computing/Documentation/DCacheStorageElement][DCacheStorageElement]] page. To make things easier to manage please copy ATLAS datasets into: <verbatim> /pnfs/pp.rl.ac.uk/data/atlas/atlasralppdisk/ </verbatim> You can use dq2-get to copy files directly into the dCache space. To do this log into heplnx109.gridpp.rl.ac.uk and do: <verbatim> cd $HOME source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh voms-proxy-init -voms atlas dq2-get -S srm://heplnx204.pp.rl.ac.uk:8443/srm/managerv2?SFN=/pnfs/pp.rl.ac.uk/data/atlas/atlasralppdisk/dewhurst user10.janstrube.ganga.data10_7TeV.00152221.physics_L1Calo.merge.AOD.r1239_p134.D3PD_v1.99.0 </verbatim> where you can replace dewhurst with your name and user10.jan with the file you wish to download from the grid. Everybody has read write permission so be careful. Note: For some reason if you try this command while somewhere inside the /pnfs/pp.rl.ac.uk/ directory structure you will get an error saying that the file system is read only. ---+ On the dCache storage element on the grid The advantage of storing data on the grid is that there is alot more avaliable storage space. By Storing data on the grid at RAL local users will be able to run over the data interactively for developing software as well as looking at smaller samples as well as running over the same data as a grid job. The dis-advantage is that you don't have full control over your data and some things can take time. If you wish to copy either ATLAS data that is already on the grid to RAL you can download a small amount using DQ2-get. If you want to run over a larger sample you can request data replication via [[http://panda.cern.ch:25980/server/pandamon/query?mode=ddm_req][Data Replication]] <verbatim> dq2-put -L SITENAME -s SOURCEDIRECTORY DATASETNAME </verbatim> The SITENAME is: UKI-SOUTHGRID-RALPP_SCRATCHDISK The DATASETNAME needs to follow the [[https://twiki.cern.ch/twiki/bin/view/Atlas/DQ2ClientsHowTo#DatasetNameRules][Dataset naming rules.]] -- Main.AlastairDewhurst - 2010-01-29
Edit
|
Attach
|
Watch
|
P
rint version
|
H
istory
:
r10
|
r7
<
r6
<
r5
<
r4
|
B
acklinks
|
V
iew topic
|
Raw edit
|
More topic actions...
Topic revision: r5 - 2010-09-07
-
TimAdye
Home
Site map
Atlas web
CMS web
Computing web
Documentation web
PPDITForum web
LHCb web
Main web
PPDLabSpace web
Sandbox web
TWiki web
Atlas Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
Register User
Edit
Attach
Copyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback