Data sharing at UChicago: Work between UChicago and CERN
Access to CERN-EOS from UChicago
The ways to list, write and read files on CERN EOS, documented here, still work at UChicago.
On the Analysis Facility login servers, try the following:
export LXPLUS_USER=<your lxplus username>
setup-eos start
this makes a directory called $HOME/your_uchicago_username/eos and mount EOS there.
To list your files located at /eos/user/d/dschrute/ :
eos root://eosuser.cern.ch ls /eos/user/d/dschrute
# remember to replace "d/dschrute" with your own username at CERN.
To create a new directory at /eos/user/d/dschrute/ :
eos root://eosuser.cern.ch mkdir /eos/user/d/dschrute/new_dir
# remember to replace "d/dschrute" with your own username at CERN.
To copy files from /eos/user/d/dschrute/ to your current directory:
xrdcp root://eosuser.cern.ch//eos/user/d/dschrute/file.txt .
# use `xrdcp -r` if you need to copy a directory
To copy files from /eos/user/d/dschrute/ to any path within your profile
xrdcp root://eosuser.cern.ch//eos/user/d/dschrute/file.txt /any-path/ # /any-path/ eg: /home/dschrute/codingx/
# use `xrdcp -r` if you need to copy a directory
Xcache at UChicago
UChicago Analysis Facility maintains an XCache server (managed through SLATE), with 25 x 1.5 TB NVMes and 2x25 Gbps NIC.
ServiceX uses the XCache by default.
Users that want do access remote data of their own on EOS or elsewhere, can manually add the prefix root://192.170.240.18:1094// to their root paths and the prefix :1094 to ther server address, eg:
If the original path is eg.:
root://someserver.org//atlaslocalgroupdisk/rucio/user/mgeyik/63/c4/user.mgeyik.26617246._000006.out.root
root://192.170.240.18:1094//root://someserver.org:1094//atlaslocalgroupdisk/rucio/user/mgeyik/63/c4/user.mgeyik.26617246._000006.out.root
Example:
# Original path
root://eosatlas.cern.ch//eos/atlas/atlastier0/rucio/data_13TeV/physics1/data_13TeV.004345.physics_Main.eaq./data_13TeV.004345.physics_Main.eaq_0001.root’
# make it:
root://192.170.240.18:1094//root://eosatlas.cern.ch:1094//eos/atlas/atlastier0/rucio/data_13TeV/physics1/data_13TeV.004345.physics_Main.eaq./data_13TeV.004345.physics_Main.eaq_0001.root
CERN Kerberos ticket
You can obtain and cache a CERN Kerberos ticket (this is also required for the way of using ssh-tunnel below) by doing:
Please notice that the domain CERN.CH must be in UPPERCASE.
Pass your CERN Kerberos ticket to your HTCondor jobs
Before running the previous command kinit <name_at_CERN>@CERN.CH pass your CERN Kerberos ticket to the batch machines:
Then add the environment variable KRB5CCNAME to your condor batch job,eg:
...
environment = <parameter_list>
...
x509userproxy = $ENV(X509_USER_PROXY) # this adds KRB5CCNAME to your condor batch job.
Always check your proxy certificate
If you are trying to use HTCondor driver to run some jobs that need data access authorization, for example on rucio. In any of the ways described to access your data always check the status of your proxy grid certificate.
Your x509proxy certificate has expiry date, once it expires you have to create an ATLAS VOMS proxy again in the usual way. As you know you create (or copy it to) on the shared $HOME filesystem so that the HTCondor scheduler can find and read the proxy. This is how you copy it to $HOME:
-out $HOME/x509proxy you create a new proxy but the one that maybe is already in your $HOME directory is still expired.
Once you renew you proxy certificate, add line1 and line2 from the following example to your job submit file so that HTCondor configures the job environment automaticcaly for x509 authenticated data access.