(0) Authentication
mkdir .globus
copy usercert.pem and userkey.pem (sftp will do)
Get lcg environment: lcg bash
voms-proxy-init --debug --voms cms
voms-proxy-info -all suggests all is peachy.
(1) Data Access
Check my own storgae area on the SE:
lcg-ls --vo=cms
srm://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/user/dbauer
Copy a file to the storage and back:
env | sort > testfile.txt
lcg-cp --vo=cms testfile.txt
srm://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/user/dbauer/testfile_cloud.txt
[dbauer@cloud00vm02 ~]$ lcg-ls --vo=cms srm://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/user/dbauer/
/pnfs/hep.ph.ic.ac.uk/data/cms/store/user/dbauer/testfile_cloud.txt
lcg-cp --vo=cms srm://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/user/dbauer/testfile_cloud.txt testfile_cloud2.txt
[dbauer@cloud00vm02 ~]$ ls -l
total 36
drwxr-xr-x 15 dbauer dbauer 4096 Jun 23 15:11 CMSSW_4_1_5
-rw-rw-r-- 1 dbauer dbauer 14411 Jun 23 15:45 testfile.txt
-rw-rw-r-- 1 dbauer dbauer 14411 Jun 23 15:46 testfile_cloud2.txt
(2) Trying to run the tutorial
(still in 'lcg bash')
export SCRAM_ARCH=slc5_amd64_gcc434
. /cvmfs/cms.cern.ch/cmsset_default.sh
using the release area I made earlier ("scram project CMSSW CMSSW_4_1_5")
cd CMSSW_4_1_5
cmsenv
Just copy the first bit:
mkdir -p Tutorial/Exercise
cd Tutorial/Exercise
mkdir src
mkdir test
mkdir python
mkdir interface
cd test/
wget -O config_cfg.py http://www.hep.ph.ic.ac.uk/~dbauer/cloud/logs/cms_tutorial/config_cfg.1.py
cmsRun config_cfg.py
fails:
%MSG-s CMSException: AfterSourceConstruction 23-Jun-2011 16:26:54 BST pre-events
cms::Exception caught in cmsRun
---- Configuration BEGIN
Error occured while constructing primary input source.
Source is of type "PoolSource"
---- TrivialFileCatalog::connect: Unable to open trivial file catalog /vols/sl5_exp_software/cms/SITECONF/local/PhEDEx/storage.xml BEGIN
This come from
"/etc/cms/SITECONF/T2_UK_London_IC/JobConfig/site-local-config.xml".
Get storage.xml:
cd /etc/cms/SITECONF/T2_UK_London_IC/
sudo mkdir PhEDEx
cd PhEDEx
sudo wget -O storage.xml http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/SITECONF/T2_UK_London_IC/PhEDEx/storage.xml?revision=1.7
edit site-local-config.xml to point to the local version of storage.xml
next try
cd /home/dbauer/CMSSW_4_1_5/src/Tutorial/Exercise/test/
[dbauer@cloud00vm02 test]$ cmsRun config_cfg.py
23-Jun-2011 16:39:57 BST Initiating request to open file dcap://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/data/Run2010B/Mu/RECO/Nov4ReReco_v1/0001/F447BC06-38EC-DF11-ACBD-485B39800B8A.root
23-Jun-2011 16:40:03 BST Successfully opened file dcap://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/data/Run2010B/Mu/RECO/Nov4ReReco_v1/0001/F447BC06-38EC-DF11-ACBD-485B39800B8A.root
Begin processing the 1st record. Run 148862, Event 109402547, LumiSection 69 at 23-Jun-2011 16:40:23.758 BST
Begin processing the 2nd record. Run 148862, Event 109389131, LumiSection 69 at 23-Jun-2011 16:40:24.001 BST
Begin processing the 3rd record. Run 148862, Event 109400955, LumiSection 69 at
23-Jun-2011 16:40:24.174 BST
[snip]
Begin processing the 5000th record. Run 148862, Event 129672331, LumiSection 80 at 23-Jun-2011 16:53:10.497 BST
23-Jun-2011 16:53:10 BST Closed file dcap://gfe02.grid.hep.ph.ic.ac.uk/pnfs/hep.ph.ic.ac.uk/data/cms/store/data/Run2010B/Mu/RECO/Nov4ReReco_v1/0001/F447BC06-38EC-DF11-ACBD-485B39800B8A.root
=============================================
MessageLogger Summary
type category sev module subroutine count total
---- -------------------- -- ---------------- ---------------- ----- -----
1 fileAction -s file_close 1 1
2 fileAction -s file_open 2 2
type category Examples: run/evt run/evt run/evt
---- -------------------- ---------------- ---------------- ----------------
1 fileAction PostEndRun
2 fileAction pre-events pre-events
Severity # Occurrences Total Occurrences
-------- ------------- -----------------
System 3 3
Woohoo !!!
(4) Compiling user code
Can I do that on the cloud or do I need to ship in precompiled binaries ?
(Ugh.)
Copy the rest of the tutorial code onto the cloud:
ParentAnalyser.cc, ParentAnalyser.h, ParentProducer.cc, ParentProducer.h,
TwoMuonFilter.cc, TwoMuonFilter.h, config_cfg.py,
parentProducer_cff.py.
in CMSSW_4_1_5: scram build -c, then scram b. Seems to build without errors.
cmsRun config_cfg.py
Some time later: The cloud found a J/Psi !!!
(5) Skimming
Can I run the example in the cloud ?
This is 32 bit code, which is almost, but not quite legacy and I should
probably try and store the data somewhere safe before it fills up a disk.
(Cloud storage ? Or just a brute force cron job ?)
Start with a clean machine:
lcg bash
export SCRAM_ARCH=slc5_ia32_gcc434
. /cvmfs/cms.cern.ch/cmsset_default.sh
mkdir .globus, install usercert.pem, userkey.pem
voms-proxy-init --voms cms
scram project CMSSW CMSSW_3_8_7
cd /home/dbauer/CMSSW_3_8_7/src
wget http://www.hep.ph.ic.ac.uk/~dbauer/cloud/logs/analysis_example1/cmssw_example_387.tar.gz
gunzip cmssw_example_387.tar.gz
tar -xf cmssw_example_387.tar
scram build -j 2
Compiles without complaining (I am on a roll here ...)
get emacs, sigh: sudo conary install emacs-nox emacs-common
setup the missing CMS environment (next time do this before makeing the image, sigh):
sudo mkdir -p /etc/cms/SITECONF/T2_UK_London_IC/JobConfig
cd /etc/cms/SITECONF
sudo ln -s T2_UK_London_IC/ local
cd local/JobConfig
sudo wget -O cmsset_local.sh http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/SITECONF/T2_UK_London_IC/JobConfig/cmsset_local.sh?revision=1.6
sudo wget -O cmsset_local.csh http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/SITECONF/T2_UK_London_IC/JobConfig/cmsset_local.csh?revision=1.6
sudo wget -O site-local-config.xml http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/SITECONF/T2_UK_London_IC/JobConfig/site-local-config.xml?revision=1.1
sudo mkdir /etc/cms/SITECONF/T2_UK_London_IC/PhEDEx
cd PhEDEx
sudo wget -O storage.xml
http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/SITECONF/T2_UK_London_IC/PhEDEx/storage.xml?revision=1.7
edit site-local-config.xml to point to local version of storage.xml
cd CMSSW_3_8_7/src/Skimming/ElectronMuonTauJet/test
(try 10000 events)
cmsRun electronMuonTauJetTree_mc.py
10000 events result in 9.2 M output and I still have 7GB free.
This should suffice as proof or principle if nothing else.