glexec tarball install

(0) Current setup
We use a WN tarball setup with shared home directories. The tarball resides on a central, fast machine that is mounted on all the WNs. The environment is sourced on the CREAM-CE in jobwrapper.tpl, in the 'section with the header 'Environment':
. /vols/grid/glexecwn/v2/external/etc/profile.d/grid-env.sh
This currently contains the tarball WN code glite 3.2.12-1.
I have a working Argus server that serves the test EMI1 cream-ce (cetest00) only. I use this CE and a single worker (we119) node with a dedicated queue:
cetest00.grid.hep.ph.ic.ac.uk:8443/cream-sge-argus_test.q
It currently supports ops, dteama and vo.londongrid.ac.uk.

(1) The tarball
Tarball. Made by Adam Huffman. Based on gLexec glite 3.2.6-3.

(2) Changes to the Argus server
Remove old policies: pap-admin lp --show-all-ids
Remove old policies: pap-admin remove-policy af8ae483-a14d-46b1-a53e-86b0b00aa221
Load new policies (for CE and WN): pap-admin add-policies-from-file combined.policies
Reset and reload: /etc/init.d/argus-pepd clearcache; /etc/init.d/argus-pdp reloadpolicy

(3) Changes to the WN
I unpack the tarball in / so I preserve the /opt/glite structure. This is mainly to make my life easier. I've tried to deploy the tarball similar to the WN tarball, but I run into permission problems. Apparently not only the config file, but also the exectuable and an assortment of libraries need to reside on the WN. Grmpf.
I run yaim (/opt/glite/yaim/bin/yaim -c -s siteinfo-glexec.conf -n GLEXEC_wn) on it to generate my glexec.conf/lcas-glexec.db file (starting from a known good siteinfo.conf, to save me from typos), though this is optional.
There are three config files:
/opt/glite/etc/glexec.conf, lcmaps/lcmaps-glexec.db and lcas/lcas-glexec.db

Our local configuration expects ld.conf files to be in the following format:
[root@we119 etc]# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
Yaim calls the relevant file "glexec", sigh.
I need to add the globus libraries which only exist in the WN tarball, so I end up with this:
[root@we119 ld.so.conf.d]# cat glexec.conf
/opt/glite/lib64/
/vols/grid/glexecwn/v2/globus/lib/
Then do 'ldconfig' to pick up the changes.
As Simon has pointed out to me, this is rather dangerous. So I stick the two missing globus rpms (vdt_globus_essentials-VDT1.10.1x86_64_rhap_5-4.x86_64.rpm, vdt_globus_essentials-VDT1.10.1x86_rhap_5-4.i386.rpm to match the WN version we use) in a mini tarball and also dump them on the WN. Might as well. This changes glexec.conf to be:
[root@we119 ~]# cat /etc/ld.so.conf.d/glexec.conf
/opt/glite/lib64/
/opt/globus/lib/

(4) Changes to the CE
In jobwrapper.tpl I need to add export PATH=$PATH:/opt/glite/sbin/ (so it can find the glexec executable) after I source grid-env.sh.
Needs a restart of tomcat to be picked up.

(5) The tests
On the WN:
Copy a valid proxy to /tmp (here my own dtem proxy which I made earlier). Change its ownership to something with a pilot role:
chown lt2-opsplt001:lt2-opsplt x509up_u97790
su - lt2-opsplt001
export X509_USER_PROXY=/tmp/x509up_u97790
export GLEXEC_CLIENT_CERT=/tmp/x509up_u97790
export GLEXEC_SOURCE_PROXY=/tmp/x509up_u97790
[lt2-opsplt001@we119 ~]$ /opt/glite/sbin/glexec /usr/bin/id -a; echo $?
uid=101546(lt2-dteam047) gid=20015(lt2-dteam) groups=20015(lt2-dteam) context=root:system_r:unconfined_t:SystemLow-SystemHigh

Using the UK nagios
In the GOCDB add "gLEXec" as an endpoint for the corresponding CE.
Here is a link to the nagios tests.

Sending your own job
In the glexec.conf file allow all of dteam to use a pilot role:
user_white_list = .lt2-opsplt,.lt2-dteam
(for a test this will do!!!)
Submit a script with the lines:
export GLEXEC_CLIENT_CERT=$X509_USER_PROXY
which glexec
/opt/glite/sbin/glexec /usr/bin/id -a; echo $?