The Imperial IRIS cloud

Dashboard

The initial install used OpenStack Rocky, but we have now upgraded to Stein.

Physical Nodes:

Head nodes: cloud00, 01, 02 (Favourite commands: pcs status, pcs resource cleanup )
CEPH nodes: cldsk03, 04, 05 (Favourite commands: ceph -s, ceph osd lspools )
Worker nodes: cla02-cla10 (dev), clb00-clb11 (iris), clc00-clc14 (iris) and others.

Openstack nodes
  1. osbase: admin node, database, message queue and other bits and pieces.
    Have a look around:
    source admin-openrc
    openstack flavor list --all
    openstack image list
    openstack project list
    openstack port list
    openstack quota show [project name]
  2. oscinder: volume storage
  3. osglance: images
  4. osheat: orchestration
  5. oshorizon: webinterface
  6. oskeystone: authentication
  7. osneutron00: networking
  8. osneutron01: networking
  9. osneutron02: networking
  10. osheat: orchestration
  11. osnova: compute
  12. osceil: metrics
  13. osacct: accounting (not really an openstack product)

Getting a new project onto the cloud (here: JINTRAC)
Note: This is all done on osbase (the admin node).
  1. source admin-openrc
  2. Create project:
    openstack project create --domain default --description "jintrac iris allocation" jintrac
    (obsolete) openstack role add --project jintrac --user admin member (so admin user can checkup on project)
    We now have a load balancer, so we might want to assign the firewall and load balancer permissions seperately. The reason for doing it this way is that all of the permissions (roles) are consistent across all of the users if we want to change them later. The accounting user should still be directly assigned and not changed.
    First we create a group:
    openstack group create jintrac_grp
    openstack role add member --group jintrac_grp --project jintrac
    Optionally repeat the above changing "member" to "security-group_member" or "load-balancer_member" for extra permissions. openstack group add user jintrac_grp admin
  3. Set Limits:
    openstack quota set --ram 360000 --instances 60 --cores 60 --secgroups 100 --secgroup-rules 1000 --volumes 0 --floating-ips 3 --per-volume-gigabytes 0 --networks 1 --gigabytes 0 --backup-gigabytes 0 --subnetpools 0 --snapshots 0 --subnets 1 --routers 1 --ports 500 jintrac (60 (virtual) cores wih a 6GB/core and no shenanigans.)
  4. Create flavour:
    openstack flavor create --ram 16000 --disk 20 --vcpus 8 --private --project jintrac jintrac.medium
    openstack flavor set --property aggregate_instance_extra_specs:nodegroup=iris e2e95685-6323-4d7a-b0e1-d34058842eb0
    (use id from previous command, or just plain flavor name)
    openstack flavor set --property acct:benchmark_name="HEPSPEC06" --property acct:benchmark_value=144.8 jintrac.medium
  5. Create Network:
    openstack network create --mtu 1500 --project jintrac jintrac_net
    openstack subnet create --network jintrac_net --dns-nameserver 146.179.236.2 --dns-nameserver 146.179.236.3 --gateway 172.16.1.1 --subnet-range 172.16.1.0/24 --project jintrac jintrac_sub

    Note: /24 assigns approximately 250 addresses. E.g. LZ needs more and therefore needs to be assigned /23.
    openstack router create --ha --project jintrac jintrac_router
    openstack router add subnet jintrac_router jintrac_sub
    openstack router set --external-gateway provider jintrac_router
    (diagnostics: openstack port list --router jintrac_router)
  6. Create User(s):
    openstack user create --project jintrac --email user@somewhere.uk --description "User Name" --password-prompt userlogin
    new openstack group add user jintrac_grp userlogin
    (obsolete) openstack role add --project jintrac --user userlogin member
  7. Setup accounting:
    on osbase:
    openstack role add --user accounting --project jintrac accounting
    openstack project set --tag caso jintrac openstack project set --property accounting:VO=jintrac jintrac

    Unlike in previous versions of caso no changes on the osacct node are needed.



Obsolete: How to make a cloud


Notes on setting up a WN (pretty complete).
When things go wrong: How to fix stuff


Old notes on networking.
Old notes on setting up CEPH.
Simon's notes on setting up Openstack Heat
Setting up ceilometer