Debugging hints

VM in Error State

Error message: Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 7a8a18e9-32eb-4ecb-b736-924fe8cdd552. Last exception: Binding failed for port c50dd452-fdb5-4658-b96b-515fa95c5c84, please check neutron logs for more information.
Notes:Binding is done on the WN, the neutron logs referred to here are the neutron logs on the WN.
Steps to take:
(a) On osnova (which schedules the VMs), find machine that hosts the instance: grep 7a8a18e9-32eb-4ecb-b736-924fe8cdd552 nova-api.log. This will yield the names of all the nodes it tried to start the machine on.
(b) On the WN check: /var/log/neutron/linuxbridge-agent.log. If this is empty (because e.g. linuxbridge-agent crashed; systemctl status neutron-linuxbridge-agent.service) find the last one that did work (zless to read gz file).
(c) This time around the error was:
OSError: [Errno 24] Too many open files: '/sys/class/net/'
(d) Fix: Increase "Max open files" for systemd:
Check status: get process id, then do cat /proc/[processid]/limits For linuxbridge-agent, create directory and then drop in file:
mkdir /etc/systemd/system/neutron-linuxbridge-agent.service.d


	[root@clc06#] cat /etc/systemd/system/neutron-linuxbridge-agent.service.d/10_nofiles.conf
 
        [Service]

        LimitNOFILE=8192

reload config files: systemctl daemon-reload
(re)start neutron-linuxbridge-agent: systemctl start neutron-linuxbridge-agent.service

Delete an instance stuck in deleting state

on osbase: (copy id from webinterface)


     openstack server set --state error 85dce352-220c-45fb-9554-cdd46dff6a98

     openstack server delete 85dce352-220c-45fb-9554-cdd46dff6a98