OSS

Created
Call Requested
SysAdmin Primary called
Call Accepted
SysAdmin Primary called
Call Acknowledged
SysAdmin Primary called
Created Jira Ticket
Resolved
Incident Tags
CHBS PostgresClusters L3UKLO Router Switch L3USHE L3AUSY L3USLV SAN DNS Squid Proxmox LogServers DatabaseServers RuneScape L3USMI web 220 Science Park
Incident Issues
Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Server pci-bastion-l01.chvm.mgnt.jagex.com is unreachable Server chbs-zbx30.mgnt.jagex.com is unreachable Server zbx30.l3uklo.mgnt.jagex.com is unreachable Server chvm-nessus-l01.mgnt.jagex.com is unreachable Server chvm-epform-l01.mgnt.jagex.com is unreachable Server chvm-snort-l01.mgnt.jagex.com is unreachable PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes L3USHE-4900A - BGP peer 172.16.34.98 is DOWN L3USHE-4900B - BGP peer 172.16.68.18 is DOWN L3USHE-4900A - BGP peer 172.16.34.18 is DOWN PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes l3ausy-asr1 - BGP peer 172.16.34.98 is DOWN l3ausy-asr2 - BGP peer 172.16.68.18 is DOWN l3ausy-asr1 - BGP peer 172.16.34.18 is DOWN L3USLV-4900A - BGP peer 172.16.34.98 is DOWN L3USLV-4900B - BGP peer 172.16.68.18 is DOWN L3USLV-4900A - BGP peer 172.16.34.18 is DOWN PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply2 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes PowerSupply1 Fan 1 Sensor 1 lost or removed, no data received for 15 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 2 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 2 Sensor 2 lost or removed, no data received for 30 minutes Fan Tray 1 Fan 1 Sensor 1 lost or removed, no data received for 30 minutes chbs-unity-480f.drac.jagex.com -> No data from zabbix_sender storage for 1 hours l3ausy-glef-l03.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-glef-l02.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-slave-ns1.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-ntp-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync hwausy-squid-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy1.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-zbx30.mgnt.jagex.com - Host clock > 60 seconds out of sync hwausy-syslog-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-syslog-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-ntp-l02.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-pxe.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy4.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-apt-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-glef-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-cache-ns1.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-slave-ns2.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-cache-ns2.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-pm-l01.mgnt.jagex.com - Host clock > 60 seconds out of sync chvm-epform-l01 - nginx is not running Billing-Core DB Cluster - chca28 (standby) - Postgresql 9.1/billing_core is offline l3ausy7.mgnt.jagex.com - Host clock > 60 seconds out of sync l3uslv21.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy6.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy3.mgnt.jagex.com - Host clock > 60 seconds out of sync l3usmi-admin1.mgnt.jagex.com - Host clock > 60 seconds out of sync l3uslv22.mgnt.jagex.com - Host clock > 60 seconds out of sync l3ausy-hdp-cn2.mgnt.jagex.com - Host clock > 60 seconds out of sync l3uslv-proxmox-l03 - Host clock > 60 seconds out of sync l3ushe-proxmox-l04 - Host clock > 60 seconds out of sync l3uslv-proxmox-l01 - Host clock > 60 seconds out of sync Billing DB Cluster - pp-billing-db-l01.chbs (master) Postgresql 9.1/billing is offline Billing DB Cluster - pp-billing-db-l01.chca (standby) - xlog Latency is 11h 12m 49s l3ushe-proxmox-l02 - Host clock > 60 seconds out of sync l3ausy9.mgnt.jagex.com - Host clock > 60 seconds out of sync l3uslv-proxmox-l04 - Host clock > 60 seconds out of sync osrs-world-135.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-39.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-content-52.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-228.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-51.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync nginx-l01.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-55.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-32.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-13.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-23.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-130.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-glef-02.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-30.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-15.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-glef-01.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-lobby-6.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-227.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-73.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-43.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-lobby-4.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync cam-zabbix-l02.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-144.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-109.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-65.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-85.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-86.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-lobby-1.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-245.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-24.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-122.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-4.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync rs3-world-229.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync osrs-world-140.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync squid-l01.l3uslv.mgnt.jagex.com - Host clock > 60 seconds out of sync RS Staffonly DB Cluster - pp-rsweb-staffonly-db-l01.chbs (master) Postgresql 9.1/rsweb_staffonly is offline l3ausy OldScape has received a large player drop with unusual traffic activity - possible DDoS