Statement on Axis network outage, 13th August 2019
Axis takes every step to avoid network outages. On the evening of 13th August 2019, however, the normally redundant and high performing core switches in one of our sites became corrupt and stopped behaving as specified, resulting in problems that the switches are designed to never encounter. While the system is designed for maximum uptime, extreme performance and complete redundancy, at 18:45 CET the issue caused a complete network outage.
Together with our network staff and the core switch/network supplier we immediately started working on an analysis and fix. Given the size of the site and complexity of the issue, this took until approximately 21:00 CET Tuesday evening, involving experts within Axis and senior experts from our supplier, both on location in Sweden and remotely from the USA.
The work to restore everything to working conditions started as soon as we had an understanding of the cause. At approximately 02:30 CET on the 14th August, the core network was restored and we could start bringing network servers back online. At 08:00 CET, the network was back to near full capacity. By 12:00 CET, all known issues had been resolved and the network was operating as normal.
We would like to apologize to any customers affected by the network outage, and reassure them that we are taking every step to prevent similar future situations.