More computer network issues in the UK

Following on from last weeks major network problems for the Natwest banking group and the difficulties faced by their customers over a four day weekend on lack of access to bank accounts comes the next fault. This time it is mobile telecommunications with O2 the UK network that was once a part of British Telcom with reports suggesting that thousands of customers are unable to make or receive phone calls. It appears that the problem may be even more severe than that affecting not just thousands but all O2 customers across the UK, with problems affecting making and receiving voice calls, text messages and even proprietary services like Blackberry messenger. The breadth and type of the outage, along with the intermittent nature and the UK wide effect suggests that this may not be the simple network issue that has been announced to the press but as with the bank network, something more sinister.

So why might a major telecoms company with several UK datacentres and secure disaster recovery strategies and resilience suffer an outage of this type? Anyone with an understanding of resilient datacentre construction and operation will tell you that under normal circumstances even if an entire data hall went down the operation could still be switched to an alternative so there is little possibility that this was the result of power loss, as all datacentres operate power generation facilities or Uninterruptible Power Supply (UPS) backup, or that it was a consequence of a software or operator error as datacentres operate a policy of automatic rollback on software changes or upgrades if instability is detected. Given the latest policies from the UK government regarding police and security services access to mobile communication data the most likely explanation is that the outage is being caused by the installation and integration of monitoring software into the core network servers.

This operation would require network shutdown for an extended period as has been seen this afternoon, followed by an extended period of 48-72 hours of intermittent service as the new software is tested and modified. Expect similar problems with Orange and Vodafone over the next two weeks in the run up to the Olympics and further issues as additional monitoring is put in place. Be careful what you say, what you text and what you message, and particularly if you are on O2, and remember, Big Brother knows where you live, what you say and pretty soon, what you think.