Major Incident #95438 Gamma/BT lease line resolved, 02/01/2020 05:21PM UTC 3rd Party Network Services


Notification history

02/01/2020 05:21PM UTC
Gamma engineers have now solved the Incident #95438

If you are still experiencing this issue, we recommend dropping us an e-mail to
service.desk@autocab.com quoting this reference number.

We apologise for any inconvenience this issue may have caused you.

Kind Regards

Major Incident Management
02/01/2020 11:59AM UTC
We understand from Gamma that the issues was resolved in the early hours of this morning. The Gamma update at 6.10am is below.


6:10am - The fibre issue in the London area has now been resolved as of 05:01, our monitoring and testing has confirmed that service is restoring across Gamma platforms and we will continue to monitor network stability throughout this morning.
We apologise for any disruption this incident has caused to your service
02/01/2020 01:24AM UTC
Gamma Network Outage
Latest Update from Gamma, next update will be at 8am

01/02 00:00- The network routing change is being implemented and we are starting to move network traffic across a newly established network connection. As we stabilise voice and data traffic, customers will continue to see disruption to service. Our engineering team will continue to focus on this activity as a priority. We can’t yet provide a timeframe to complete but will do so at the earliest opportunity.
Fibre engineers have now isolated the fault in the London area to a smaller patch and are in the process of testing fibre sections in that patch.
Next update will be by 01:00 or sooner if we can.

01/31/2020 11:36PM UTC
Further update from Gamma below

Gamma Network Outage
11pm - The fibre engineers are in the process of isolating the fibre break in the London area and we are expecting a clear view on the fibre restoration before 00:30. We have encountered further technical challenges on the network routing change to restore services in the interim, this was planned for 22:00 but is still work in progress and we will provide a further update on that change before 00:00.
The next update will be provided before 00:00.
We apologise for the ongoing disruption and all of our resources remain committed to restoring service as soon as we can.
01/31/2020 10:40PM UTC
Gamma Status Update


9:30pm – Fibre engineers are now on site and beginning work on the restoration.
Our network team have been continuing the work on the network re-routing of voice and connectivity services and we expect to restore those services by 10pm tonight.
We’re confident the re-routing work will fully restore services, however services remain at risk until the fibre restoration work is completed.
We will provide a further update by 11pm.

01/31/2020 08:20PM UTC
Service Status
We have isolated the fault to a dual fibre break on the East coast and the London area of our national network. Our technology partner has been engaged and they are deploying engineers to address both breaks. The incident continues to impact our voice and connectivity services and we are taking mitigating action at a network level to try to restore connectivity where we can.
We expect to be able to provide a clearer view on time to resolve by 20:00 hours, as deployed engineers establish the extent of the work at each break.
We now estimate engineers to arrive at the fibre break in the London area at 9pm at which point they can assess the extent of the work required to restore the fibre connectivity. Work will begin at 9pm on fibre break within the Leicester area, however we know the restoration work here is extensive due to an electrical incident in that area. We have taken steps within our network to re-route voice and connectivity services and this action will restore the majority of services, but we expect some ongoing intermittent problems on voice quality and connectivity speed until at one of the two fibre breaks is resolved. The next update will be provided at 21:30.

01/31/2020 06:49PM UTC
Further Update from Gammas' Twitter page

"We have isolated the fault to a dual fibre break on the East coast and the London area of our national network. Our technology partner has been engaged and they are deploying engineers to address both breaks."

"We expect to be able to provide a clearer view on time to resolve by 20:00 hours as deployed engineers establish the extent of the work at each break"

https://twitter.com/GammaSvcSupport
01/31/2020 03:58PM UTC
Gamma have updated their Twitter page with the following

"We are currently experiencing some issues impacting services across the network.

Our engineering teams are fully engaged to resolve.

A further update to be provided by 5pm"
01/31/2020 03:54PM UTC
Please note that Gamma have issued the update below via twitter

"We are currently experiencing some issues impacting services across the network. Our engineering teams are fully engaged to resolve. A further update to be provided by 5pm"


If you are also experiencing this issue, we recommend dropping us an e-mail to service.desk@autocab.com quoting this reference number and checking back on our service status page for any updates or change in status.

We apologise for any inconvenience and we are continuing to work on the problem as a priority.

Kind Regards

Major Incident Management
01/31/2020 03:17PM UTC
We have identified a Major Incident and we are actively working to gather accurate information Our engineers are aware of the issue and as a result our phone lines are expected to be busier than usual.

If you are also experiencing this issue, we recommend dropping us an e-mail to service.desk@autocab.com quoting this reference number and checking back on our service status page for any updates or change in status.

We apologise for any inconvenience and we are continuing to work on the problem as a priority.

Kind Regards

Major Incident Management