ggCircuit - We have been made aware of an issue with Amazon Web Services. – Incident details

We have been made aware of an issue with Amazon Web Services.

Resolved
Major outage
Started almost 3 years agoLasted about 18 hours

Affected

ggRock

Major outage from 3:54 PM to 9:35 AM

Add New Hardware Wizard

Major outage from 3:54 PM to 9:35 AM

ggLeap

Major outage from 3:54 PM to 9:35 AM

Desktop Client

Major outage from 3:54 PM to 9:35 AM

Web Admin

Major outage from 3:54 PM to 9:35 AM

Updates
  • Resolved
    Resolved

    All issues are now fully resolved.

  • Update
    Update

    With the network device issues resolved, Amazon is now working towards recovery of any impaired services. Some centers are reporting the ability to successfully access ggLeap web admin as well as the ggLeap client. At this time affected components are being downgraded to a "Partial outage" status as there are some backend components of ggLeap that appear to still be affected, but most center operations should be restored at this time. A ticket has been logged with the development team for further investigation. We will update this incident once more information is available.

  • Update
    Update

    While the outage is ongoing, we are seeing services slowly return to normal. https://adminbeta.ggleap.com should be mostly operational at this time. Amazon has reported that as of 3:03 PM PST most services have been restored. It appears that the services that ggLeap relies on are not yet restored, however. Thank you for your patience in this matter.

  • Update
    Update

    While the outage is ongoing, we are seeing services slowly return to normal. https://adminbeta.ggleap.com should be mostly operational at this time. It may be the better part of an hour before Amazon services are fully restored. Thank you for your patience in this matter.

  • Update
    Update

    Amazon continues seeing impact to multiple AWS APIs in the US-EAST-1 Region, including services used by ggLeap. The root cause of this issue is an impairment of several network devices in the US-EAST-1 Region. Amazon is pursuing multiple mitigation paths in parallel, and have seen some early signs of recovery. We will provide an update with further information soon.

  • Update
    Update

    Amazon continues seeing impact to multiple AWS APIs in the US-EAST-1 Region, including services used by ggLeap. The root cause of this issue is an impairment of several network devices in the US-EAST-1 Region. Amazon is pursuing multiple mitigation paths in parallel, and have seen some signs of recovery. We will provide an update with further information soon.

  • Update
    Update

    Amazon is seeing impact to multiple AWS APIs in the US-EAST-1 Region, including services used by ggLeap. The root cause of this issue is an impairment of several network devices in the US-EAST-1 Region. Amazon is pursuing multiple mitigation paths in parallel, and have seen some signs of recovery. We will provide an update with further information soon.

  • Update
    Update

    Amazon has identified root cause of the issue causing service API and console issues in the US-EAST-1 Region, and we are starting to see some signs of recovery. Amazon does not have an ETA for full recovery at this time, however we continue to monitor ggLeap services and will provide an update soon.

  • Update
    Update

    Amazon has identified the root cause of the issue causing service API and console issues in the US-EAST-1 Region, and are starting to see some signs of recovery. Amazon does not have an ETA for full recovery at this time, however we continue to monitor ggLeap services and will provide an update soon.

  • Update
    Update

    Amazon has reported that they have recognized the severity of the outage which is primarily affecting the US-EAST-1 region. Amazon has identified the root cause and is actively working towards recovery.

  • Update
    Update

    We are continuing to investigate the issue and will update shortly.

  • Update
    Update

    We are continuing to investigate the issue and will update shortly.

  • Update
    Update

    We are continuing to investigate the extent of the issue and will update shortly.

  • Monitoring
    Monitoring

    AWS status page has acknowledged the issue.

    https://status.aws.amazon.com/

    We are continuing to investigate the extent of the issue and will update shortly.

  • Identified
    Identified

    We have confirmed the issue lies with Amazon Web Services.

    Many other webservices around the world are also having problems with AWS.

    https://twitter.com/search?q=aws&src=typed_query&f=live

  • Investigating
    Investigating

    We are currently investigating this incident. We will report back with an update every 15 minutes.