Jun 7, 19:11 UTC
Completed - Our datacenter provider has informed us that they are not able to perform this maintenance during the communicated window and will be rescheduling it for a later date. We apologize for any confusion from the previous update, but this maintenance did not begin and there was no impact to any services. We will notify you via email and a new maintenance post on status.digitalocean.com with the updated date and timings for this maintenance.

If you have any questions or continue to experience issues, please reach out via a Support ticket on your account.

Jun 7, 11:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jun 7, 02:44 UTC
Scheduled - Start: 2024-06-07 11:00 UTC
End: 2024-06-07 21:00 UTC

During the above window, our data center team will be performing maintenance on redundant upstream electrical systems in our SFO2 region.

Expected impact:

We don’t expect to see any impact to customer resources as the electrical systems are redundant and our team will isolate one unit at a time during the maintenance. In the highly unlikely event that we lose power to both systems simultaneously, affected Droplets and Droplet-based services would experience downtime for a brief period of time. We will endeavor to keep this to a minimum should that situation arise.

If you have any questions or concerns related to this maintenance, please send us a ticket from your cloud support page https://cloudsupport.digitalocean.com/s/createticket.

Jun 5, 15:30 UTC
Resolved - From 12:10 - 12:41 UTC our Engineering team identified and resolved an issue impacting the Cloud Control Panel and API across multiple regions.

During this time, users may have experienced trouble while viewing the IP address on their newly deployed droplets. Our Engineering team was able to take quick action to mitigate the impact and resolve the issue. All services are now functioning as expected.

Thank you for your patience, and we apologize for any inconvenience. If you continue to experience any issues, please open a Support ticket for further analysis.

Jun 3, 15:29 UTC
Resolved - Our engineering team has resolved the issues with increased latency in the time for updating DNS records. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Jun 3, 12:27 UTC
Monitoring - Our engineering team has identified the cause and implemented a fix to resolve the issue with increased latency in the time for updating DNS records and is monitoring the situation. We will post an update as soon as the issue is fully resolved.

Jun 3, 11:30 UTC
Investigating - Our Engineering team is investigating an issue with intermittent failures and increased latency in DNS resolution across multiple regions.

During this time, users may experience issues with DNS resolution, or see errors returned when querying DNS records which are hosted on the DigitalOcean nameservers. Users may also experience DNS resolution issues while reaching out to their Managed Kubernetes Clusters.

We apologize for the inconvenience and will share an update once we have more information.

May 30, 22:07 UTC
Resolved - Our Engineering team has confirmed that this incident has been fully resolved.

Thank you for your patience. If you continue to experience any problems, please open a ticket with our support team for further review.

May 30, 20:51 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issue with event processing on the two nodes in TOR1. At this time, users should no longer face issues with powering on, powering off, and power cycling Droplets.

We are monitoring the situation closely and will post an update as soon as we confirm the issue is fully resolved.

May 30, 20:16 UTC
Identified - After further investigation, our Engineering team has determined the scope of this issue to be limited to two physical nodes in our TOR1 datacenter. Power event errors seen in NYC3 are not related to this incident and are a result of a single hypervisor issue, which is being remediated separately.

The root cause for event issues in TOR1 has been identified and a fix is underway. We will update the title of this incident shortly to accurately reflect the scope and will post an update as soon as the fix has rolled out or there is additional information to share.

May 30, 19:42 UTC
Investigating - Our Engineering team is investigating an uptick in reports of issues with Power on, Power off, and Power cycle events in our TOR1 and NYC3 regions. During this period, customers might experience errors and/or delays in powering on, powering off, or rebooting Droplets.

We will post an update as soon as additional information is available.

May 29, 11:47 UTC
Resolved - Our Engineering team has confirmed that the issue with accessing Kubernetes and App Platform for new users has been completely resolved. New users should now be able to access all aspects in the Cloud Control Panel.

If you continue to experience problems, please open a ticket with our support team.

Thank you for your patience and we apologize for any inconvenience.

May 29, 10:47 UTC
Monitoring - Our Engineering team has identified the root cause and deployed a fix for the issue impacting access to Kubernetes and App platform pages for new users. New users should now be able to access Kubernetes and App platform in our Cloud Control Panel successfully.

We will continue to monitor this at our end and will share an update once the problem is resolved completely.

May 29, 07:19 UTC
Investigating - Our Engineering team is investigating an issue where newly signed up users are unable to access Kubernetes and App platform pages in our Cloud Control Panel. Beginning at 07:00 UTC on May 29th, new users may see "Oops, you do not have access to view this resource or page" in our Cloud Control Panel.

We apologize for the inconvenience and will share an update once we have more information.

May 24, 00:41 UTC
Resolved - Our engineering team has resolved the issue issue with creating new Uptime checks in the Monitoring section of the Cloud Control Panel.

Thank you for your patience. If you continue to experience any problems, please open a support ticket from within your account.

May 24, 00:07 UTC
Monitoring - Our Engineering team has implemented a fix regarding the issue with creating new Uptime checks in the Monitoring section of the Cloud Control Panel. We are monitoring the situation to ensure there is no recurrence.

We will post another update once we confirm the issue is fully resolved.

May 23, 23:26 UTC
Investigating - Our Engineering team is investigating an issue with creating new Uptime checks in the Monitoring section of the Cloud Control Panel. From 15:00 UTC on the 21st, customers may experience errors while creating new Uptime checks on the Monitoring page.

We apologize for the inconvenience and will share an update once we have more information.

May 29, 14:00 UTC
Completed - The scheduled maintenance has been completed.

May 29, 10:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 22, 17:13 UTC
Scheduled - Start: 2024-05-29 10:00 UTC
End: 2024-05-29 14:00 UTC

During the above window, we will be performing maintenance on core switches in our NYC3 datacenter as part of network upgrades.

Expected Impact:

Inbound and outbound traffic for Droplet-based services in NYC3 may experience a few brief interruptions as traffic is isolated from core switches being upgraded. This impact would be for a duration of 5 to 10 seconds and could occur a few times within the maintenance window. Our team will make every effort to seamlessly drain traffic in order to limit customer impact.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

May 21, 09:47 UTC
Resolved - As of 09:05 UTC, our Engineering team has confirmed the full resolution of this issue. All users should be able to deploy Serverless Functions from the App UI without any issues.

If you continue to experience problems, please open a ticket with our support team. Thank you for being so patient and we apologize for any inconvenience.

May 21, 09:18 UTC
Update - We are still monitoring the issue impacting Serverless Functions. From the App UI, Serverless Functions should be deployable by all users.

Apologies for the inconvenience caused. We will post an update as soon as the issue is fully resolved.

May 21, 09:12 UTC
Monitoring - Our Engineering team has deployed a fix to resolve an issue with Serverless Functions. All users should be able to deploy Serverless Functions from the App UI.

We are monitoring the situation closely and will share an update once the issue is resolved completely.

May 21, 08:26 UTC
Investigating - As of 06:38 UTC, our Engineering team is investigating an issue with Serverless Functions.

Users may see errors when attempting to deploy Serverless Functions from the App UI.

We apologize for the inconvenience and will share an update once we have more information.

May 24, 02:00 UTC
Completed - The scheduled maintenance has been completed.

May 24, 00:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 16, 13:53 UTC
Scheduled - Start: 2024-05-24 00:00 UTC
End: 2024-05-24 02:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the EU region.

Expected Impact:

Our team will be closely monitoring and we do not expect any disruption to customer workloads

If an unexpected issue arises, affected Droplets and Droplet-based services in AMS, LON, and FRA regions could experience increased latency or a disruption in public network traffic for a short period of time. If an unexpected issue does occur, our team is prepared to rollback to a safe state to quickly restore public networking. We expect that traffic would be impacted for a period of 5-10 minutes at most in that case.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

May 24, 02:00 UTC
Completed - The scheduled maintenance has been completed.

May 23, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 16, 13:31 UTC
Scheduled - Start: 2024-05-23 22:00 UTC
End: 2024-05-24 02:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC2 region. This is a rescheduled maintenance event previously scheduled for 2024-05-14.

Expected impact:

These upgrades are designed and tested to be seamless and we do not expect any impact to customer traffic due to this maintenance. If an unexpected issue arises, affected Droplets and Droplet-based services may experience increased latency or a brief disruption in network traffic. We will endeavor to keep any such impact to a minimum.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

May 13, 20:49 UTC
Resolved - Our Engineering team has identified and resolved an issue that impacted the provisioning of multiple products from 17:51 UTC until 18:59 UTC. During this time, users might have experienced delays or errors while provisioning their MongoDB Managed Database clusters, Container Registries and also while creating Spaces Keys.

Swift action was taken by our Engineering team that restored services and now every thing is operating normally.

We apologize for any inconvenience this may have caused. If you have any questions or continue to experience issues, please reach out via a Support ticket on your account.

May 7, 20:03 UTC
Resolved - Our Engineering team has confirmed that the issue with Spaces Buckets creates in multiple regions has been fully resolved.

If you continue to experience any problems, please open a support ticket from within your account. Thank you for your patience and we apologize for the inconvenience.

May 7, 19:06 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issue affecting creation of Spaces Buckets in multiple regions and is monitoring the situation closely.

The impact has subsided and users should no longer experience issues creating Spaces Bucket both through the cloud control panel and via API.

We will post an update as soon as the issue is fully resolved.

May 7, 14:07 UTC
Investigating - Our Engineering team is investigating an uptick in error rates for users creating Spaces Buckets in multiple regions, starting May 3.

We have rolled back a recent change in our systems in an effort to address the issue. However, we are still investigating and users may still experience errors when attempting to create Spaces Buckets, both through the cloud control panel and via API.

We apologize for the inconvenience and will share an update once we have more information.

May 15, 01:38 UTC
Completed - During the maintenance, our Engineering team observed possible connectivity issues between Droplets and networking services like LBaaS.

Due to such unforeseen issues, our Engineering team has decided to rollback this maintenance for now and will be rescheduling it at a later date.

We apologize for the inconvenience and the unexpected downtime to services. If you have any questions or continue to experience issues, please reach out via a Support ticket on your account.

May 14, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 7, 04:13 UTC
Scheduled - Start: 2024-05-14 22:00 UTC
End: 2024-05-15 02:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC2 region.

Expected impact:

These upgrades are designed and tested to be seamless and we do not expect any impact to customer traffic due to this maintenance. If an unexpected issue arises, affected Droplets and Droplet-based services may experience increased latency or a brief disruption in network traffic. We will endeavor to keep any such impact to a minimum.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

May 7, 04:41 UTC
Resolved - Our Engineering team has confirmed the complete resolution of the issue at 03:51 UTC.

Creating Droplets, all Droplet-based services including Kubernetes Clusters, Managed Databases, Load Balancers, and Snapshots are now working as expected in the NYC3 region.

Thank you for being so patient. If you continue to experience any problems, please open a support ticket from within your account.

May 7, 04:11 UTC
Monitoring - Our Engineering team has investigated and identified an issue with creating resources in the NYC3 region. Between approximately 03:18 UTC - 03:51 UTC, customers may have experienced errors when creating Droplets and Droplet-based services including Managed Kubernetes, Managed Databases, Load Balancers, and snapshots.

Our team has implemented a fix to restore the ability to create resources and is monitoring the region. We will post an update as soon as the issue is fully resolved.

May 9, 14:31 UTC
Completed - The scheduled maintenance has been completed.

May 9, 12:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 3, 19:17 UTC
Scheduled - Start: 2024-05-09 12:00 UTC
End: 2024-05-09 15:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC2 region. This is a rescheduled maintenance event previously attempted on 2024-05-02.

Expected Impact:

These upgrades are designed and tested to be seamless and we do not expect any impact on customer traffic due to this maintenance.

If an unexpected issue arises, affected Droplets and Droplet-based services could experience increased latency or disruption in public network traffic for a short period of time. If an unexpected issue does occur, our team is prepared to rollback to a safe state to quickly restore public networking. We expect that traffic would be impacted for a period of 5-10 minutes at most in that case.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

May 6, 18:43 UTC
Resolved - Our Engineering team has confirmed that the issue with our App Platform and Container Registry services has been fully resolved. If you continue to experience any trouble with these services, please open a ticket with our support team. Thank you for your patience and we apologize for the inconvenience.

May 6, 18:00 UTC
Monitoring - Our engineering team has implemented a fix to resolve the issue with our App Platform and Container Registry services and is monitoring the situation closely. We will post an update as soon as the issue is fully resolved.

May 6, 15:30 UTC
Update - Our Engineering team is still investigating the issue with App Platform and Container Registry. Currently, only a subset of users will continue to see errors with App Platform deployments and Container Registry.

We will post an update as soon as additional information is available.

May 6, 10:29 UTC
Investigating - Our Engineering team is investigating an issue with the DigitalOcean Container Registry and App Platform. At this time, users may find App Platform builds failing. Users may also experience issues while pushing images to the DigitalOcean Container Registry.

We apologize for the inconvenience and will share an update once we have more information.

May 5, 16:50 UTC
Resolved - Our Engineering team has resolved the issue impacting DNS resolution across multiple regions. The functionality is restored completely and the DNS queries should now be resolving normally.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

May 5, 15:31 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issues with the DNS resolution and is monitoring the situation. DNS queries are now succeeding without any failures as the functionality has started to recover.

We will continue to monitor this at our end and will share an update once the problem is resolved completely.

May 5, 14:08 UTC
Identified - Our engineering team continues to work on a fix for the issue impacting DNS resolution across multiple regions. At this time, users will continue to experience intermittent issues with DNS resolution impacting droplet based services.

We apologize for the inconvenience, and will post an update as soon as additional information is available.

May 5, 12:55 UTC
Investigating - Our Engineering team is investigating an issue with intermittent failures and increased latency in DNS resolution across multiple regions.

During this time, users may experience issues with DNS resolution, or see errors returned when querying DNS records which are hosted on the DigitalOcean nameservers. Users may also experience DNS resolution issues while reaching out to their Managed Kubernetes Clusters.

We apologize for the inconvenience and will share an update once we have more information.

May 2, 13:44 UTC
Completed - From 12:14 UTC to 12:32 UTC, our Engineering team observed an issue where a subset of Droplets (any Droplets with an assigned public IP in the 192.241.160.0/19 range) and all Load Balancers in the NYC2 region were unreachable due to some backend changes made during the maintenance. Our Engineering team has resolved the issue and all services were reachable again at 12:32 UTC.

Due to such unforeseen issues, our Engineering team has decided to rollback this maintenance for now and will be rescheduling it at a later date.

We apologize for the inconvenience and the unexpected downtime to services. If you have any questions or continue to experience issues, please reach out via a Support ticket on your account.

May 2, 12:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 30, 15:32 UTC
Scheduled - Start: 2024-05-02 12:00 UTC
End: 2024-05-02 15:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC2 region.

Expected Impact:

These upgrades are designed and tested to be seamless and we do not expect any impact on customer traffic due to this maintenance.

If an unexpected issue arises, affected Droplets and Droplet-based services could experience increased latency or disruption in public network traffic for a short period of time. If an unexpected issue does occur, our team is prepared to rollback to a safe state to quickly restore public networking. We expect that traffic would be impacted for a period of 5-10 minutes, at most, in that case.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

Apr 30, 16:58 UTC
Resolved - Our Engineering team has confirmed the root cause of this issue. Preparation work for the upcoming maintenance in NYC2 (https://status.digitalocean.com/incidents/z9r8gg95bqqm) was in-flight and had an unexpected impact, causing the connectivity outage. This was corrected and should not occur again.

All services were restored as of 16:06 UTC and we confirmed this incident impacted only a subset of customer Droplets in NYC2.

We apologize for the disruption and appreciate your patience while the root cause was confirmed. If you have any questions or concerns, please reach out to Support from within your account. Thank you.

Apr 30, 16:23 UTC
Monitoring - Our Engineering team has investigated and identified an issue with networking in our NYC2 region. From 15:54 - 16:06 UTC, services in our NYC2 region experienced a complete disruption in network availability. Users attempting to access Droplet and Droplet-dependent resources would have seen connections failing.

Our team has identified a breaking change and a rollback was completed. We are confirming full scope of impact and ensuring this issue will not reoccur.

We'll post another update soon.

Apr 26, 00:36 UTC
Resolved - Our Engineering team has continued to actively monitor the situation resulting from multiple subsea fiber faults in our Singapore (SGP1) region, and customers in Asia attempting to reach DigitalOcean resources hosted in our Amsterdam (AMS2, AMS3), London (LON1), and Frankfurt (FRA1) regions.

Our Support teams continue to surface customer escalations to our Networking team and work to shift to better/less latent routes will continue as well.

Repair of all cables from crews is still pending, and periodic network instability may continue to present itself during hours of high traffic, for multiple weeks until repairs are fully completed.

We will now close out this incident and provide any needed updates separately.

If users experience a disruption in service or have any questions, we invite them to submit a support ticket from within their account.

Thank you for your patience and understanding throughout this incident.

Apr 21, 11:56 UTC
Update - Our Engineering team continues to work with upstream providers to resolve network connectivity issues that are occurring in multiple regions. As a result of multiple subsea fiber faults between Europe and Asia regions, the issues appear to be a direct consequence of congestion resulting in packet loss and increased latency connecting to the SGP1 region. Customers in the Asian region are having difficulty reaching DigitalOcean resources located in Amsterdam (AMS2, AMS3), London (LON1), and Frankfurt (FRA1).

We have no ETA for the upstream issues being restored but we will communicate any relevant information as we have it. We apologize for the inconvenience.

Apr 20, 16:20 UTC
Identified - We've confirmed the degraded network performance in our first update is due to a cable cut, which occurred at approximately 18:00 UTC, April 19. This cable is located just west of Singapore and impacts the cabling system connecting 17 countries from Singapore to Europe. Users can read more about the cable here: https://www.submarinenetworks.com/en/systems/asia-europe-africa/smw5


We continue to monitor this situation and progress of the repair. Our Engineering team will also continue to tune for performance and alternate routes where possible, until the issue is resolved.

If you have any questions, please open a Support ticket from within your account.

Apr 20, 12:10 UTC
Investigating - Due to multiple simultaneous subsea fibre cuts between Europe and Asia, we are observing degraded performance for European customers attempting to reach DigitalOcean resources hosted in our Singapore (SGP1) region, and Asian customers attempting to reach DigitalOcean resources hosted in our Amsterdam (AMS2, AMS3), London (LON1), and Frankfurt (FRA1) regions.

Engineers are attempting to improve performance where possible, but as this issue is largely out of our control we do not expect a substantial improvement in performance until one of the fibre cuts is repaired.

We apologize for any inconvenience.