Jan 12, 00:00 UTC
Completed - The scheduled maintenance has been completed.

Jan 11, 21:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jan 11, 20:25 UTC
Scheduled - Start: 2025-01-11 21:00 UTC
End: 2025-01-12 24:00 UTC

Hello,

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC3 region.

Expected impact:

During the maintenance window users may experience delays or failures with event processing for a brief duration on Droplets and Droplet-based services including Droplets, Managed Kubernetes, Load Balancers, Container Registry, and App Platform. We will endeavor to keep this to a minimum for the duration of the change.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

Jan 11, 21:12 UTC
Resolved - As of 18:55 UTC, our Engineering team has confirmed the full resolution of the issue that impacted Managed Kubernetes Clusters in all of our regions. All of the services should now be working normally.

If you continue to experience problems, please open a ticket with our Support team from within your Cloud Control Panel.

Thank you for your patience and we apologise for any inconvenience.

Jan 11, 18:29 UTC
Investigating - Our Engineering team is currently investigating an issue with Managed Kubernetes clusters. During this time, some users may experience delays when provisioning Kubernetes clusters, specifically version 1.31.1-5.

We apologize for the inconvenience and will provide an update as soon as we have more information.

Jan 9, 18:01 UTC
Resolved - From 15:28 to 15:32 UTC, an internal service disruption may have resulted in users experiencing errors while using the Cloud Panel or API to manage Spaces Buckets, Apps, Managed Database Clusters, or Load Balancers as well as other actions due to impacted downstream services.

If you continue to experience problems, please open a ticket with our support team.

Thank you and we apologize for any inconvenience.

Jan 7, 16:43 UTC
Resolved - From 13:09 - 14:48 UTC, we experienced an issue with IPv6 network connectivity in our SFO3 region. Our Engineering team was able to identify the root cause, revert changes, and resolve the issue.

During that time, connectivity via IPv6 to services in SFO3 was impacted. This included, in particular, connectivity to and provisioning of Load Balancers, control plane connectivity to Kubernetes clusters, and connectivity to App Platform Apps.

If you continue to experience problems, please open a ticket with our support team. Thank you for your patience and we apologize for any inconvenience.

Jan 7, 15:51 UTC
Monitoring - Our engineering team has implemented a fix to resolve the networking issue in our SFO3 region. Users should no longer encounter connectivity issues, provisioning Load Balancers, latency, and timeout errors while interacting with resources in the SFO3 region.

We will post an update as soon as the issue is fully resolved.

Jan 7, 14:47 UTC
Update - Our Engineering team is continuing to investigate the cause of the issue with networking in our SFO3 region and is taking action to mitigate this incident by reverting recent changes. In addition to connectivity issues, users may see failures with provisioning Load Balancers in SFO3.

We are seeing some services recover, but expect to see a few more short periods of network connectivity issues as the reverts progress.

We will continue to share updates as soon as we have new information.

Jan 7, 13:35 UTC
Investigating - Our engineering team is investigating an issue related to the SFO3 network maintenance here: https://status.digitalocean.com/incidents/lf0njg8ysj0w

During this time users may experience connectivity issues, latency, and timeout errors while interacting with resources in the SFO3 data center.

We apologize for the inconvenience and will share an update once we have more information.

Jan 7, 14:14 UTC
Resolved - Our engineering team has resolved the issue preventing some users from adding new credit cards from the control panel. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Jan 7, 13:38 UTC
Monitoring - Our engineering team has implemented a fix to resolve issues that some users experienced when attempting to add new credit cards to the Cloud Control Panel.

We are monitoring the situation and will post an update as soon as the issue is fully resolved.

Jan 7, 13:24 UTC
Investigating - Our engineering team is investigating reports of issues occurring when users attempt to add a new credit card in the billing settings page of the Cloud Control Panel.

During this time, some users may be unable to input card information. We apologize for the inconvenience and will share an update once we have more information.

Jan 7, 14:00 UTC
Completed - The scheduled maintenance has been completed.

Jan 7, 10:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jan 5, 10:15 UTC
Scheduled - Start: 2025-01-07 10:00 UTC
End: 2025-01-07 14:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the SFO3 region.

Expected impact:

These upgrades are designed and tested to be seamless and we do not expect any impact to customer traffic due to this maintenance. If an unexpected issue arises, affected Droplets and Droplet-based services may experience increased latency or a brief disruption in network traffic. We will endeavor to keep any such impact to a minimum.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

Dec 19, 05:03 UTC
Resolved - From 1:23 UTC to 3:04 UTC, users may have experienced issues with events being stuck or delayed, such as powering on/off, and resizing Droplets in the NYC3, AMS2, BLR1, SGP1, and SYD1 regions. Additionally, Managed Database creates were delayed in all regions.

Our Engineering team has confirmed full resolution of the issue, delayed events and new events should complete as normal now.

Thank you for your patience through this issue. If you continue to experience any issues, please open a support ticket from within your account.

Dec 19, 04:28 UTC
Update - Our Engineering team has confirmed that new events are succeeding, and we're finalizing cleaning up any stalled events. We will post an update as soon this issue is fully resolved.

Dec 19, 03:08 UTC
Monitoring - Events in NYC3 has been enabled now and users may continue to submit new events. We are starting to see success rates with event processing go up.

We will post another update soon. Thank you for your patience.

Dec 19, 02:42 UTC
Update - Our Engineering team continues to work to mitigate the issue with event processing in multiple regions.

To help address the issue, we are temporarily disabling all Droplet-related actions in the NYC3 region for a period of 20 minutes. During this time, users in NYC3 will not be able to submit actions, such as creating, resizing, powering on/off, etc.

We apologize for the inconvenience and appreciate your patience as we work to resolve this issue. Further updates will be shared as they become available.

Dec 19, 01:45 UTC
Investigating - Our Engineering team is investigating an issue impacting Droplet event processing in multiple regions.

At this time, users may experience issues with events appearing to be stuck or delayed, such as powering on/off, and resizing Droplets in NYC3, AMS2, BLR1, SGP1, and SYD1 regions. Additionally, Managed Database creates are delayed in all regions.

We apologize for the inconvenience and will share an update once we have more information.

Dec 13, 13:25 UTC
Resolved - From 13:25 to 14:45 UTC, our Engineering team observed a Networking issue in our NYC3 region. During this time, users may have experienced Droplet and VPC connectivity issue, Users should no longer be experiencing these issues.
We apologize for the inconvenience. If you have any questions or continue to experience issues, please reach out via a Support ticket on your account.

Dec 13, 11:01 UTC
Resolved - Our Engineering team has confirmed that the issues with Spaces CDN functionality has been fully resolved. Users should now be able to use CDN functionality normally.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Dec 13, 10:55 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the CDN functionality issue and is currently monitoring the situation.

We will continue to monitor this at our end and will share an update once the issue is resolved completely.

Dec 13, 09:55 UTC
Investigating - Our Engineering team is investigating an issue affecting Spaces CDN functionality globally. During this time, users may experience issues when enabling Spaces CDN and with existing CDN services.

We apologize for the inconvenience and will share an update once we have more information.

Dec 13, 04:05 UTC
Resolved - Our Engineering team has confirmed the full resolution of the issue impacting Managed Database Operations, and all systems are now operating normally. Users may safely resume operations, including upgrades, resizes, forking, and ad-hoc maintenance patches.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Dec 13, 03:13 UTC
Monitoring - Our Engineering team has implemented a fix for the issue impacting Managed Database Operations. The team is monitoring the situation, and we will share an update once this is fully resolved.

Dec 12, 22:03 UTC
Update - Our Engineering team is continuing to work on investigating the root cause of the issue affecting Managed Database Operations. At this time, we are seeing improvement and some new cluster creations are completing successfully.

To avoid potential downtime, we continue to ask users to refrain from performing operations that trigger node rotations, such as upgrades, resizes, forking, and ad-hoc maintenance patches.

We apologize for any inconvenience caused and appreciate your patience. We will post further updates as soon we have more information.

Dec 12, 19:35 UTC
Update - During our investigation, we identified that operations triggering node rotation, such as upgrades, resizes, forking and ad-hoc maintenance patches, may cause issues connecting to the cluster.

To prevent potential downtime, we recommend avoiding these operations until the issue is fully resolved.

We apologize for the inconvenience and appreciate your patience as we work to address the situation.

Dec 12, 18:34 UTC
Investigating - Our Engineering team is investigating an issue with the Managed Database clusters which is causing forks, restores and new creates to fail. The Mongo clusters are unaffected and should be operating normally.

Our team is currently assessing the root cause and working to resolve the issue as quickly as possible.

We apologize for any inconvenience this may cause and will share an update as soon as we have more information.

Dec 13, 01:24 UTC
Resolved - Our Engineering team has confirmed that the issues with Authoritative DNS resolution in NYC1 has been fully resolved. DNS queries should now be resolving normally.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Dec 13, 01:10 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issues with Authoritative DNS Resolution in NYC1 and is currently monitoring the situation.

We will continue to monitor this at our end and will share an update once the issue is resolved completely.

Dec 12, 23:20 UTC
Investigating - Our Engineering team is investigating an issue with intermittent failures and increased latency in DNS resolution in NYC1.

During this time, a subset of users may experience issues with DNS resolution, or see errors returned when querying DNS records which are hosted on the DigitalOcean authoritative DNS infrastructure.

We apologize for the inconvenience and will share an update once we have more information.