Normal view

SFO3 Network Maintenance

27 November 2024 at 07:00

Nov 27, 12:00 UTC
Completed - The scheduled maintenance has been completed.

Nov 27, 10:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Nov 25, 10:11 UTC
Scheduled - Start: 2024-11-27 10:00 UTC
End: 2024-11-27 14:00 UTC

During the above window, our Networking team will be performing maintenance on core switches in our SFO3 datacenter as part of network upgrades.

Expected Impact:

These upgrades are designed and tested to be seamless, and we do not expect any impact to customer traffic due to this maintenance. Should an unexpected issue arise, a possible outcome would be temporary loss of connectivity or increased latency for a brief period of time. We will endeavor to keep any such impact to a minimum.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

NYC3 Network Maintenance

26 November 2024 at 22:00

Nov 27, 03:00 UTC
Completed - The scheduled maintenance has been completed.

Nov 27, 01:52 UTC
Update - Our Engineering team has decided to extend the maintenance window by an hour to monitor the situation.

Maintenance is still underway and we will update once the maintenance is fully completed.

We sincerely apologize for any inconvenience this may have caused and appreciate your understanding.

Nov 27, 00:01 UTC
Update - From 22:25 - 23:15 UTC, all event-related actions including creating or deleting resources, were temporarily disabled in NYC3.

Users attempting any event-based operations across products hosted in NYC3 would have encountered error messages. While we anticipated event failures and delays, this impact period was longer than expected. We are taking steps to ensure this does not happen in future maintenances.

As of now, maintenance is still underway and we will update once the maintenance is fully completed.

We sincerely apologize for any inconvenience this may have caused and appreciate your understanding.

Nov 26, 22:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Nov 24, 22:03 UTC
Scheduled - Start: 2024-11-26 22:00 UTC
End: 2024-11-27 02:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC3 region.

Expected impact:

During the maintenance window users may experience delays or failures with event processing for a brief duration on Droplets and Droplet-based services including Droplets, Managed Kubernetes, Load Balancers, Container Registry, and App Platform. We will endeavor to keep this to a minimum for the duration of the change.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

SFO3 Network Maintenance

25 November 2024 at 17:00

Nov 25, 22:00 UTC
Completed - The scheduled maintenance has been completed.

Nov 25, 16:30 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Nov 23, 16:35 UTC
Scheduled - Start: 2024-11-25 16:30 UTC
End: 2024-11-25 22:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the SFO3 region.

Expected impact:

During the maintenance window users may experience delays or failures with event processing for a brief duration on Droplets and Droplet-based services including Droplets, Managed Kubernetes, Load Balancers, Container Registry, and App Platform. We will endeavor to keep this to a minimum for the duration of the change.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

Droplet creates in NYC3

22 November 2024 at 12:44

Nov 22, 17:44 UTC
Resolved - From 09:05 to 09:50 UTC (November 21), our Engineering team observed an issue with Droplet creation.

During this time, users may have experienced intermittent errors while creating the Droplets via the Cloud Panel and API. Users should no longer be experiencing these issues.

We apologize for the inconvenience. If you have any questions or continue to experience issues, please reach out via a Support ticket on your account.

Customer Support Ticket Portal

25 November 2024 at 14:20

Nov 25, 19:20 UTC
Resolved - Given the absence of outages for the Support Portal, we will now resolve this incident. Our Engineering team will continue to work with our vendor to ensure continued stability of this service. If we observe further outages, we will communicate those to our users via new updates on our status page.

We sincerely apologize and thank you for your patience as we worked through this issue. In case of any questions or concerns, please open a ticket with our Support team.

Nov 25, 15:56 UTC
Update - Our Engineering team continues to work with our vendor to find a solution, but we have not observed any further outages of our Support Portal over the last 48 hours. We will provide additional updates as information becomes available. If an outage does occur, users may still contact us via the form here: https://www.digitalocean.com/company/contact/support

Thank you for your patience. We will post an update as soon as more information becomes available.

Nov 22, 03:49 UTC
Update - Our Engineering team continues to work with our vendor in order to resolve the issue with users accessing the Support Portal. The issue has been escalated and we are awaiting feedback.

As a reminder, if users need to reach out, they can utilize the form here as a workaround:

https://www.digitalocean.com/company/contact/support

Thank you for your patience. We will post an update as soon as further information becomes available.

Nov 21, 15:40 UTC
Investigating - Our Engineering team has observed issues with our Support Portal. During brief, recurring outages, users are unable to access the Support Portal at https://cloudsupport.digitalocean.com/. We are aware of these events and our engineers are working with our vendor to resolve this.

As this issue is intermittent, we expect users will be able to log support tickets normally. However, during any periods of unavailability, as a temporary workaround, users may still contact us via the form here: https://www.digitalocean.com/company/contact/support

Thank you for your patience. We will post an update as soon as further information becomes available.

The Philippine army is recruiting young tech civilians to fight cyber attacks

21 November 2024 at 06:00
Earlier this year, the Philippine Army put out an unusual call on Facebook, inviting civilian hackers to join its cybersecurity unit. “We have a greater enemy that wants to devour...

© Photography by Iya Forbes for Rest of World

Recruits of the Philippine Army’s Cyber Battalion are seen working on computers at their headquarters on Monday, November 18, 2024, in Manila, Philippines.

User Creation for PostgreSQL Managed Databases via Cloud Control Panel

19 November 2024 at 20:54

Nov 20, 01:54 UTC
Resolved - Our Engineering team has confirmed that the issue with displaying users for PostgreSQL Managed Databases across all regions has been fully resolved.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Nov 20, 01:34 UTC
Monitoring - Our Engineering team has rolled out a fix to resolve the issue with displaying users for PostgreSQL Managed Databases across all regions.

Users should no longer see errors while creating new users for PostgreSQL Managed Databases using Cloud Control Panel.

We will continue to monitor this at our end and will share an update once the problem is resolved completely.

Nov 20, 00:50 UTC
Investigating - Our Engineering team is investigating an issue with displaying users for PostgreSQL Managed Databases across all regions.

During this time, users are unable to see existing users in the Cloud Control Panel. Additionally, when creating users in the Cloud Control Panel, an error will be displayed, despite the user being created successfully.

Creating and listing users for PostgreSQL clusters via the API remains functional.

We apologize for the inconvenience and will share an update once we have more information.

Spaces Object Storage, DigitalOcean Container Registry & App Platform

18 November 2024 at 14:33

Nov 18, 19:33 UTC
Resolved - Our Engineering team has confirmed complete resolution of the issue that was impacting Spaces Object Storage, DigitalOcean Container Registry, and App Platform, across all regions.

From 16:52 UTC - 18:41 UTC, users may have experienced increased error rates when accessing Spaces objects, interacting with the DigitalOcean Container Registry and while creating/deploying Apps with App Platform. Functionality is completely restored and all operations are succeeding normally.

If you continue to experience any issues with these services please submit a ticket to our customer support team for assistance. Thank you for your patience.

Nov 18, 19:07 UTC
Monitoring - Our Engineering team has completed the mitigation efforts and a fix is now deployed in all regions.

As of 18:41 UTC, error rates have returned to normal and users should no longer experience issues accessing, creating, or performing updates on their existing services with the impacted products.

We are monitoring the situation and will provide an update shortly.

Nov 18, 18:32 UTC
Investigating - Our Engineering team is investigating an issue impacting Spaces Object Storage, DigitalOcean Container Registry, and App Platform, across all regions.

Beginning at 16:52 UTC, users may have started experiencing 500 errors when trying to access and perform operations with their Spaces bucket, as well as increased failure rates when pushing to Container Registries and creating/deploying Apps.

Our team has started to apply mitigations to correct the issue, which are rolling out across all regions.

We'll provide another update once that process has completed.

DNS Resolution in Multiple Regions

18 November 2024 at 16:42

Nov 18, 21:42 UTC
Resolved - Our Engineering team has confirmed that the issues with Authoritative DNS resolution across multiple regions has been fully resolved. DNS queries should now be resolving normally.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Nov 16, 22:52 UTC
Update - We are continuing to monitor our Authoritative DNS Service and have seen improvements since implementing the fix. The engineering team is waiting for full confirmation that the issue is corrected before we resolve this incident completely.

If you are facing any issues with our Authoritative DNS Service, please reach out to our support team.

Nov 15, 22:05 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issues with Authoritative DNS Resolution across multiple regions and is currently monitoring the situation.

We will continue to monitor this at our end and will share an update once the problem is resolved completely.

Nov 15, 11:12 UTC
Identified - Our Engineering team has identified the cause of the issue impacting Authoritative DNS Resolution across multiple regions and is actively working on a fix. We will post an update as soon as we have more information.

Nov 15, 08:33 UTC
Update - We are actively investigating the ongoing issues with Authoritative DNS Resolution across multiple regions.

We have engaged multiple teams to investigate the issue, and we apologize for the continued inconvenience. We will provide additional updates as soon as more information becomes available.

Nov 15, 04:03 UTC
Investigating - Our Engineering team is investigating an issue with intermittent failures and increased latency in DNS resolution across multiple regions.

During this time, users may experience issues with DNS resolution, or see errors returned when querying DNS records which are hosted on the DigitalOcean authoritative DNS infrastructure.

We apologize for the inconvenience and will share an update once we have more information.

❌