Normal view

Managed Kubernetes Connectivity with Internal Droplets

Mar 10, 21:49 UTC
Resolved - From March 1st 12:00 UTC to March 10th 18:43 UTC, a subset of users experienced issues connecting Managed Kubernetes nodes to Droplets as a source to allow traffic, due to an issue with tags being applied internally. Some users may also have experienced issues adding Managed Kubernetes Clusters as a source in Cloud firewalls with related Droplets.

Our Engineering team has confirmed full resolution of the issue. Users should now be able to connect Managed Kubernetes nodes to Droplets and should also be able to add Managed Kubernetes Cluster as a source in the Cloud Firewall with related Droplets.

If you continue to experience problems, please open a ticket with our support team from within your Cloud Control Panel.

Mar 10, 14:38 UTC
Identified - Our Engineering team has identified the cause of the issue affecting recently created Managed Kubernetes nodes. These nodes are currently unable to connect to Droplets that use the Managed Kubernetes Cluster as a source to allow traffic.
Additionally, users who add a Managed Kubernetes Cluster as a source in the Cloud firewall may experience issues with related Droplets not being added to the firewall as expected.

As a temporary solution, users can continue to add the tag k8s:CLUSTER_UUID, where CLUSTER_UUID is the UUID of the user's Managed Kubernetes Cluster. The UUID can be found in the Cloud Panel URL of the Managed Kubernetes Cluster or by running the following command:

doctl kubernetes cluster list

We are working on a solution and will provide further updates as we progress. We appreciate your patience and understanding.

Mar 10, 12:03 UTC
Investigating - Our Engineering team is currently investigating an issue impacting recently created Managed Kubernetes nodes not being able to connect to Droplets that utilize the Managed Kubernetes Cluster source to allow traffic.

Users adding a Managed Kubernetes Cluster as a source in the Cloud firewall may see related Droplets not added to the Cloud firewall.

As a workaround users can add the tag "k8s:CLUSTER_UUID" where "CLUSTER_UUID" is the user's Clusters UUID.
Users can find the UUID details in the Cloud Panel URL of Managed Kubernetes Cluster or by using the command "doctl kubernetes cluster list"

We apologize for the inconvenience and will share more information as soon as it's available.

Managed Certificate

Mar 5, 23:11 UTC
Resolved - From 20:08 UTC to 21:49 UTC, users may have experienced issues creating new certificates, provisioning or scaling MongoDB clusters, and issuing certificates for use with LBAAS and CDN services. Additionally, a subset of new custom domains on the App Platform may have taken longer than usual to configure.

Our Certificate Authority has fully resolved the issue with our managed certificates service, and all services have returned to normal operation.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Mar 5, 22:07 UTC
Monitoring - Our Certificate Authority has implemented a fix for the issue with our managed certificates service. Services are beginning to recover, and we are closely monitoring the situation to ensure full stability.

Users should now be able to create new certificates, provision and scale MongoDB clusters, and issue certificates for use with LBAAS and CDN services. Additionally, new custom domains on the App Platform should start configuring normally.

We apologize for the inconvenience and will share another update once the matter is fully resolved.

Mar 5, 21:22 UTC
Investigating - Our engineering team is investigating an issue with our managed certificates service due to an unexpected outage from our Certificate Authority.

During this time, users may experience issues creating new certificates, provisioning or scaling MongoDB clusters, and issuing certificates for use with LBAAS and CDN services. Additionally, this may cause a subset of new custom domains on App Platform to take longer than usual to become active.

We apologize for the inconvenience and will provide updates as soon as we receive more information.

Colombia’s EV sales soar, but battery recycling startups struggle to keep pace

5 March 2025 at 06:00
For three years, Juan Vega sped across Cartagena, on Colombia’s Caribbean coast, on his electric motorcycle — one of thousands of recent converts to the environment-friendly option. When he needed...

© Photography by Santiago Mesa for Rest of World

Batteries ready to be repaired and brought back to life at the company Batx in Medellín, Colombia.

NYC3 Network Maintenance

Mar 5, 23:52 UTC
Completed - The scheduled maintenance for our core networking infrastructure in the NYC3 region, originally planned for March 5, 2025, from 23:00 UTC to March 6, 2025, 01:00 UTC, has been cancelled. No changes will be made at this time, and all services will continue to operate as usual.

We will provide an update when this maintenance has a new confirmed date. If you have any questions, please reach out to our support team via Cloud Support.

Thank you for your understanding.

Mar 5, 23:00 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 3, 16:48 UTC
Scheduled - Start: 2025-03-05 23:00 UTC
End: 2025-03-06 01:00 UTC

During the above window, our Networking team will be making changes to core networking infrastructure, to improve performance and scalability in the NYC3 region.

Expected impact:

During the maintenance window, users may experience brief delays or failures in event processing for up to 5 minutes. This may affect Droplets and Droplet-based services, including Managed Kubernetes, Load Balancers, Container Registry, and App Platform. Additionally, regular requests via the Public API, as well as viewing and submitting requests through the Cloud Control Panel, may time out or fail for up to 5 minutes.We will endeavor to keep this to a minimum for the duration of the change.

If you have any questions related to this issue please send us a ticket from your cloud support page. https://cloudsupport.digitalocean.com/s/createticket

Accessing Cloud Control Panel

25 February 2025 at 13:28

Feb 25, 18:28 UTC
Resolved - From 17:14 UTC to 17:34 UTC, users experienced issues accessing the Cloud Control Panel and API.

Our Engineering team has resolved the issue, and the Cloud Control Panel, API, and events should now be functioning normally.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Feb 25, 17:58 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issue impacting the Cloud Control Panel and API. Users should no longer experience issues with accessing the Cloud Control Panel and API.

We will continue to monitor the situation and post an update once this issue is fully resolved. Thank you for your patience.

Feb 25, 17:44 UTC
Identified - Our Engineering team has identified the cause of the issue impacting our Cloud Control Panel and API and is actively working on implementing a fix. We are already seeing signs of recovery, and users should now have access to the Cloud Control Panel and API.

We will post an update as soon as additional information is available.

Feb 25, 17:32 UTC
Update - We are continuing to investigate this issue.

Feb 25, 17:27 UTC
Investigating - Our Engineering team is investigating an issue impacting our Cloud Control Panel and API.

During this time, users will experience errors when trying to use the public API at api.digitalocean.com or while accessing our Cloud Control Panel at https://cloud.digitalocean.com. Users may also see issues with processing Droplet and Kubernetes cluster creations along with Droplet-based events.

We are now confirming the full impact. We regret any inconvenience and will let you know as soon as more information is available.

Droplet creates in NYC3

25 February 2025 at 15:16

Feb 25, 20:16 UTC
Resolved - From 6:00 UTC to 17:45 UTC, users have experienced issues with Droplet creation and Droplet-based services in the NYC3 region.

Our Engineering team has fully resolved the issue impacting Droplet creation in the NYC3 region. Users can now create Droplets and Droplet-based services, including Managed Kubernetes, Managed Databases, Load Balancers, and snapshots, without any issues.

If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

Feb 25, 18:43 UTC
Monitoring - Our Engineering team has implemented a fix to resolve the issue impacting the creation of Droplets in the NYC3 region. Users should now be able to create Droplets and Droplet-based services including Managed Kubernetes, Managed Databases, Load Balancers, and snapshots without any issues.

We are actively monitoring the situation and will post an update as soon as the issue is fully resolved.

Feb 25, 16:02 UTC
Identified - Our Engineering team has identified the reason behind the failed and delayed Droplet creates in the NYC3 region and implemented a partial mitigation to address this issue. As of now, customers may be able to create Droplets and Droplet-based services including Managed Kubernetes, Managed Databases, Load Balancers, and snapshots, however, there might be still some delay in the creates.

We apologize for the inconvenience and appreciate your patience as we work to resolve this issue. Further updates will be shared as they become available.

Feb 25, 12:45 UTC
Investigating - Our Engineering team is investigating an issue with Droplet creation in the NYC3 region. During this time, users may see an elevated failure rate for the creation of Droplets and other Droplet-based products, such as Load Balancers, Managed Databases, or Kubernetes in the NYC3 region.

We apologize for the inconvenience and will share an update once we have more information.

❌