Degraded performance to multiple Atlassian experiences

Incident Report for Compass

Postmortem

SUMMARY

On September 22, 2025, between 04:38 and 04:48 UTC, Atlassian customers experienced connection errors preventing access to their atlassian.net sites. Some customers observed intermittent errors as services gradually recovered until 06:17 UTC.

The event was triggered by a faulty configuration change to our Content Delivery Network (CDN). The change included an invalid hostname which prevented customers from successfully connecting to atlassian.net domains.

The incident was detected within 1 minute by our monitoring systems and mitigated by rolling back the change, which put Atlassian systems into a known good state. The acute impact was resolved in 10 minutes and all lingering errors resolved in 1 hour 38 minutes.

IMPACT

The acute impact occurred on September 22, 2025, between 04:38 UTC and 04:48 UTC to Confluence, Compass and Jira, including Jira Service Management. The incident caused service disruption to customers when they attempted to load those products in their browser or interact with APIs. Between 04:48 UTC and 06:17 UTC some customers continued to observe intermittent errors as services gradually recovered.

ROOT CAUSE

The issue was caused by a change to the hostname configuration of our Content Delivery Network (CDN). As a result, the products mentioned could not receive connections, and the users received TLS handshake errors, followed by HTTP 503 and 403 errors.

More specifically a new CDN configuration contained a resource name which conflicted with the existing customer-serving CDN resource, and was able to be deployed, overwriting it. The root cause of the incident was the failure in the detection of the bug by our pre-deployment validations and tests.

REMEDIAL ACTIONS PLAN & NEXT STEPS

We know that you rely on our products for your daily operations and productivity, and we sincerely apologize for any impact this disruption has had on you, your team or your organisation.

We are prioritizing the following improvement actions to help avoid repeating this type of incident:

  • Improved pre-deployment change controls and testing.
  • Improved validation of target configurations before deployment to ensure customer-serving CDN resources cannot their hostnames changed.
  • Sharding of our CDN configuration to enable progressive changes.

We sincerely appreciate your understanding and patience as we improve our processes to provide a better customer experience.

Thank you,

Atlassian Customer Support

Posted Oct 01, 2025 - 06:24 UTC

Resolved

A configuration change to our CDN configuration for the atlassian.net domain resulted in customer traffic temporarily being dropped. This was immediately identified by our monitoring systems and rolled back. A public PIR with full details will be issued as a follow up.
Posted Sep 22, 2025 - 07:17 UTC

Monitoring

Our team has investigated and enabled configuration changes which should be restoring services across all impacted products starting at 06:17AM UTC.
We are monitoring for recovery across all products and services at this time, and will provide further update when available.
Posted Sep 22, 2025 - 06:55 UTC

Identified

Our team is performing an investigation into all Atlassian products and services to understand the full impact of the prior certificate changes that caused this incident.
While many services are restored we are continuing to monitor for additional impacted service issues that customers may be experiencing.
Posted Sep 22, 2025 - 05:59 UTC
This incident affected: API (GraphQL, REST), Integrations (Bitbucket, Gitlab, CircleCI, JSM, JSW, NewRelic, Opsgenie, PagerDuty, Slack, SwaggerUI, Statuspage, Github), Notifications (Webhooks, Email), Catalog (Component Templates, Components, Search, Activity Timeline), Health (Scorecards, Metrics, CheckOps), 3rd Party Services (GitHub API Requests, GitHub Webhooks, CircleCI CircleCI Webhooks), Operations (On-call flows, Alert flows), and Authentication and User Management, Purchasing & Licensing, Signup.