User Sessions Are Terminated Unexpectedly
Incident Report for PagerDuty
Postmortem

Summary

Between Sunday, 16 April 04:00 UTC and Monday, 17 April 17:00 UTC, some customers experienced irregular session durations, causing them to get logged out, sometimes repeatedly.

What Happened?

One of our servers got in a state that caused it to give out invalid session cookies. This resulted in customer web sessions to end prematurely which required users to repeatedly log in again. After we identified which server was in this state, we remediated the problem by restarting the server.

During the incident, no other systems were impacted and notifications went out according to our SLA.

What Are We Doing About This?

We have taken specific measures to prevent this from happening again and will flag it earlier should it reoccur. We are also reviewing our deployment strategy for this application to prevent it from causing irregular behavior in individual application servers.

We regret if this affected your team’s ability to receive alerts in a timely manner. As always, if you have any questions or concerns, feel free to contact us at support@pagerduty.com.

Posted 7 months ago. May 04, 2018 - 16:16 UTC

Resolved
We believe that the issue is isolated although we have yet to determine a root cause. Our investigation is ongoing. Please contact our Support team at support@pagerduty.com if your teams continue to report this behavior.
Posted 7 months ago. Apr 17, 2018 - 05:39 UTC
Update
We believe that the issue is isolated although we have yet to determine a root cause. Our investigation is ongoing. Please contact our Support team at support@pagerduty.com if your teams continue to report this behavior.
Posted 7 months ago. Apr 17, 2018 - 05:37 UTC
Investigating
We are investigating an issue affecting users where they are unexpectedly logged out of PagerDuty. Logging back in will allow your teams to continue their work. Notifications are not impacted.
Posted 7 months ago. Apr 16, 2018 - 21:20 UTC