Semgrep App is experiencing an outage
Incident Report for Semgrep
Postmortem

On Monday, March 11th Semgrep Cloud Platform experienced a large, unexpected increase in traffic that uncovered a misconfiguration in one of our web services. This misconfiguration meant that we were not caching static data, resulting in our web service handling 10-15x its normal traffic. As traffic was spiking, the web service hosts were unable to handle the dramatically increased load, and began to crash loop. After configuring caching on our static data, our services quickly exited their crash loops and began to operate as normal.The Semgrep Engineering Team has performed a postmortem since the incident and have determined a number of action items that will help prevent and mitigate issues like this in the future.

  1. Configure all static data to be cached through our CDN
  2. Implement improved metrics and alerting at multiple levels of our infrastructure
  3. Revaluate resource allocation for our critical web services
Posted Apr 26, 2024 - 15:57 UTC

Resolved
All traffic appears to be operating normally now that the fix has been applied
Posted Mar 11, 2024 - 19:02 UTC
Monitoring
A fix as been applied and the team is monitoring and starting to conduct a post-mortem
Posted Mar 11, 2024 - 18:38 UTC
Update
We are continuing to investigate this issue.
Posted Mar 11, 2024 - 18:38 UTC
Investigating
Hello all,

We've seen a spike in errors on the Semgrep API backend. The team is investigating and working on a fix. We apologize for the inconvenience.
Posted Mar 11, 2024 - 17:42 UTC
This incident affected: semgrep.dev (Login/SSO, Dashboard, API, Registry, Scans and CI/CD).