Navigate Incident Management Like a Pro: MyFitnessPal's Sr. Director of Engineering Shares Insider Strategies with Lee Atchison
How much time are engineering teams spending on incidents?
Are you trying to set your engineering team free to do their best work? Read our new case study to learn how Blameless can help you do that.

Understanding the 5 Incident Severity Levels

Myra Nizami
|
3.15.2022

Wondering about severity levels? We explain what incident severity levels are, how to classify them, and how they will affect your incident management process.

What are severity levels?

Incident severity levels are the measure of the impact an incident will have on a system. In general, a lower number severity level, such as SEV-1, denotes a higher impact on the system.

Severity levels are one of the starting points of an incident management response because they serve as a way to classify the incident. Having clearly defined severity levels alongside clear workflows, runbooks, and other tools linked to each severity level will resolve incidents while minimizing negative impact. Additionally, having severity level definitions outlined ahead of time in a classification system ensures that teams understand any incident's impact right away and how much time it may take to resolve the issue.

Putting together severity level definitions

While every incident response plan should include severity levels, the actual definitions tend to vary, as do security level classifications. The first step in the process is to map out the different types of incidents and how bad each one would be. 

What would a minor outage look like, and what would constitute a major outage? Defining what types of incidents could occur helps teams classify severity levels. Think about what kinds of incidents require more resources and bandwidth from the team versus more minor incidents that teams can resolve quickly with minimal disruption. Another key consideration is how much an incident will impact user satisfaction. An unpopular service having a total outage might be less severe than a small outage for a popular service.

Creating severity level classifications

Now that incidents and severity levels have been designed, the next step is to classify them so that service requests are tagged accordingly. Organizations tend to set up their classifications in various ways. For example, you might see that some have severity levels going from 1-3, and incidents are classified accordingly. Think about critical workflows and the threats across each part of the workflow and how those would be classified.

The most common type of severity level classification is SEV-1 up to SEV-4, defined against business impact. When an incident is logged, it’s classified into a severity level accordingly. This helps the team understand what to prioritize and the service level attention needed. As they continue to work on the issue, teams can change the severity level higher and lower as more information appears.

Examples of severity levels

SEV-1:

System has a critical issue where most customers are actively affected, which could include functionality issues, or customer data leaks

SEV-2:

Critical system issue that affects how customers use the product, including app unavailability, notification issues, severe performance issues

SEV-3:

Stability or performance issues that aren’t critical but require immediate attention, such as losing functionality on some parts of the product.

SEV-4:

Minor issues in the app that do not actively impact customers but need to be addressed, such as performance or speed issues, cosmetic issues, and bugs.


The example above is just one way to do severity level classifications. Some organizations may choose to have up to 5 severity levels, while others prefer to keep it simple. Defining and classifying incident security levels will largely depend on factors such as on-call team, customer activity time, incident frequency, and internal resources. You can find more resources for incident classification in our previous blog post.

How do severity levels work in practice?

Once incident severity levels are classified and defined, teams can work together to implement them and see how it’s working out and whether there is alignment on the definitions and classifications. As part of the process, teams will also need to define incident priority and how that relates to incident severity to understand their workloads better.

Incident management tools are used to monitor issues, proactively classify incidents, and automation is set up across each part of the process. For example, incident response notifications can be sent to team members immediately so that everyone has access to the same information. In addition, teams can set up runbook automation to manage the incident depending on the incident itself. Finally, for severe issues where manual intervention is needed, incident management tools help teams stay on top of work and communication. Once the incident is resolved, build retrospectives to drive systemic improvement. The more severe the incident is, the more attention it should be given in retrospectives, to help ensure that the incident doesn’t happen again.

Why do severity levels matter?

As incidents are inevitable, a strong incident management process is essential to maintain customer happiness and business value. Having a protocol in place to prioritize and respond to incidents ensures that customers get the best service delivery possible, which helps the business make more money in the long run. And for teams, severity levels establish a transparent working process and a framework for impact measurement against which they can work against. As a result, there is far more transparency around incident management and less pressure for teams overall. 

Defining and classifying severity levels are useful so that everyone on the team has access to the same information, and they know what to prioritize and what to focus their efforts on. Not all incidents require the same amount of attention and panic – some are worse than others, and severity levels help tell things apart. Higher severity levels correspond with more disruption, outages, and losses. Lower severity levels denote incidents that won’t cause a hugely negative impact on the business. 

How can Blameless help?

Once your incident classification systems are put in place, Blameless is there to make sure that you’re making the best use out of them. Blameless offers tools across different parts of incident management to help teams work together better to resolve incidents faster. That includes SLOs and error budgeting to understand how incident severity impacts customers. 

Blameless also has features such as role-based incident checklists to standardize incident response without needless delays. Blameless also enables teams to facilitate communication through features like war rooms and role assignments based on incident severity. And with Blameless incident retrospectives, all data and communication are actively scribed during the incident resolution process into an automated timeline that teams can review. Schedule a demo with one of our SRE experts and learn how it helps make incident management a smoother process. Make sure to subscribe to our newsletter for more insights and articles. 

Resources
Book a blameless demo
To view the calendar in full page view, click here.