Navigate Incident Management Like a Pro: MyFitnessPal's Sr. Director of Engineering Shares Insider Strategies with Lee Atchison
How much time are engineering teams spending on incidents?
Are you trying to set your engineering team free to do their best work? Read our new case study to learn how Blameless can help you do that.

Promoted to SRE Advocate: A Dream Turned Reality

Matt Davis

I get chills thinking about a line from the first film adaptation of Roald Dahl's Charlie and the Chocolate Factory, Gene Wilder as Wonka nearly whispers it to Charlie, as if it is secret information: We are the music makers, and we are the dreamers of dreams.

For me, the quote (taken from a poem by Arthur O'Shaughnessy) is austere: We are the creators of what we create, and what we create becomes what we are. Both our intelligence and emotions speak when we do things, the creative spirit of humans is why Craft and Expertise exist at all. It is reflected in our art, our writing, our music, and even in our making technology to realize our dreams.

To put this in the language of SRE, I might say: We are the flexibility at the sharp end, and we are the sources of resilience. This describes the music I feel the most in our work at Blameless. It has a sound of Diversity and Trust, of Empathy and Reciprocity, of Vulnerability and a Just Culture.

To be more involved with helping both our customers and ourselves do very difficult work in complexity and evolve SRE, I am taking on a new role at Blameless as SRE Advocate. Let's talk about why we've created this Role, why I am stepping up, and how a trained opera singer ended up knowing a lot about Reliability.

Why do we believe in SRE?

In operating complex systems there are diverse streams of growth that gurgle together into a river of Reliability. They become interdependent entities, yet contain the ability to Adapt and change the course of their flow. This is called Complexity, and the "Adapt" part? That's us! Resilience is People. Developing the human experience is what we mean by Continuous Improvement, and that is what's happening to the field of SRE. Human relationships are being brought to the fore, emphasized as the essential bit of reaching big Reliability dreams.

What excites me the most about advocating this is how we can collectively improve areas of Systems Robustness, Resilience, and Reliability Engineering. Today we call it "SRE" by a kind of folksonomy, communities gathering behind an easy-to-recognize term and tagging it with symbolism: this is difficult to define because it deals with complex systems and human beings. It started as guidelines inside one company, but its meaning has grown to encompass the Emergence of us all.

Operating Complexity requires Humans to effectively Collaborate, Coordinate, and Communicate. To do this, we build Trust and Agency in ourselves and each other. We exhibit Common Grounding in our work. We share and examine our Sharp-End and how it supports sources of Resilience. This is SRE as a Practice and it is special: we believe it is evolving the practice of running software in our increasingly complex world.

Why have I made the change?

A wise woman recently clued me in: an Advocate can be compared to an Opera Conductor. As I consider this more, I realize how on-the-nose she was. The Conductor is someone who understands stage direction, but is not acting in the play. This human leads a group of other humans to connect with each other, but cannot prescribe how to play each part. They know the orchestration rules of every instrument, but may only know how to play a small number of them. Understanding the fach of each singer and how that voice type affects the dynamics of the whole does not require them to be a singer at all.

The Opera Conductor does a lot more than just demarcate actions; they have a job to understand the music and libretto to their creative core, to interpret sound and action for the audience, to Educate as well as Guide. This requires knowing how each part incorporates as a whole, while allowing the whole to be fluid and emergent. The Opera Conductor is fluent in the language of Opera.

My job as an SRE Advocate draws on my strengths as someone fluent in the language of Socio-Technical Engineering. Because I have such a fondness for complex systems like an opera performance, I found myself attracted to the systems thinking side of our industry. Expertise in improvisation prepared me for entering the world of Resilience Engineering in a way that surprised me. So while I climbed the ranks of Rack Monkey and NOC Nerd to reach SRE, I have always found my skills in Conductor Mode more pronounced, more a part of who I am.

Thus, my study of being in musical systems led to my expertise in technical ones, my abilities synthesized like an Opera Conductor. Understanding sharp-end directions, not prescribing them. Leading a group of humans in the practice of building empathy, allowing their agency and creativity to flourish. I know the orchestration rules of deployment, but have no ability to deeply internalize them all. I also know a bit about individual autonomy and how personality types and their cognitive challenges fit the larger whole.

True story, I failed my only Computer Science class in undergrad. I was a touring musician, radio DJ, event coordinator, and studying to be an opera singer! Nevertheless, my attraction to hacking hardware meandered into an immersive career of Tech instead of Opera, leading me across diverse disciplines. My story is one of experiential learning: I swung to management twice and back, became a storage and data expert in AdTech, reinvented my hardware career into a cloud one, and attempted my best at software engineering. I finally feel confident that Education and Advocating fits my brain the best of all.

Resilience is what I sing!

My journey into SRE feels like a series of reaching for dreams that I didn't know I had until they looked back at me and asked why I wasn't coming along for the ride. I seem to have just barely caught the edge of the last train car, swinging an arm up to a friendly face in the back, hoping I can join the band.

Aphex Twin also fancies the Willy Wonka quote, a favorite downtempo track of mine called "We are the music makers" samples the film. Maybe like me, he finds Wilder's voice a calming reassurance. "We are the dreamers of dreams" signifies an acceptance of emergence, ambiguity, discovery, and adaptation. It's the uncertainty and promise of a dream that keeps us going.

But listen, I did not fall out of bed last month and decide I wanted to do SRE advocacy work. It's been a latent dream since I started doing talks as an SRE, when I realized it is Music that helps me get far in Tech. I had managed to marry the two things I loved the most: performing on-stage and helping others learn. For instance, my trilogy about The Practice of Practice goes like this:

So far in 2022 I have been honored to appear in several podcasts and interviews. One of my favorites is from on how we can shed light on reducing toil. We also talked with DevOps Paradox on Why Incidents Are Slowing Down Companies. In our locally produced podcast-style webinar called "From Theory to Practice'', a member of the Blameless team joins me with two guests from different parts of the industry to talk about overcoming difficulties in Regular Work. We discuss what things are like at the Sharp-End and keep it real, we ask questions like "What's Difficult About Tech Debt?".

We are turning up our SRE practice at Blameless to 11. We have more than doubled the size of our Infra/SRE team over the past nine months, it has been a rush of new energy and diversity. Onboarding these new SREs quickly has been instrumental in creating force multipliers of efficiency, allowing us to perform sweeping upgrades and component improvements. It has also allowed me to shift my focus to areas of cross-functional work, like the On-Call / Incident Command reboot we're currently undergoing. The story of why we needed such a reboot and the challenges we face as we enable it is fascinating… and I cannot wait to share it with all of you.

Book a blameless demo
To view the calendar in full page view, click here.