(Ab)use of the R Language, Part 2: latencylens
Most graphs we see as SREs are time based along the X axis, whether it is request rates, error rates, latencies or some other parameter along the Yaxis.
Type Conversion
Type Conversion: What Changes (and What Doesn’t) When You Start a New SRE Job
Pilots have a specific term for moving from one airplane to another: a “type conversion”. Not “learning to fly” again — you already know how to fly. It’s the process of taking everything you already know and re-mapping it onto a new machine that does the same job with a different cockpit.
Continue reading “Type Conversion”
AI and SRE
AI and SRE: The Same Force Cuts Both Ways
I’ve spent most of my career in the “in-between” role — the one that sits between software engineering and operations, making sure the systems that carry the weight of the business stay up, stay fast, and stay honest about what they’re actually doing. Listen, follow the data, don’t do anything you can’t undo quickly. That’s been the job for a couple of decades.
In the last year, the job changed more than it did in the previous ten.
Continue reading “AI and SRE”
openssl wrapper script
openssl is a powerful utility, but how often do you need to look up the options when you want to use it?
Anagrams for Fun
During the recent Holidays I visited some friends who are avid Scrabble players. After the visit, I thought it would be fun to write a simple anagram script.
Good Luck and Happy New Year!
The next time Friday 13th falls in January is in 2034.. Good luck!
K8S Tips and Scripts #2
I’m a big believer in “wrapper scripts”.
K8S Tips and Scripts #1
I’ve often come across situations where pods won’t die properly. Either the underlying node is in a strange state, temporary networking issues, whatever. If a node has gone south, pods will often stick in an “Evicted” state for example.
Where Are We Going?
“Where are we going?” my wife asks as we leave the runway on her first flight with me after I received my pilot license. Continue reading “Where Are We Going?”
Finding the Information You Need
Nothing sucks enthusiasm for a job faster than a frustrating search for information you need to do the job that turns up empty or irrelevant. Even worse is when the information you find is outdated or even lies to you!
Thoughts on Documentation
When I worked at RIM (Blackberry), we had the luxury of having technical writers on staff. They were brilliant at yanking information out of engineering and organizing it into cohesive sets of documentation. Not all companies or departments can afford a good technical writers on staff, so we all need to pitch in and do it. What constitutes good technical writing?
K8S Tips and Scripts #0
It’s time to start writing again.. It’s been awhile..
I’ll start by sharing some tips I’ve learned and scripts I’ve written for dealing with Kubernetes.
Implementing Service Level Objectives
I picked up a new book recently that is a great companion to the other SRE books published by O’Reilly. Continue reading “Implementing Service Level Objectives”
NOTAM for SREs
In aviation, NOTAMs are “Notices to Airmen” for conditions that are generally temporary and hence not information published in the usual places. Continue reading “NOTAM for SREs”
Command Line Interface Guidelines
Anyone who knows me, knows that I am most comfortable and at home on the unix/linux command line. Continue reading “Command Line Interface Guidelines”
The Tail at Scale Approximation
This article demonstrates a quick and easy approximation for the probability formulae which I described in two previous articles. Continue reading “The Tail at Scale Approximation”
Riddler: Can you solve the not-so-corn maze?
I love AWK and I’ve written about it before… Continue reading “Riddler: Can you solve the not-so-corn maze?”
The Tail at Scale Revisited
My last article discussed some of the missing math related to setting back-end objectives. This article presents a chart which is useful in understanding the relationship to the user experience and we examine ways to dramatically improve the overall performance. Continue reading “The Tail at Scale Revisited”
The Tail at Scale
The landmark “Tail at Scale”[1] article was missing some of the math. We’re diving into it a bit here to show how the math can be used in setting objectives for latency budgets in back end systems. Continue reading “The Tail at Scale”
Hiring Questions, Problem 3
This was an interesting question, so I thought I’d share it here.. Continue reading “Hiring Questions, Problem 3”
Remote Work
Many companies and individuals are contemplating remote work now. Embrace it! Continue reading “Remote Work”
What is SRE?
The current state of confusion around what a “Site Reliability Engineer” (SRE) role is..
Continue reading “What is SRE?”
BPF Performance Tools
BPF is one of the Swiss Army Knife tools for Performance Engineering on Linux. Continue reading “BPF Performance Tools”
Event Logs and A.I.
Many companies in the logging/monitoring space will try to sell you on AI and ML (Artificial Intelligence and Machine Learning) to find abnormal. Continue reading “Event Logs and A.I.”
Event Logs and K.I.S.S.
I’ve worked with event logs for, well, decades. There are quite a few companies that offer services for managing logs and, afaik, only a few doing it right. Continue reading “Event Logs and K.I.S.S.”
SPOFs and Partial Panel
In both aviation and systems we build in redundancies wherever practical to avoid unpleasantness when components or subsystems fail. Continue reading “SPOFs and Partial Panel”
Traffic At 2 O’clock!
Up in the air, your eyes can’t be everywhere, all the time. You’re trained to scan the skies for “traffic” (other flying machines) as well as scanning instrumentation in the cockpit. Continue reading “Traffic At 2 O’clock!”
Own It !!
We were heading back from the practice area to the airport. I didn’t have my pilot license yet and my instructor says: “Push the throttle to Rental Speed!”. Continue reading “Own It !!”
Systems and Gardening
It’s been awhile since I’ve written. I’ve been busy coming up to speed as an SRE with an awesome new team!
It’s gardening season up here in the Northern Hemisphere and while I was dealing with some trees and bushes that had died with a recent ice storm, I thought about the similarities in dealing with systems. Continue reading “Systems and Gardening”
Reading Week #5
Large numbers are difficult to comprehend. The national debt for example is kind of mind numbing. The fun article this week is about insane numbers; imagining the unimaginable.. Continue reading “Reading Week #5”