Author: prefrontal

Slava Ukraini

Topeka, Kansas is a small city that can sometimes surprise you. One of the surprising bits was the ability to take four years of Russian language classes in high school. By some crazy random happenstance Topeka High School had Russian instruction going back to 1973.

The local Russian teacher, Mr. Lonard, went into the middle schools every year to drum up interest in the classes, making sure there were always a critical mass of students. The highlight of the program was a biennial exchange where high school students from Kansas would host high school students from the former Soviet Union. A few months later, students from Kansas would then be hosted in Russia or Ukraine.

I wanted to go. Sitting in that middle school classroom, I knew I wanted to go.

Fast forward three years. I was a junior in high school and on a plane to Kharkiv, Ukraine. I had been to Mexico once or twice while visiting family in San Diego, but this was a different scale of international travel. Having two-ish years of Russian classes had prepared me somewhat, but my conversational language wasn’t great and I was getting thrown in the deep end.

I was scared out of my mind.

My host brother Dimitri stayed with his maternal grandparents in a suburb of Kharkiv called Piatykhatky. This was done so he could attend the local secondary school, which was one of the best in Kharkiv. Piatykhatky was home to the former Ukrainian Physics and Technology Institute, now called the Kharkiv Institute of Physics and Technology. With such a concentration of scientific talent the local school had an amazing set of teachers.

As told by my Russian teacher, Piatykhatky was completely closed to foreigners while Ukraine was part of the Soviet Union. Now, a group of high school students from Kansas would call it home for six weeks – just four years after the Soviet Union collapsed.

I spent the majority of each week with Dimitri and his grandparents in their small apartment. We then spent the weekends with Dimitri’s parents and sister. They lived on an Army base on the outskirts of town.

I could go on for many pages about how wonderful Dimitri and his whole family were to me. There are dozens of stories to tell of Dimitri’s visit to Kansas as well. I could also go on for pages about how friendly the population of Kharkiv were to a bunch of wayward American high school students as we tried to navigate a large town half a world away from home.

The original exchange program between Topeka High and School 62 in Piatykhatky ended in 1998. There were rumors of corruption as the Director of School 62 allegedly solicited bribes during the exchange selection process. I was happy to see recent news that students from Ukraine are once again attending Topeka High School.

It is enough to say that I have been thinking about Dimitri and his family a lot lately. Russia invaded Ukraine and Kharkiv Oblast over two years ago. According to some reports, the invasion started at Piatykhatky, which is only 17 miles from the Russian border. Kharkiv, the city, has been under unrelenting artillery shelling and missile strikes ever since. School 62? The building where I took classes is reported to have been destroyed.

It is a cruel action taken against wonderful people, and a country that I have held in high regard for almost thirty years. 

I am looking forward to the war being over one day. I dream of returning to Piatykhatky after three decades away and walking the same streets I walked as a teenager. For now, this is an impossibility. It will have to be enough that my thoughts are with Ukraine and its citizens.

Slava Ukraini.

Director: Other duties as assigned

At some point in your career the job description becomes 100% “whatever it takes for your team and the company to be successful”. The people that get ahead are the ones that realize this fact early.

Steve Jobs is always a controversial character when evaluating management styles, but I have always loved this story regarding his expectations for VPs:

Steve Jobs told employees a short story when they were promoted to vice president at Apple. Jobs would tell the VP that if the garbage in his office was not being emptied, Jobs would naturally demand an explanation from the janitor. “Well, the lock on the door was changed,’ the janitor could reasonably respond. “And I couldn’t get a key.”

The janitor’s response is reasonable. It’s an understandable excuse. The janitor can’t do his job without a key. As a janitor, he’s allowed to have excuses.

“When you’re the janitor, reasons matter,” Jobs told his newly-minted VPs. “Somewhere between the janitor and the CEO, reasons stop mattering.”

“In other words,’ (Jobs continued,) “when the employee becomes a vice president, he or she must vacate all excuses for failure. A vice president is responsible for any mistakes that happen, and it doesn’t matter what you say.”

John Rossman, Think like Amazon

There is a lot of power in the Jobs story. As a senior leader you own it all – both successes and failures. The reasons for each stop mattering. My addition to that perspective is that ownership shouldn’t start at the VP level. You should be going above and beyond to deliver results, and owning all outcomes, wherever you are in your career journey. This trait is rare and puts you far above your peers in any industry.

I have had numerous direct and indirect reports that requested a kind of checklist for promotion. “Just tell me what to do and I will do it” is a common refrain. This is relatively easy to provide to recent grads up through Senior engineers. The expectations at these levels are more straightforward, almost mechanical. At the Staff+ levels it becomes much more difficult to create that kind of to-do list. This is because managers are indexing less on your technical knowledge and project completion rate. Instead, we are focusing more on your soft skills, including communication, collaboration, and ownership. Establishing strong ownership early shows a senior level of execution, accelerating your career growth.

This same ownership effect occurs within the management ladder. While software management is already a bit like four different jobs masquerading as one, at the Director+ levels is starts boiling down to “whatever it takes”. This explains why so many Director+ job postings are oddly vague, with terms like “create a healthy team culture”, “collaborate with product management”, or “ensure technical excellence”. As a job posting there has to be a job description, but how do you capture the reality of “responsible for everything”?

One of my personal Top 50 management books is Extreme Ownership by Jocko Willink and Lief Babin.  You can distill the entire book down to a single view: you always have some control over a situation, so stop giving excuses and pretending that there was nothing you could do. The individuals who get ahead quickly are the ones that internalize this fact early.

Photo: The Cessna 152 that I did most of my flight training in. I have a lot of fond memories in that plane, including my first solo flight. On any plane the pilot-in-command (PIC) is the individual directly responsible for and the final authority regarding operation of that aircraft. No excuses.

Aurora are epic as well

Space Weather Message Code: ALTK09
Serial Number: 6
Issue Time: 2024 May 10 2334 UTC

ALERT: Geomagnetic K-index of 9o 
Threshold Reached: 2024 May 10 2254 UTC
Synoptic Period: 2100-2400 UTC

Active Warning: Yes
NOAA Scale: G5 - Extreme

...

Aurora - Aurora may be seen as low as Florida to southern Texas and southern California.

The Space Weather alert by NOAA was no joke. Aurora could be seen in southern California with the naked eye, and the long-exposure photos were incredible. We had to head up into the hills to get above the marine layer in Santa Barbara, but it was totally worth it.

Total eclipses are epic

It took a lot of prep, but I got the photos I wanted of the 2024 total eclipse. It also helps to have luck on your side, which you can facilitate by taking 670 shots of the eclipse over several hours…

The weather in Dallas was right on the edge in the weather models, and it was totally overcast in the morning. By the time the eclipse started the skies were partly cloudy. When totality hit the skies were almost clear. What a wonderful experience to share with several hundred people in Turtle Creek Park. Definitely a bucket list item if you can make it happen in your lifetime.

Getting ready for the April 2024 total eclipse

Practice makes perfect. Will be in Dallas next month for the big event. Look at the sunspots!

Equipment: Canon 7D, 400mm f5.6 lens, 1.4x teleconverter, with a NiSi PRO Nano ND100000 neutral density filter. That provides the equivalent of a ~900mm lens with the APS-C sensor crop factor. I used an external HDMI monitor for positioning and focus since I really love having healthy retinas…

Have the bookshelf you wish the library had

Create a powerful collection of reference materials that help you understand and explore your craft. Keep building and iterating that collection over time.

I loved the library as a kid. Once a month or so my sister and I would talk our parents into a trip to the Topeka Public Library and we would spend hours browsing the stacks. Anything you were interested in – there was a book for it. I spent a lot of time in the nonfiction area checking out books on things like electronics, weather, and geology. I still remember the excitement of venturing out of the kids section for the first time and walking over to the regular collection. Thousands of new books and an entire new wing of the library to explore.

Fast forward to 2012. I was working as a firmware engineer at the time and happened to be visiting friends in Cambridge, MA. As we wandered around we stopped by the Harvard/MIT Coop bookstore. In the Coop I found one of the most amazing sights for a software engineer: an entire wall of O’Reilly technical books. These are the ones with bright covers and random print animals on the front. There were books for C++, Python, Data Engineering, Flask web services – you name it. In that moment I told my wife “this is the library I wish I had at home”. I bought a book, if only to remember the experience.

Now fast forward to today. Every September my wife and I go to the local book sale fundraiser for Planned Parenthood. Like kids in a candy store, we buy dozens of books every year. As I was shelving my new finds last fall I had to stop for a second. I realized that my collection of software and management books had finally iterated to the point where I very nearly had the perfect bookshelf – the bookshelf I wished the library had.

All my favorite authors. A ton of community favorites. A few stinkers, sure, but this bookshelf has served me well by opening my eyes to new perspectives and new challenges. With each book I have improved my engineering and my management. While most of the books I have finished reading, there are still a number that I haven’t opened up yet. I want the same reference library, and the same benefits, for everyone.

A quick thanks to Planned Parenthood of the California Central Coast for putting on their annual book sale in September. It has always been a treat. Thanks also to my current employer, who graciously gives every employee an enrichment stipend each year for professional growth. I spend it all on books…

Photo: two out of my eight shelves in the home office. Also seen: a dumpster fire plushie, a plastic airplane model, a Highway to the Danger Zone patch, an incomplete 3D print of a tricorder from Star Trek: The Next Generation, and a static light scattering flow cell from my days at Wyatt Technology.

Take your lemons and make unbelievably good lemonade

Use whatever advantages you can to create the best possible output. Be so good at navigating constraints that others think you are breaking the rules.

In college I was a photographer for the school paper, the University Daily Kansan. I loved photography and saw the journalism gig as a great way to get better. I started out doing speaker and podium shots. If someone was giving a talk on campus then I was usually there, getting a photo. It was actually a fabulous exercise. Sure, you could take the same boring shot dozens of times a month and get the job done. Alternatively, you could use it as an exercise to get creative. I tried to do the latter.

Over time I got to be a better photographer and I earned trust with the photo editors. My next evolution was sports, covering the football team (KU wasn’t known for great football) and occasionally covering basketball. To say that it was a treat to sit on the court covering a game at Allen Fieldhouse is putting it mildly…

In November of 2000 a huge gig came up. The paper was sending two reporters and one photographer to New York to cover the Coaches vs Cancer Classic basketball event (now called the Empire Classic). I got tapped to go since I had taken the time to learn the new Nikon digital camera system and could send photos back for publication same-day. I hopped on the plane with the reporters and got to sit on the court at Madison Square Garden to shoot the Jayhawks playing UCLA and St John’s.

The lighting conditions weren’t great at the venue. The professional photographers that surrounded me all had powerful strobe flashes linked in the ceiling to get the light they needed. The strobes provided enough light to freeze the action, making basketball shots easy. All I had was an ƒ2.8 70-700mm lens to get the job done. I had other tricks up my sleeve though. First, I knew how far I could push the ISO of the camera without getting too much noise, meaning that I could get good light sensitivity (and faster shutter speeds). I also had shooting techniques that helped, like following a moving target through the longer exposure to help keep the player centered on the sensor. This reduced blur. In the end, I got some decent shots that went onto the front page of the Kansan as the basketball team won the event championship.

When I got back to Kansas my photo editor called me into his office. “Hey, nice work on the Coaches vs Cancer event. That went really well. By the way, did you heavily Photoshop any of your photos? The photo editor at another paper says they look way over-edited.”

I was shocked. Event photography is all about capturing the moment as it happened. Here I was, being accused of bending reality. With a bit of anxiety I went to the nearest workstation and showed my editor the workflow from the raw image. First, crop the shot. Next, adjust the exposure curve with one click. Finally, sharpen the image and ship it.

My editor was satisfied with that. I head rumors that he called the other editor up and said he was full of it. Ultimately, the reason my photo was better was exactly because of my disadvantages.

When strobes fire they tend to illuminate everything in the arena, including the audience/background. The strobes are somewhat directional, but with that much light bouncing around there is a lot of spill and reflection. By using the ambient lighting focused on the court, it made the background darker and the players pop out more. No strobes led to more contrast, which led to a better photo.

With the benefit of hindsight, it was a tremendous compliment to be called a faker. I was able to make the most of my situation to deliver something that was at least on the level of full-time professionals. I think we should all aspire to something similar in our own work. Every day we run headlong into constraints and limitations. What you do with those constraints makes all the difference in the end. Try to leverage them into something so good people doubt you did it fairly.

Photo: Mario Kinsey with the ball at Madison Square Garden during the Coaches vs Cancer Classic in November 2000. I had to dig through my archives to find this one. I don’t know if it is the same photo that was published, but it was the same event.

Gear: Mystery Ranch 3 Way Bag

Looking for a great shoulder bag to haul your tech when flying? Check out the Mystery Ranch 3 Way 18L Expandable Briefcase bag: https://www.mysteryranch.com/3-way-briefcase-expandable

I have a handful of irrational obsessions. Great bags and backpacks are definitely one of them. For travel, I try hard to one-bag it. This involves picking a single bag, usually a backpack, to carry all the things you need for a trip. Once you get the hang of optimizing it can be a lot of fun. I did a three day to New York last December with a 26 liter Goruck GR1. That included a spare pair of shoes and my running gear. Sometimes you need a bit more flexibility though, and a recent trip to Australia screamed for something new.

This trip required three bags since we would be gone for most of a month. First, the checked luggage, carrying all the day-to-day items that will be needed throughout the trip. This included things like clothes, snorkeling gear, and cycling gear. On my back, I had the same Grouck GR1 that I took to New York. It had three days worth of clothes and toiletries in case the checked luggage was late or lost. That left all the tech, which went into the Mystery Ranch shoulder bag and lived behind my legs against my seat during the flight.

Full list of the items carried:

• MacBook Air M2

• iPad Mini, in a Pivot case (https://pivotcase.com)

• Kindle Oasis, with a few physical books as well

• Charging kit, with cables and an Anker USB-C charger (https://www.amazon.com/Anker-Charger-4-Port-MacBook-Laptops/dp/B098WQRGNQ)

• Remote meeting kit, with Belkin USB light and wired headphones (https://www.amazon.com/gp/product/B097QZGRCQ)

• Anker 10k mAH USB battery (https://www.amazon.com/Anker-Portable-10000mAh-Delivery-PowerCore/dp/B096TP4B1T)

• Magsafe iPhone battery (1460 mAH; https://www.amazon.com/Apple-MJWY3AM-A-MagSafe-Battery/dp/B099BWY7WT)

• Wired and wireless in-ear headphones

• Bose QuietComfort 45 noise canceling headphones

• Eyeglasses and sunglasses

• Passport and wallet

• Field Notes Pitch Black Notebook (https://fieldnotesbrand.com/products/pitch-black-notebook)

• Pilot Juice 0.38mm pens (https://www.jetpens.com/Pilot-Juice-Gel-Pen-0.38-mm-Black/pd/10687)

• Hat and handkerchief

• Albanese gummi bears (https://www.albanesecandy.com/gummies/)

This bag is for 13” and smaller laptops. You can fit a 15” laptop in the main area of the bag, but it is a tight fit and it gets no foam protection from knocks and bangs. I wouldn’t recommend it unless you are in a pinch, which I have been and I have used the bag for a 15” MacBook Pro.

Check it out. I think it is worth your time if you want a small-ish over the shoulder bag that fits most of your tech things and stows easily under an airline seat.

Mystery Ranch also makes a larger 26L version:
https://www.mysteryranch.com/3-way-27-briefcase

You can spend hours reading the One Bag subreddit:
https://www.reddit.com/r/onebag/

Mystery Ranch makes tons of great gear:
https://www.mysteryranch.com

I have been a fan of Grouck in the past as well:
https://www.goruck.com/products/gr1

Scrutinize advice from the lucky

Never accept career advice blindly.

Sometimes I get asked to do something like a brown bag lunch talk on careers and career advice. The above comic is always the first thing that I show the audience. If I leave them with nothing else, I want them to not simply follow the advice of some schmuck like me just because I/we have achieved some measure of “success”. If I can get the audience to think critically about my presentation then I have succeeded in the most important goal.

I like to think that I am reasonably good at what I do. After ten years in academia and twelve years in industry I have learned a few things that make me more effective as a researcher, engineer, and manager. Still, a significant degree of luck has also shaped my career path. Here are a few, off the top of my head:

1) Having my grad school interview schedule changed at the last minute and randomly meeting the person who would be my future advisor.

2) Finding a great postdoc position because I literally ran into a professor at the Cognitive Neuroscience Society conference in New York and he said that his lab had a job available.

3) My first software manager deciding to take a risk and hire me even though I had no formal training as a software engineer.

4) Getting my first software management position because the CEO happened to remember that I organized large research projects as a postdoc.

5) The startup I joined getting acquired by a FAANG nine months after they almost ran out of money and laid off 30% of the company.

Hard work is always necessary. A little bit of luck can make all the difference though. Survivorship bias pops up when only the successful people are around at the end. Put another way, the people who have gotten lucky are much more likely to be giving presentations than the people who never got a fortunate nod from the statistical universe. Always know that.

Comic from XKCD: https://xkcd.com/1827/

Associated explainer: https://www.explainxkcd.com/wiki/index.php/1827:_Survivorship_Bias

Shine bright, but don’t burn out

I have ridden the burnout train before. It was a miserable experience.

Rewind to 2020. The startup I had joined in 2016 was acquired by a FAANG company in 2017. The new company gave us a healthy grace period to ramp up, but as time marched forward the corporate expectations on our org grew ever-larger. Information security demanded an audit. Finance dictated we shrink cloud spending by 10%. API response latency was up and we needed to bring it down by 50ms. I was running the SRE team, and it was becoming a lot.

The point I knew something was wrong was when I got paged during Thanksgiving 2019. We had traveled to the Sierra Nevada mountains with friends and the feast preparations had begun. My pager went off. I sprinted upstairs to grab my laptop, sliding in my socks on the carpet. My knee went down hard into the floor. 

I opened my laptop, saw that a service had run out of memory, restarted the instance, and went back to my holiday business. Total distraction time: 15 minutes. As I nursed my rug burned knee something was clear to the people in the room that was still dawning on me: I was hating every minute of my job.

The hate I felt wasn’t toward any person or any situation. It was a generalized frustration and irritability. I hated that I was underwater. I hated that I felt frustrated by my workload. I was quick to anger when things didn’t happen properly. I wanted to blame my manager, but I knew that I had willingly taken on my responsibilities. I hated that I shot myself in the foot by not communicating more effectively with my manager. That fact made me even more frustrated, but purely at myself this time.

The next week I (finally) talked with my supervisor and let him know that I was burnt out. He moved quickly to lighten my load and divide some of my responsibilities. This didn’t fix my burnout, but it did provide some breathing room. It was also a great career advancement opportunity for my peers. In hindsight, by holding on to so many responsibilities myself I was preventing others from demonstrating the skills to get them promoted. That is a blog post for another time…

My burnout recovered bit by bit. It took six months, but by the next summer I was feeling back to “normal”. Still, that July a former coworker let me know about a great new opportunity with their company. If I didn’t just go through burnout recovery I probably would have ignored him. Because I was still harboring some of that frustration and irritability I ended up saying “yes” to the recruiter when they reached out. The rest is history.

What can we learn from all of this? I think there are several lessons, but we can break them down into three for now:

1) Be vigilant for signs of burnout in yourself and your employees. Steer clear early.

The total cost for a bad case of burnout is high, for both the business and the individual. It is far better for everyone involved to steer clear of burnout as soon as the early signs are present. Don’t wait for them to say they are overloaded. Constantly ask them about their stress level, their workload, and week-to-week changes in their relationship with their work.

2) Burnout may hit your most-engaged people hardest.

In retrospect, the root cause of my burnout was a failure to say “no”. I wanted to help the org whenever we were in a tough spot. I had the skills and the knowledge to address problems that were constantly popping up. Why shouldn’t I help? I felt needed and I felt important. It felt good. However, the constant piling-on just served to erode the foundations of my motivation until it all collapsed on itself.  If you have someone that is constantly volunteering to take care of challenges make sure to check in with them more often.

3) Recovery from burnout can take ages. 

It is not a matter of just fixing the issues or dialing down the total amount of work. I can say with authority that it is an effortful journey back to an engaged state of mind. Time away from work, new challenges, resetting expectations, and setting boundaries are just a few things that can help.

None of this is pleasant. None of this is happy. I think the best step toward addressing burnout as an industry is to talk about it. If a manager decides to consciously ignore burnout as a factor and optimize for short-term deliveries from their team then, hey, more power to them. If you have a desire to optimize around the long-term success of your organization then navigating the burnout cycle is a critical pillar of sustainable engineering.

Photo: The Amtrak Pacific Surfliner stopping at the San Luis Obispo station. It’s a beautiful ride north from Santa Barbara.