Voxelwise

Engineering leadership, organizational health, and why smart teams still struggle.

Iterate, iterate, iterate

Pretty much anything great comes from constant iteration. The vast majority of the time, it is better to start building and learning than taking forever to create a perfect plan that is uninformed by experience.

Last year I had to get a halloween costume with 48 hours of notice. I decided to go with something easy to pull together: a classic Star Trek uniform. Black shoes, black pants, a gold shirt (command staff, naturally), and a tricorder mockup. All I needed was the badge to put on the shirt.

There wasn’t time to order a badge online, so I turned to my 3D printer. I found a few badge files online and began the process of iteration:

  • #1: Too small, needs to be bigger
  • #2: Right size but had a printing error, so couldn’t really try the vinyl overlay
  • #3: Tried to use cut vinyl for an overlay, but it was hard to place and didn’t look good
  • #4: Try it a bit smaller. Ok overall shape
  • #5: Printed a gold final version, but the vinyl star still didn’t look great
  • #6: Made a window in the top layer and put a black layer underneath. Still looked messy.
  • #7: Combined elements of the previous tries: larger badge, flatter height, and a smaller window for the black layer underneath. SHIP IT.

If there is one thing that 3D printing has taught me is that you often need to just give it a try and see how things turn out. All the preparation in the world in a CAD program helps, but nothing is better than printing your current best guess and holding that iteration in your hands. Sure, you will print a few things that get thrown immediately into the trash can. Fortunately, plastic is cheap and the rate limiting step is just the time it takes to create a print.

I had been at my current company for just a few months the first quarter of 2020. We were working on a major new vendor integration. A staff engineer on my team came to me and said “we have been planning the project for four weeks, but we will need four more weeks to complete the planning and be ready to start coding”. I weighed my options. I trusted him, and I trusted his assessment that more time was needed for planning. At the same time, I knew that we had to begin the process of iteration so we could learn how the product plan would come together in real life.

“No. I’d like you to work with the team and start coding next week. Even if we have to throw away the code, we need to start building.”

It was a risk on my part, but a calculated one. In the end, I asked what the team needed more. Did it need greater knowledge of the vendor interface or did it need greater knowledge of using the interface in real world circumstances? I decided it was the later. While it is impossible to say if this was a key project decision, we did end up shipping the system on time that quarter.

It would be cavalier to leap into a big project with zero planning. Likewise, if your goal is to reduce risk to zero then you will be planning forever. The key element then is finding the right balance between speed and risk. One of the best ways to reduce risk is by gathering data on real world operation through constant iteration. Gaining experience and letting data guide you is worth many weeks of conversation on theoretical points.

Photo: The iterations on the badge, starting from #1 on the left to the final #7 version on the right. Hot glued some magnets to the back and I was in business.

Don’t throw away your shot

There are pivotal moments that can ruin promising careers. Don’t lose it all because you were self-focused and couldn’t stop yourself.

I learned a lot about how the brain works while in grad school. One thing I learned is that the brain largely works through inhibition. That is, you don’t so much “hit the gas” to do something as opposed to “tap the brakes” to ensure the right things happen. 

Think about that one moment where you wanted to do something so bad but decided not to. It could be something innocent, like wanting to call in sick and sleep longer but you went to work anyway. More dramatically, it could be something like a time when you wanted to yell at your boss but kept your mouth shut instead.

Most decisions have a small blast radius, but there exist a small number of pivotal decisions with the power to blow up entire careers. The ability to stop yourself and inhibit your initial response is a critical skill for leadership. It really is the thing that separates long-term success from sudden and dramatic failure. This is because it takes years to build the foundation for success and only moments to tear it all down.

Let’s explore four short examples where leaders, and people that I had respected, threw it away.

It’s not your money

An acquaintance was the head of a local city department. He was seen as an effective leader, well liked by his staff and city leadership. Things were going well, but this individual had a deep political streak. The Republican candidate for mayor stopped by his office and they talked about how desperately the city needed conservative leadership. The acquaintance agreed, and (allegedly) gave the candidate money out of the petty cash box to help fund his campaign. Let’s pause for a second. This was city money given to a political candidate – a huge mistake. A disgruntled member of his staff reported the incident and the leader was investigated by law enforcement. He quit the job and moved out of town before he would be forced to step down. Years of investment gone in an instant because he couldn’t stop himself in that moment.

Don’t date your followers

A new minister moved to town and was a huge hit with his congregation. He brought a new, youthful energy to his work that inspired and invigorated the church. Over several years he built a dynamic new organization, always pushing to do more and serve the community in new ways. This included helping to heal wounds from decades past, when a previous minister engaged in unprofessional sexual conduct with congregants. The new minister was a superstar.

Five years after he first arrived the minister was leaving for a larger role with a bigger congregation. As he moved away it became apparent that the minister was cheating on his wife with one of the local congregants. He continued to behave badly in his new role. Three years later he was under investigation and put on administrative leave for inappropriate relationships with congregants. He was dismissed from the congregation and ultimately resigned his fellowship with the larger faith pending allegations of ministerial misconduct. An incredibly promising minister and a career that benefitted hundreds or thousands of people – all of it thrown away because he couldn’t be trusted with the people of his congregation(s).

Perceptions matter

The best conference I attended while in academia was the “New Horizons in Human Brain Imaging” gathering in Hawaii. Small conference. Great talks. Amazing location. On the last night of the conference in 2010 I was with several attendees in the hot tub hanging out and talking shop. As we talked one of the professors and a postdoc started touching and whispering to each other. They jumped out of the hut tub and chased each other around for a minute before chasing each other into the bushes. Another attendee mentioned “isn’t he married?”, to which a second member of the hot tub responded in the affirmative. Rumors only grow after an episode like that. Conversation about the conference continued for years afterward.

The ultimate misappropriation of trust

The worst cases of professional malpractice were those I saw in grad school. Twenty years ago the Psychological and Brain Sciences department at Dartmouth College had a very permissive atmosphere. The lines between professors, postdocs, and grad students got very blurry. It was not unusual to walk into a local bar* and see professors drinking with grad students like they were in their 20s again.

The grad students knew which professors were “handsy” and who got a little too close for comfort. Several people switched labs when they found out the hard way that their advisor was a creep. Some had affairs with professors. Permissiveness continued to escalate over time. Eventually, three professors clearly crossed the line and were dismissed. The resulting lawsuit cost the College at least $14M dollars. The details of the situation are worth their own separate post, but suffice it to say that three promising professors ruined their reputations and their careers because they couldn’t control themselves**. More critically, they hurt a lot of people over many years in the process. The misconduct allegations in 2018 were a contributor to the department chair taking his own life one year later.

Summary

I have heard leadership defined as helping any group of two or more people achieve their common goals. We leverage respect, influence, and power to make that happen and move our group toward those goals. That same respect, influence, and power can also be misappropriated to further selfish ends. This action is incisive, and it cuts deep. Not only do we abuse the trust of those within our leadership but we harm our own advancement by behaving badly.

Don’t fall into this trap. Don’t throw away your shot at greatness. Make sure to hit the brake pedal when the time is right. It matters, both to you and the people under your leadership.

* Hanover, NH was a small town of 8,000 people, so you run into others often.

* All the more ironic that a key area of research for one of the professors was failure of self-regulation: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3062191/

Photo: A Founder’s Fizz, from the Richard Rodgers Theatre in 2016 as we watched Lin-Manuel Miranda and the original cast of Hamilton perform. “My Shot” is one of the best songs of the show.

Slava Ukraini

Topeka, Kansas is a small city that can sometimes surprise you. One of the surprising bits was the ability to take four years of Russian language classes in high school. By some crazy random happenstance Topeka High School had Russian instruction going back to 1973.

The local Russian teacher, Mr. Lonard, went into the middle schools every year to drum up interest in the classes, making sure there were always a critical mass of students. The highlight of the program was a biennial exchange where high school students from Kansas would host high school students from the former Soviet Union. A few months later, students from Kansas would then be hosted in Russia or Ukraine.

I wanted to go. Sitting in that middle school classroom, I knew I wanted to go.

Fast forward three years. I was a junior in high school and on a plane to Kharkiv, Ukraine. I had been to Mexico once or twice while visiting family in San Diego, but this was a different scale of international travel. Having two-ish years of Russian classes had prepared me somewhat, but my conversational language wasn’t great and I was getting thrown in the deep end.

I was scared out of my mind.

My host brother Dimitri stayed with his maternal grandparents in a suburb of Kharkiv called Piatykhatky. This was done so he could attend the local secondary school, which was one of the best in Kharkiv. Piatykhatky was home to the former Ukrainian Physics and Technology Institute, now called the Kharkiv Institute of Physics and Technology. With such a concentration of scientific talent the local school had an amazing set of teachers.

As told by my Russian teacher, Piatykhatky was completely closed to foreigners while Ukraine was part of the Soviet Union. Now, a group of high school students from Kansas would call it home for six weeks – just four years after the Soviet Union collapsed.

I spent the majority of each week with Dimitri and his grandparents in their small apartment. We then spent the weekends with Dimitri’s parents and sister. They lived on an Army base on the outskirts of town.

I could go on for many pages about how wonderful Dimitri and his whole family were to me. There are dozens of stories to tell of Dimitri’s visit to Kansas as well. I could also go on for pages about how friendly the population of Kharkiv were to a bunch of wayward American high school students as we tried to navigate a large town half a world away from home.

The original exchange program between Topeka High and School 62 in Piatykhatky ended in 1998. There were rumors of corruption as the Director of School 62 allegedly solicited bribes during the exchange selection process. I was happy to see recent news that students from Ukraine are once again attending Topeka High School.

It is enough to say that I have been thinking about Dimitri and his family a lot lately. Russia invaded Ukraine and Kharkiv Oblast over two years ago. According to some reports, the invasion started at Piatykhatky, which is only 17 miles from the Russian border. Kharkiv, the city, has been under unrelenting artillery shelling and missile strikes ever since. School 62? The building where I took classes is reported to have been destroyed.

It is a cruel action taken against wonderful people, and a country that I have held in high regard for almost thirty years. 

I am looking forward to the war being over one day. I dream of returning to Piatykhatky after three decades away and walking the same streets I walked as a teenager. For now, this is an impossibility. It will have to be enough that my thoughts are with Ukraine and its citizens.

Slava Ukraini.

Director: Other duties as assigned

At some point in your career the job description becomes 100% “whatever it takes for your team and the company to be successful”. The people that get ahead are the ones that realize this fact early.

Steve Jobs is always a controversial character when evaluating management styles, but I have always loved this story regarding his expectations for VPs:

Steve Jobs told employees a short story when they were promoted to vice president at Apple. Jobs would tell the VP that if the garbage in his office was not being emptied, Jobs would naturally demand an explanation from the janitor. “Well, the lock on the door was changed,’ the janitor could reasonably respond. “And I couldn’t get a key.”

The janitor’s response is reasonable. It’s an understandable excuse. The janitor can’t do his job without a key. As a janitor, he’s allowed to have excuses.

“When you’re the janitor, reasons matter,” Jobs told his newly-minted VPs. “Somewhere between the janitor and the CEO, reasons stop mattering.”

“In other words,’ (Jobs continued,) “when the employee becomes a vice president, he or she must vacate all excuses for failure. A vice president is responsible for any mistakes that happen, and it doesn’t matter what you say.”

John Rossman, Think like Amazon

There is a lot of power in the Jobs story. As a senior leader you own it all – both successes and failures. The reasons for each stop mattering. My addition to that perspective is that ownership shouldn’t start at the VP level. You should be going above and beyond to deliver results, and owning all outcomes, wherever you are in your career journey. This trait is rare and puts you far above your peers in any industry.

I have had numerous direct and indirect reports that requested a kind of checklist for promotion. “Just tell me what to do and I will do it” is a common refrain. This is relatively easy to provide to recent grads up through Senior engineers. The expectations at these levels are more straightforward, almost mechanical. At the Staff+ levels it becomes much more difficult to create that kind of to-do list. This is because managers are indexing less on your technical knowledge and project completion rate. Instead, we are focusing more on your soft skills, including communication, collaboration, and ownership. Establishing strong ownership early shows a senior level of execution, accelerating your career growth.

This same ownership effect occurs within the management ladder. While software management is already a bit like four different jobs masquerading as one, at the Director+ levels is starts boiling down to “whatever it takes”. This explains why so many Director+ job postings are oddly vague, with terms like “create a healthy team culture”, “collaborate with product management”, or “ensure technical excellence”. As a job posting there has to be a job description, but how do you capture the reality of “responsible for everything”?

One of my personal Top 50 management books is Extreme Ownership by Jocko Willink and Lief Babin.  You can distill the entire book down to a single view: you always have some control over a situation, so stop giving excuses and pretending that there was nothing you could do. The individuals who get ahead quickly are the ones that internalize this fact early.

Photo: The Cessna 152 that I did most of my flight training in. I have a lot of fond memories in that plane, including my first solo flight. On any plane the pilot-in-command (PIC) is the individual directly responsible for and the final authority regarding operation of that aircraft. No excuses.

Aurora are epic as well

Space Weather Message Code: ALTK09
Serial Number: 6
Issue Time: 2024 May 10 2334 UTC

ALERT: Geomagnetic K-index of 9o 
Threshold Reached: 2024 May 10 2254 UTC
Synoptic Period: 2100-2400 UTC

Active Warning: Yes
NOAA Scale: G5 - Extreme

...

Aurora - Aurora may be seen as low as Florida to southern Texas and southern California.

The Space Weather alert by NOAA was no joke. Aurora could be seen in southern California with the naked eye, and the long-exposure photos were incredible. We had to head up into the hills to get above the marine layer in Santa Barbara, but it was totally worth it.

Total eclipses are epic

It took a lot of prep, but I got the photos I wanted of the 2024 total eclipse. It also helps to have luck on your side, which you can facilitate by taking 670 shots of the eclipse over several hours…

The weather in Dallas was right on the edge in the weather models, and it was totally overcast in the morning. By the time the eclipse started the skies were partly cloudy. When totality hit the skies were almost clear. What a wonderful experience to share with several hundred people in Turtle Creek Park. Definitely a bucket list item if you can make it happen in your lifetime.

Getting ready for the April 2024 total eclipse

Practice makes perfect. Will be in Dallas next month for the big event. Look at the sunspots!

Equipment: Canon 7D, 400mm f5.6 lens, 1.4x teleconverter, with a NiSi PRO Nano ND100000 neutral density filter. That provides the equivalent of a ~900mm lens with the APS-C sensor crop factor. I used an external HDMI monitor for positioning and focus since I really love having healthy retinas…

Have the bookshelf you wish the library had

Create a powerful collection of reference materials that help you understand and explore your craft. Keep building and iterating that collection over time.

I loved the library as a kid. Once a month or so my sister and I would talk our parents into a trip to the Topeka Public Library and we would spend hours browsing the stacks. Anything you were interested in – there was a book for it. I spent a lot of time in the nonfiction area checking out books on things like electronics, weather, and geology. I still remember the excitement of venturing out of the kids section for the first time and walking over to the regular collection. Thousands of new books and an entire new wing of the library to explore.

Fast forward to 2012. I was working as a firmware engineer at the time and happened to be visiting friends in Cambridge, MA. As we wandered around we stopped by the Harvard/MIT Coop bookstore. In the Coop I found one of the most amazing sights for a software engineer: an entire wall of O’Reilly technical books. These are the ones with bright covers and random print animals on the front. There were books for C++, Python, Data Engineering, Flask web services – you name it. In that moment I told my wife “this is the library I wish I had at home”. I bought a book, if only to remember the experience.

Now fast forward to today. Every September my wife and I go to the local book sale fundraiser for Planned Parenthood. Like kids in a candy store, we buy dozens of books every year. As I was shelving my new finds last fall I had to stop for a second. I realized that my collection of software and management books had finally iterated to the point where I very nearly had the perfect bookshelf – the bookshelf I wished the library had.

All my favorite authors. A ton of community favorites. A few stinkers, sure, but this bookshelf has served me well by opening my eyes to new perspectives and new challenges. With each book I have improved my engineering and my management. While most of the books I have finished reading, there are still a number that I haven’t opened up yet. I want the same reference library, and the same benefits, for everyone.

A quick thanks to Planned Parenthood of the California Central Coast for putting on their annual book sale in September. It has always been a treat. Thanks also to my current employer, who graciously gives every employee an enrichment stipend each year for professional growth. I spend it all on books…

Photo: two out of my eight shelves in the home office. Also seen: a dumpster fire plushie, a plastic airplane model, a Highway to the Danger Zone patch, an incomplete 3D print of a tricorder from Star Trek: The Next Generation, and a static light scattering flow cell from my days at Wyatt Technology.

Take your lemons and make unbelievably good lemonade

Use whatever advantages you can to create the best possible output. Be so good at navigating constraints that others think you are breaking the rules.

In college I was a photographer for the school paper, the University Daily Kansan. I loved photography and saw the journalism gig as a great way to get better. I started out doing speaker and podium shots. If someone was giving a talk on campus then I was usually there, getting a photo. It was actually a fabulous exercise. Sure, you could take the same boring shot dozens of times a month and get the job done. Alternatively, you could use it as an exercise to get creative. I tried to do the latter.

Over time I got to be a better photographer and I earned trust with the photo editors. My next evolution was sports, covering the football team (KU wasn’t known for great football) and occasionally covering basketball. To say that it was a treat to sit on the court covering a game at Allen Fieldhouse is putting it mildly…

In November of 2000 a huge gig came up. The paper was sending two reporters and one photographer to New York to cover the Coaches vs Cancer Classic basketball event (now called the Empire Classic). I got tapped to go since I had taken the time to learn the new Nikon digital camera system and could send photos back for publication same-day. I hopped on the plane with the reporters and got to sit on the court at Madison Square Garden to shoot the Jayhawks playing UCLA and St John’s.

The lighting conditions weren’t great at the venue. The professional photographers that surrounded me all had powerful strobe flashes linked in the ceiling to get the light they needed. The strobes provided enough light to freeze the action, making basketball shots easy. All I had was an ƒ2.8 70-700mm lens to get the job done. I had other tricks up my sleeve though. First, I knew how far I could push the ISO of the camera without getting too much noise, meaning that I could get good light sensitivity (and faster shutter speeds). I also had shooting techniques that helped, like following a moving target through the longer exposure to help keep the player centered on the sensor. This reduced blur. In the end, I got some decent shots that went onto the front page of the Kansan as the basketball team won the event championship.

When I got back to Kansas my photo editor called me into his office. “Hey, nice work on the Coaches vs Cancer event. That went really well. By the way, did you heavily Photoshop any of your photos? The photo editor at another paper says they look way over-edited.”

I was shocked. Event photography is all about capturing the moment as it happened. Here I was, being accused of bending reality. With a bit of anxiety I went to the nearest workstation and showed my editor the workflow from the raw image. First, crop the shot. Next, adjust the exposure curve with one click. Finally, sharpen the image and ship it.

My editor was satisfied with that. I head rumors that he called the other editor up and said he was full of it. Ultimately, the reason my photo was better was exactly because of my disadvantages.

When strobes fire they tend to illuminate everything in the arena, including the audience/background. The strobes are somewhat directional, but with that much light bouncing around there is a lot of spill and reflection. By using the ambient lighting focused on the court, it made the background darker and the players pop out more. No strobes led to more contrast, which led to a better photo.

With the benefit of hindsight, it was a tremendous compliment to be called a faker. I was able to make the most of my situation to deliver something that was at least on the level of full-time professionals. I think we should all aspire to something similar in our own work. Every day we run headlong into constraints and limitations. What you do with those constraints makes all the difference in the end. Try to leverage them into something so good people doubt you did it fairly.

Photo: Mario Kinsey with the ball at Madison Square Garden during the Coaches vs Cancer Classic in November 2000. I had to dig through my archives to find this one. I don’t know if it is the same photo that was published, but it was the same event.

Gear: Mystery Ranch 3 Way Bag

Looking for a great shoulder bag to haul your tech when flying? Check out the Mystery Ranch 3 Way 18L Expandable Briefcase bag: https://www.mysteryranch.com/3-way-briefcase-expandable

I have a handful of irrational obsessions. Great bags and backpacks are definitely one of them. For travel, I try hard to one-bag it. This involves picking a single bag, usually a backpack, to carry all the things you need for a trip. Once you get the hang of optimizing it can be a lot of fun. I did a three day to New York last December with a 26 liter Goruck GR1. That included a spare pair of shoes and my running gear. Sometimes you need a bit more flexibility though, and a recent trip to Australia screamed for something new.

This trip required three bags since we would be gone for most of a month. First, the checked luggage, carrying all the day-to-day items that will be needed throughout the trip. This included things like clothes, snorkeling gear, and cycling gear. On my back, I had the same Grouck GR1 that I took to New York. It had three days worth of clothes and toiletries in case the checked luggage was late or lost. That left all the tech, which went into the Mystery Ranch shoulder bag and lived behind my legs against my seat during the flight.

Full list of the items carried:

• MacBook Air M2

• iPad Mini, in a Pivot case (https://pivotcase.com)

• Kindle Oasis, with a few physical books as well

• Charging kit, with cables and an Anker USB-C charger (https://www.amazon.com/Anker-Charger-4-Port-MacBook-Laptops/dp/B098WQRGNQ)

• Remote meeting kit, with Belkin USB light and wired headphones (https://www.amazon.com/gp/product/B097QZGRCQ)

• Anker 10k mAH USB battery (https://www.amazon.com/Anker-Portable-10000mAh-Delivery-PowerCore/dp/B096TP4B1T)

• Magsafe iPhone battery (1460 mAH; https://www.amazon.com/Apple-MJWY3AM-A-MagSafe-Battery/dp/B099BWY7WT)

• Wired and wireless in-ear headphones

• Bose QuietComfort 45 noise canceling headphones

• Eyeglasses and sunglasses

• Passport and wallet

• Field Notes Pitch Black Notebook (https://fieldnotesbrand.com/products/pitch-black-notebook)

• Pilot Juice 0.38mm pens (https://www.jetpens.com/Pilot-Juice-Gel-Pen-0.38-mm-Black/pd/10687)

• Hat and handkerchief

• Albanese gummi bears (https://www.albanesecandy.com/gummies/)

This bag is for 13” and smaller laptops. You can fit a 15” laptop in the main area of the bag, but it is a tight fit and it gets no foam protection from knocks and bangs. I wouldn’t recommend it unless you are in a pinch, which I have been and I have used the bag for a 15” MacBook Pro.

Check it out. I think it is worth your time if you want a small-ish over the shoulder bag that fits most of your tech things and stows easily under an airline seat.

Mystery Ranch also makes a larger 26L version:
https://www.mysteryranch.com/3-way-27-briefcase

You can spend hours reading the One Bag subreddit:
https://www.reddit.com/r/onebag/

Mystery Ranch makes tons of great gear:
https://www.mysteryranch.com

I have been a fan of Grouck in the past as well:
https://www.goruck.com/products/gr1

Newer Posts Older Posts