Design Reviews: Meeting Format

Today is the day! You know what a design review is supposed to achieve. You also know who to invite and why they're there. You're ready to walk into the room and have a super productive design review, right? Right?!

Anyone who's been around design reviews for a while has seen a variety of approaches. We saw a few in the first entry of the series as examples of why people don't often get much out of design reviews. We want to do better.

Take a step back...

I know I wrote a whole post about the goals of a design review process, but we never really addressed the goals of a particular review. What you want out of the review can definitely change how you want to run the meeting.

Throughout my career, I've seen three general types of review that can work, especially considering the attendees that we addressed in the previous entry. Let's take these in order of increasing preparation.

Consultation

While I worked at Amazon, the principal community noticed the same problems we've all seen. Design reviews occurred too late, and feedback really couldn't affect the software that shipped. To try to address this issue, they created the principal engineer consultation. I have many criticisms of Amazon's culture, but this was a great idea!

The key benefit to the consult was that you didn't need to have a design to present. You could walk in with only a problem you were trying to solve.

For consults, I recommend having some diagrams about the system or the business case along with the specific issues you want advice on. Don't come with the requirements doc and expect a full design to come out of the review. I've found the best preparation comes from having 2 or 3 specific questions that, when answered, would inform your decisions of technology or patterns.

When the meeting finally comes, the presenter needs to use whatever materials provided to give context and then identify the areas where advice and guidance are needed. The rest of the meeting^[1] can be a conversation about those topics.

Focused Review

Taking the next step up in the design review continuum brings us to focused reviews. In the consultation above, the presenters had questions about an area of their design and needed advice and guidance to proceed.

The focused review flips that. The presenters have a design, but there are complex areas they want to get feedback on. You know that design that is super elegant except for this one place that gets really messy? That's your focus for the review. You know where those are, and it's perfectly normal to have ugly pieces to an otherwise simple design.

When focusing on the difficult and ugly parts of a system, you can gain a lot of benefit from the more experienced engineers. Maybe there's a software concept that already exists (either in open source or commercial software) that would take the complexity off your hands. Maybe you're over-thinking it. Maybe you just need to make sure you monitor it well enough to deal with the fact that it will never be cleaner or simpler.

Since you're reviewing an existing design, you should probably bring that design to the review. I want to stress this next point, though.

Do not wait until you have revised the design a dozen times before having the review.

I've mentioned this before, but reviews need to happen early. If you wait too long, you're wasting effort.

There's another aspect to waiting too long to get feedback from others: defensiveness. In my experience^[2], earlier feedback reduces the emotional response to criticism. I'm sure many readers will have experienced the engineer who worked on something for months and vigorously pushes back on any constructive feedback. That's not good for anyone. We want to work in the open and build the right thing^[3].

In any case, the meeting should be similar to the consult with a little more presentation of the context and the existing design. There's a good chance you'll have a design doc, but I think visual representations are a better way to communicate many technical design ideas rather than text. Focus on the diagrams and let the document provide supporting information.

General/Full Review

We've finally made it to the full review. I think the term should be self-explanatory. I also think this is usually a bad idea.

Conversations without focus are easy to derail. You could spend 2 hours discussing an aspect of the design when there is something else that could use more attention.

So my advice on full reviews?

1. Don't do them
2. If you do them, still try to have areas of focus where you think the most benefit lies.
3. Time box them and stick to it

The preparation and presentation should look similar to the focused review, but it's obviously going to be more wide-ranging. Unfortunately, it's also going to be harder to do these reviews early because you have to have a full design in place. If you have to do an unrestricted review like this, you should still try to do it as early as you can. There's also a lot of value in this mantra: "Thank you for the feedback/idea/suggestion. We've recorded it^[4] and will follow-up later."

Finishing the meeting

The end of the meeting might be the most important.

First, the meeting should end on time. As I've mentioned, principals and architects can talk about anything for hours. Don't let us.

The upshot of ending on time is that there will be conversations left unfinished. That's great! Unfinished conversations lead to follow-ups that can create the working relationsihp between the engineer designing the system and the principal engineers advising them. Everybody wins!

Finally, commit to a timeline for incorporating feedback and continuing those unfinished conversations. It's easy to come out of the review and immediately get bogged down in implementation. Keep the improvement of the design going, even if implementation is already in progress. Don't let the sunk cost fallacy take over.

Follow-ups

Follow-ups from design reviews are not lip service. Do them. Learn from them. Improve your design. I know it won't always be pleasant, but these relationships are important. At the same time, you should not automatically do anything the senior engineers say. Like the review itself, the follow-ups should be conversations.

By adhering to your commitments, you will gain supporters for your design. Those supporters can help shield you and your team from randomization or interference from other principal engineers. I can't count how many times I've seen concerns raised by one principal and easily addressed by another principal offline without involving the team that actually owned the system. Use their knowledge as well as their position to help you and your team succeed.

But what's the agenda?

For those of you expecting me to provide a detailed agenda with time suggestions, I'm sorry. That's not usually how these things work. That said, I've tried to set you up for success. The meeting is small enough to be conversational. You know what you're trying to get out of it, and you know how it should end. I hope that gives you enough guard rails to be successful.

Next week, I'll discuss patterns and anti-patterns in the reviews, so come back for the barn-burner of good and bad design review behavior!