24 Comments
author

"Running a sprint is kind of like baking a cake: If you don’t follow the recipe, you might end up with a disgusting mess.

For your first few sprints, follow all the steps. Once you’ve got it down, feel free to experiment, just like an experienced baker."

Reminds me of Picasso's quote Learn the rules like a pro, so you can break them like an artist 👩🏼‍🎨

Expand full comment
author

"Throughout the book, you learned a handful of unconventional ideas about how to work faster and smarter:

• Instead of jumping right into solutions, take your time to map out the problem and agree on an initial target. Start slow so you can go fast.

• Instead of shouting out ideas, work independently to make detailed sketches of possible solutions. Group brainstorming is broken, but there is a better way.

• Instead of abstract debate and endless meetings, use voting and a Decider to make crisp decisions that reflect your team’s priorities. It’s the wisdom of the crowd without the groupthink.

• Instead of getting all the details right before testing your solution, create a façade. Adopt the “prototype mindset” so you can learn quickly.

• And instead of guessing and hoping you’re on the right track—all the while investing piles of money and months of time into your ideas—test your prototype with target customers and get their honest reactions."

Expand full comment
author

"The Five-Act Interview

This structured conversation helps the customer get comfortable, establishes some background, and ensures that the entire prototype is reviewed.

Here’s how it goes:

1. A friendly welcome to start the interview

2. A series of general, open-ended context questions about the customer

3. Introduction to the prototype(s)

4. Detailed tasks to get the customer reacting to the prototype

5. A quick debrief to capture the customer’s overarching thoughts and impressions"

This reminds me Behzod Sirjani's Rapport > Build > Peak > Wrap up trail guide structure we explore during Reforge User Insights for Product Decisions.

https://www.reforge.com/user-insights-product-decisions

I have a great example from META using the system.

Expand full comment
author

"We know that the idea of testing with such a small sample is unsettling to some folks.

Is talking to just five customers worthwhile? Will the findings be meaningful?

Earlier in the week, you recruited and carefully selected participants for your test who match the profile of your target customer.

Because you’ll be talking to the right people, we’re convinced you can trust what they say.

And we’re also convinced that you can learn plenty from just five of them."

— 

I believe you can get some sort of signal talking to five customers that match the desired audience. Nielsen has extensive research to justify the diminishing returns of a larger interview sample.

It's enough confidence to decide on proceed to the next experiment.

Expand full comment
author

"Great solutions often come along at the wrong time, and the sprint can be a perfect opportunity to rejuvenate them.

Also look for ideas that are in progress but unfinished — and even old ideas that have been abandoned."

For our first Sprint, we ended up prioritizing an idea that was sitting on the backlog for 8 months.

17% increased conversion from acting on it.

Take time to source ideas 🎯

Expand full comment
author

"On Monday, you and your team defined the challenge and chose a target. On Tuesday, you’ll come up with solutions.

The day starts with inspiration: a review of existing ideas to remix and improve.

Then, in the afternoon, each person will sketch, following a four-step process that emphasizes critical thinking over artistry.

Later in the week, the best of these sketches will form the plan for your prototype and test."

I'm still writing about this powerful concept I learned from Mike Taylor.

Everything is a remix 👉🏼 https://youtu.be/nJPERZDfyWc

Expand full comment
author

"To prioritize the notes, you’ll use dot voting. It’s one of our favorite shortcuts for skipping lengthy debate. Dot voting works pretty much the way it sounds:

1. Give two large dot stickers to each person.

2. Give four large dot stickers to the Decider because their opinion counts a little more.

3. Ask everyone to review the goal and sprint questions.

4. Ask everyone to vote in silence for the most useful How Might We questions.

5. It’s okay to vote for your own note, or to vote twice for the same note."

Dot voting works.

Used yesterday to prioritize a team discussion as we're selecting an agency to work on a particular project.

Think of many reactions and copy pastes from four different agency offerings on a Miro board.

10 minutes on the timer for people to go through, refresh ideas, and place dots.

As we zoom out the dots form a heatmap of where the conversation should focus on.

We start by covering decider dots.

Assertive 🎯

Expand full comment
author

"We do have a technique that results in organized, prioritized notes from the entire team. The method is called How Might We.

It was developed at Procter & Gamble in the 1970s, but we learned about it from the design agency IDEO.

It works this way:

Each person writes his or her own notes, one at a time, on sticky notes.

At the end of the day, you’ll merge the whole group’s notes, organize them, and choose a handful of the most interesting ones.

These standout notes will help you make a decision about which part of the map to target, and on Tuesday, they’ll give you ideas for your sketches.

With this technique, you take notes in the form of a question, beginning with the words “How might we . .

And because every question shares the same format, it’s possible to read, understand, and evaluate"

I feel like I was exposed to this idea of How Might We from a complete different (and weird) angle.

It's a standardised note taking system for Ask the Experts!

Expand full comment
author

"Knowledge is distributed.

Somebody knows the most about your customers; somebody knows the most about the technology, the marketing, the business, and so on. In the normal course of business, teams don’t get the chance to join forces and use all of that knowledge.

Most of Monday afternoon is devoted to an exercise we call Ask the Experts: a series of one-at-a-time interviews with people from your sprint team, from around your company, and possibly even an outsider or two with special knowledge."

I introduced the concept to the team yesterday and they said it's a must for our second Sprint.

Bringing two members of leadership group and a subject matter expert.

I expect them to help us refine the conversion map and take questions from Sprint participants 🎯

Expand full comment
author

"No matter how complicated the business challenge, it can be mapped with a few words and a few arrows.

Your map should be simple, too. You won’t have to capture every detail and nuance.

Instead, you’ll just include the major steps required for customers to move from beginning to completion."

— 

I'm onboarding a second group of talented software engineers to our Experimentation Sprint program.

A key difference in our process is that we start the Sprint with clarity on our testing hypothesis.

In Google's version they close Monday with that definition in place.

The logistics of that involve backlog refinements. This time I kicked off the call with our conversion map.

For the builders in the call, they were able to connect the dots between opportunity area and sample size risk.

For the leaders in the call, the map became a roadmap of how the Sprint Program expand operations tackling different areas of the journey.

Must have for alignment and onboarding.

Expand full comment
author
Oct 1, 2022·edited Oct 1, 2022Author

"List sprint questions.

We have a few prompts for getting teams to think about assumptions and questions:

• What questions do we want to answer in this sprint?

• To meet our long-term goal, what has to be true?

• Imagine we travel into the future and our project failed. What might have caused that?

An important part of this exercise is rephrasing assumptions and obstacles into questions."

This reminds me of how Elena Verna suggests the importance of Pre-Mortem [29 min] in this epic 20VC podcast 👉🏼 https://open.spotify.com/episode/5hHB2M6dkpedtkV6bGW105?si=RsnBEDC-R_OxHwcdi4Gk2Q

Expand full comment
author

"Set a long-term goal.

To start the conversation, ask your team this question:

Why are we doing this project?

Where do we want to be six months, a year, or even five years from now?”

— 

We learned that "starting with the end in mind" set a high-level filter to the ideas that become eligible for Experimentation Sprint.

Does it deliver on revenue growth for these key markets?

Is this something that can be white labeled and deployed across the network?

If yes for both it gets into the prioritization pipeline.

All from starting with the end in mind.

Expand full comment
author
Sep 29, 2022·edited Sep 29, 2022Author

“Monday’s structured discussions create a path for the sprint week.

In the morning, you’ll start at the end and agree to a long-term goal.

Next, you’ll make a map of the challenge. In the afternoon, you’ll ask the experts at your company to share what they know.

Finally, you’ll pick a target: an ambitious but manageable piece of the problem that you can solve in a week.”

I was just listening to 20VC's episode with Lenny K.

The way he describes Airbnb's working backwards implementation is fascinating.

Read more about Amazon's Working Backwards concept here👇🏼https://www.linkedin.com/posts/joaoostrowski_working-backwards-amazons-culture-book-activity-6817868895538470912-Jov4

Expand full comment
author
Sep 26, 2022·edited Sep 29, 2022Author

“Sprints are most successful with a mix of people: the core people who work on execution along with a few extra experts with specialised knowledge.

[…]

Recruit a team of seven: A Decider, and experts in those fields Finance, Marketing, Customer, Tech/Logistics, and Design”

— 

Since our version is more engineering focused, we ensure the right mix during Sprint 0, our preparation round counts on UXRs, Designers, business leaders, and domain experts.

Expand full comment
author
Sep 24, 2022·edited Sep 29, 2022Author

"It's about focus.

Be honest about your motivations.

If the quality of your work is suffering because your team's regular work schedule is too scattered, say so. Instead of doing an ok job on everything you'll do an excellent job on one thing. Fragmentation hurts productivity."

— 

One of the drivers for our Experimentation Sprint was to find focus by staying away from scrum meetings and "BAU tickets" for a week.

It's funny how Product organizations can get lost in the bureaucracy of lean systems.

In five days, we were much more agile with Agile.

Expand full comment
author
Sep 22, 2022·edited Sep 29, 2022Author

“When we talk to startups about sprints, we encourage them to go after their most important problem.

Running a sprint requires a lot of energy and focus. Don’t go for the small win.”

Our best decision in the first sprint, go for a big implementation in a product area with enough traffic volume that will yield learnings after four weeks of testing.

Avoid flat experiments.

Expand full comment
author

"On Monday, you’ll map out the problem and pick an important place to focus.

On Tuesday, you’ll sketch competing solutions on paper.

On Wednesday, you’ll make difficult decisions and turn your ideas into a testable hypothesis.

On Thursday, you’ll hammer out a realistic prototype.

And on Friday, you’ll test it with real live humans."

Great gist of the process and confirmation that our PowerWeek™️ (Experimentation Sprint) has a different scope.

In short, Monday to Wednesday must happen beforehand, what we call Sprint 0.

For us, code meets the console after we turned the selected idea into a testable hypothesis.

This book is helping me to enhance our Sprint 0.

Expand full comment
author
Sep 20, 2022·edited Sep 21, 2022Author

"Working together with startups in a sprint, we shortcut the endless-debate cycle and compress months of time into a single week. Instead of waiting to launch a minimal product to understand if an idea is any good, our companies get clear data from a realistic prototype"

Here's the main difference between Google's Sprint and the Experimentation Power Week we're running.

"Clear Data" in their process come from five user interviews.

That doesn't sound appealing to experimenters.

In PowerWeek we focus on getting an experiment coded and live the next Monday (no experiment launch on Friday, you know that). That means our preparation (Sprint 0) requires more energy and attention.

Expand full comment
author

"It was exciting. The sprints worked. Ideas were tested, built, launched, and best of all, they often succeeded in the real world. The sprint process spread across Google from team to team and office to office. A designer from Google X got interested in the method, so she ran a sprint for a team in Ads. The Googlers from the Ads sprint told their colleagues, and so on. Soon I was hearing about sprints from people I’d never met."

We're looking to create new ways of working for innovative experiments that are wildly different than the incremental marginal gains "BAU type" of test the rest of the org is doing — some would call CRO tests.

We expect the system to spread as other PMs and EMs become keen to replicate.

Growing this muscle is about getting enough reps 💪🏽

Expand full comment
author
Sep 16, 2022·edited Sep 29, 2022Author

"The other key ingredients [for Sprint effectiveness] were the people.

The engineers, the product manager, and the designer were all in the room together, each solving his or her own part of the problem, each ready to answer the others’ questions."

In this remote-first world (which I love don't get me wrong), when was the last time you got everyone into a room for extreme focus solving a business challenge? 🤔

Expand full comment
author
Sep 16, 2022·edited Sep 16, 2022Author

"Improving team processes became an obsession for me (yes, weird again). My first attempts were brainstorming workshops with teams of engineers."

The reason I prioritized this book is Experimentation Sprints 🎯

We recently finished our first round of PowerWeek™️ and it's fascinating to see what a group of talented engineers can put together in five days with enough preparation 🚀

After V1.0 we feel we're missing method to the madness.

I expect to pick some gems from how Jake did at Google Ventures 🤓

Expand full comment
author
Sep 16, 2022·edited Sep 16, 2022Author

"I took a hard look at my habits — and saw that I wasn’t spending my effort on the most important work. So I started optimizing. I read productivity books. I made spreadsheets to track how efficient I felt when I exercised in the morning versus at lunchtime, or when I drank coffee versus tea. During one month, I experimented with five different kinds of to-do lists."

This book opens with a banger 💥

Habits. Productivity. Optimize. Experimented.

It checks all boxes ✅

Off to a good start 🚀

Expand full comment