Growth Experiments
Feedback management systems for experimentation
Product intuition, the #1 concept from Mastering Product Management, my latest Reforge cohort.
When I add big ideas to the knowledge base, I exercise first principles by reviewing morphology, how words are formed, and their relationship to other words.
As I dig, Reforge's definition felt too broad. What is the limit for understanding something without the need for conscious reasoning?
I don’t have that answer, so I decided to leave the question aside for now.
Obsidian, the shiny new tool I'm using for knowledge management, will eventually bring me back to it.
—
Assuming a Product Manager can develop unique Product Intuition, what are the tools and systems that support their learning curve?
This is a better question to explore as I am positive I can add value.
I will surface methods seen in the cohort adding my view and experience.
Feedback systems
I expect readers to agree that testing hypotheses are as strong as the research backing them up, otherwise, we're at the unsustainable spaghetti testing territory — Ben Labay has strong opinions in the matter.
When operating in smaller companies, and supporting few product teams, research is ideally prioritized with a decision-first approach.
This system will output a manageable backlog of research that's worked over sprints.
How does that work at scale?
What happens in larger organizations where experimentation supports multiple product teams and research is done by multiple supporting departments?
In an ideal world, information is centralized by a processing system that helps Product Managers to be on top of learnings generated across the business while preserving context.
When I first saw the image above, my agency days came to mind where I would explain the research backlog to clients using the CXL Research system.
When I started with Conversion Rate Optimization at Ladder.io, wearing multiple hats was common practice and research operations looked much more waterfall.
First, we will run heuristics analysis, then review your analytics, followed by user recordings and heatmap reviews.
One after the other, after the other. So the client was constantly exposed to deliverables. Agency life.
As I experience the reality of more mature product-led organizations with a continuous discovery culture, the research dynamic changes from push to pull.
This 'push' dynamic removes the need for the Product Managers to remember to look at feedback, and the steady flow of new feedback creates a regular trigger for the PM to go read it.
System components
I love how Reforge put this together.
Product leaders will develop intuition from constant exposure to qualitative and quantitative insights from a feedback river.
On the other hand, the feedback system of record shows the count aggregate of themes that bubble up from the feedback river eliminating recency bias.
Feedback rivers in practice
Imagine Slack channels that host automated messages from internal tools and templated summary findings from supporting teams.
Using the ResearchXL lingo, the configuration I have for one business I advise follows this structure:
#user-insights — User testing, copy testing, and qualitative surveys are all aggregated and maintained by the UXR chapter.
#analytics-insights — Web analytics, technical, and market analysis are all aggregated, ran by product or business analysts, and maintained by the Analytics chapter.
#experiment-results — heuristics analysis, mouse tracking analysis, and relevant experiment learnings are all aggregated and maintained by the Experimentation chapter.
Feedback systems as an onboarding tool
I should not have to play detective in order to learn the goal of a project, different areas of work, or where to go to learn more.
— Naomi Gleit, Canonical Everything
Imagine how much onboarding of new members could be accelerated by having this system in place.
The one place to understand what users are telling us, what trends and patterns we see in data, and what are we learning from experiments.
If you take on side projects, I believe you can visualize a much smoother first-month discovery by having access to a feedback system.
Is this for everyone?
I don't think so.
If you are wearing multiple hats as part of a start-up maintaining those communication channels alone might be unsustainable.
If you are an agency servicing multiple clients you will need a massive resource to justify a push dynamic. Decision-first approach tends to be a more cost-effective model.
If your business is taking the product-led approach, expanding the Product Management function, and investing in supporting departments the system will support growing product intuition across the business at a faster pace.
Q: Do you have a push dynamic for research?
Life as a product
Strengths assessment
Emotional intelligence made it to the shortlist of skills to 10x in 2022.
During my discovery to understand what to do, I learned from Daniel Goleman about the five components of emotional intelligence at work.
With a prioritization spirit, I sought guidance and captured one comment and one challenge:
- Self-awareness is an important personality trait to look for when hiring leaders.
- How could one feel self-aware when they struggle to define weaknesses?
As Goleman defines in his epic HBR post What Makes a Leader:
Sometimes called maturity, self-aware people know their values and goals.
They speak accurately and openly about how their emotions impact their job.
They are open and welcome failure as the desire for feedback and self-deprecating humor.
Self-confidence is a common trait as they understand their capabilities, strengths, and shortcomings.
Cliffton Strengths
My coach gave me one initiative to pursue: Go through the Gallup's Cliffton Strengths assessment.
Here are my top five:
I understand these assessments are written to make it easy to self-project oneself and find commonalities, just like horoscopes and numerology stuff.
To balance my skepticism, I took three different assessments, from Jordan Peterson, Cristal Knows, and Gallup — they all converge.
I get value from describing my strengths better, calling them by the right name, and being specific about my weaknesses. No more I’m a perfectionist.
Doubling down on strengths
I believe doubling down on strengths is not just avoiding uphill battles, it's accepting reality the way it is and working with it.
When I play according to my strengths it's a lot easier to outperform by merely “doing the average thing when everyone else around you is losing their mind.” — and that's a Big Skill.
What's next
I'm taking my time to digest the information I have with a performance coach.
My objective is to have a clear understanding of my blind spots and collect feedback on what to do about them.
There's no perfect professional profile and I made peace with that.
I want clarity about what holds me back from the next career leap.
A practical example that I'm pondering, from being "Focused".
When you are absorbed in something, you may be slow to respond to others’ immediate needs and appear emotionally distant. Acknowledge that sometimes, you will have to stop and respond to interruptions from the important people in your life.
—
I continue to pursue clarity and self-awareness for a more rational life.
Q: Are you confident in defining your strengths and weaknesses?
Handpicked from other brilliant minds
Naomisms
April was the month I found Naomi Gleit. After 15 years at Facebook (Meta), she opened up a series of short and practical principles she collected throughout her journey up to VP of Product.
Her writing is refreshing. Reminds me of Extreme Ownership.
Standing on the shoulder of giants.
First-principles thinking
First-principles thinking means boiling things down to “the most fundamental truths” and then reasoning up from there.
– Elon Musk
In Mastering Product Management, Reforge explored the concept of lever dashboards and the right rituals that allow PMs to develop stronger Product Intuition by constantly looking at the right metrics.
One of the exercises dissected growth companies' metric driver trees. I run something similar with the teams I coach in preparation for OKR cycles.
The concept reminds me of a great post from Bhavik Patel, from CRAP Talks, on how people struggle to build a coherent picture from top-line revenue to actionable levers.
20VC — The memo
Growth is about connecting people to the value that's already been created by the business.
— Casey Winters
The most value-packed podcast I heard this month. The LinkedIn Post reached 10K people.
Time analysis — April 2022
22 hours reading or listening to audiobooks;
13 spent studying the financial market;
24 producing content;
19 networking;
10 meditating;
18 days traveling — 27% total year to date.
Meditation weekend — Vajrasana
“Meditation as a superpower”, posting soon on Timeless thoughts.
There's a white paper available which is a great read as well, check it here.