Finding Great Product Managers
In the few short years before we aren’t mostly replaced by AI, we’ll still need to hire a few humans. Here are some classics on product manager hiring along with value-added commentary. Plus a bonus.
The “Google way” of hiring, as presented in a widely-read article, is often cited as a model to follow. However, despite its popularity, it is not being implemented enough.
Research shows that unstructured interviews are only able to predict 14% of future job performance. Slightly less useless are the case interview formats used by many top-tier consulting firms. These formulaic interviews can be aced with practice, aided a well-established industry.
Meanwhile, the product management hiring industry has apparently made a fetish out of its own formulaic interview questions which only prove that you’ve exhaustively studied the rote patterns they rely on. For example, “Estimate the number of drones necessary to complete all deliveries for Amazon in NYC” only proves you’re moderately clever in a very useless way.
By contrast, work samples, cognitive testing, and structured interviews all provide more accurate predictions, with cognitive testing and structured interviews each offering roughly twice the predictive power of unstructured interviews.
While work samples can be challenging to obtain, tests of cognitive ability, such as those offered by Criteria, ESkill, or PredictiveSuccess, are a viable alternative. Unfortunately, despite their effectiveness, few employers make use of cognitive tests, maybe due fear it turns them off, or sensitivity to accusations of systemic bias.
Structured interviews do about as well and they already are widespread. These ideally combine behavioral questions (“Tell me about a time…?”) with situational ones (“What would you do if…?”) Combining cognitive tests with structured interviews yields even better results.
Even with just structured interviews, institute a rubric, or else you will not be able to track everyone either accurately nor fairly. Should your rubric evolve as you proceed, you may re-engage earlier candidates using the revised rubric. Here’s one such example:
https://www.rocketblocks.me/blog/interviewing-product-managers.php
Ken Norton’s essay suggests questions that can be used in a structured interview. He also highlights the significance of product Spidey sense (#3). This sort of insight requires at least one unstructured opportunity, which might come during the “Do you have any questions for me?” phase. Here’s one possible rubric for that phase:
Freshman. The candidate asked generic questions.
Sophomore. The candidate asked questions that show they did their research.
Junior. The candidate asked questions that show they did their research, and also made some interesting points.
Senior. The candidate asked questions that show they did their research, made some interesting and some seriously deep and well-considered points.
For candidates passing the structured portion of an interview in equally flying colors, the questions they ask and the observations they make can be a key differentiator.
- Hire smart people by posing some analytical questions.
- Bias hires toward formerly technical people because they communicate better with the engineers and they also can translate technical details into lay terminology. That is, unless the product job is decidedly non-technical.
- Look for innate product smarts, which might be revealed by candidates who actually say interesting things about your own product (particularly those that make you wince). People with these smarts have an indefinable edge that others don’t show, although you have to give a candidate enough room to show it.
- Look for the ability to lead, which is super important because product managers typically have no directs. This can be partially revealed by a set of questions.
- Can the candidate process multiple points-of-view? Important, as their job is to mediate between diverse stakeholders.
- Beyond everything above, find someone who’s actually shipped product.
Marty Cagan’s essay describes qualities that are probably better gauged for internal hires, as some are difficult to reveal in external interviews.
- In a job that basically has no daily routine, the candidate had better demonstrate passion for the craft and the product.
- Empathy for the customer’s point of view.
- Smarts, which can be demonstrated by repeated problem solving. Just not those inane word math problems.
- Integrity, which is needed to influence others, for without integrity they won’t trust you. This is a really hard quality to measure, though.
- Confidence, which is a critical element in effective communication.
- Communication skills, which should become evident in the interview. I’d add, give the candidate plenty of room to talk.
- A good attitude, because things often don’t work out quite as planned. In my experience, success is measured by all the things that go wrong before things go right.
- Comfortable with technology, and eager to learn the technical specifics of the product.
- Focus, which is needed to keep one’s eye on what matters and not get distracted. A distracted product manager distracts the team and detracts from what’s important.
- Writing and presentation skills.
- Business skills, because many of your colleagues and stakeholders will be on that side of the house. However, hiring an MBA is often a recipe for failure.
This meta-study of research on personnel selection probably underlies part of the “Google” method.
a) The sole standout test is a general measure of mental ability (GMA), referred to above as “cognitive testing”.
b) GMA coupled with a measure of personal integrity is an excellent predictor of job success. While a measure of personal integrity is not itself highly predictive, it is highly uncorrelated with GMA, thus boosting overall predictive power.
c) GMA coupled with a structured interview is nearly as good as coupling with a measure of personal integrity.
Use of integrity tests must however come with a fair amount of education; consult Further Reading below for a study from the US Office of Technology Assessment. Integrity tests also should be subject to interpretation depending on the hiring circumstance, for example, certain attributes such as risk-taking may sometimes be desirable.
Bonuses
This article by Daniel Rizea adds a number of great process notes.
This post by the 280 Group emphasizes the fact that, no matter what I’ve said above or even what your colleagues tell you, you’re going to have your own emphasis on how to choose a PM.
https://280group.com/product-management-blog/choosing-a-great-product-manager
Conclusion
I’ve relied largely on authoritative sources for this article and so I’ll take a bit of latitude in this conclusion. My friend Sameer Ahuja of the fabulous company GameChanger recently told me that their finalists do decks. It might go like this:
1) Take an actual problem at your company and scrub it as necessary.
2) Ask your candidates to work up a treatment for this problem.
3) Choose the candidate with the best work.
This is a fabulous approach because at the very end, the choice between highly qualified folks probably is random. Having your candidates provide work can clearly differentiate them. I once did something like this, giving a bunch of programmers the same work assignment. We hired the one who produced the best work. We were asking people to do real work and so we paid them. Only fair, right?
4) Pay everyone for their time. Otherwise you’re just being cheap and exploitative.
There also is an excellent chance that you as the interviewer are going to say “Tell me about yourself,” which doesn’t fit neatly into anything above. But I’m not going to dissuade you! If the candidate has done their homework, they will have woven the job description into their response, showcasing their thoroughness and ingenuity.
Further Reading
The Use of Integrity Tests for Pre-Employment Screening
