Imagine If with Josh Kopelman

New episode of the Decision Education Podcast just dropped!

Dec 12, 2024

How do you know a good founder when you see one?

Josh Kopelman, a co-founder and managing partner of First Round Capital, and I tackle this question by exploring the transition from gut feeling to quantifiable evaluation in venture capital. I have been working with First Round for over five years now, re-imagining and refining their decision process, and working with the team to take a data-driven approach to answering difficult questions.

In this conversation, we highlight First Round Capital’s rigorous process for assessing startups, and the role of probabilistic frameworks in enhancing decision quality. The goal is to mitigate the difficulties of investing in companies at the earliest stages, when there is the greatest influence of both incomplete information and luck.

Key takeaways from this episode include the importance of making the implicit explicit, addressing long feedback loops, and the necessity of truth-seeking and humility for effective decision-making.

This is The Decision Education Podcast, where we explore the science of decision-making with a diverse range of experts and offer practical strategies to transform our understanding of the role decisions play in our lives.

Thanks to First Round Capital for supporting The Decision Education Podcast—empowering leaders to make choices that shape our future.

Transcript and Show Notes Below

Guest bio

Josh has been an active entrepreneur and investor in the internet industry since its commercialization. While a student at the Wharton School of the University of Pennsylvania, Josh co-founded Infonautics Corporation and took it public on the NASDAQ stock exchange. Josh founded Half.com in 1999, and led it to become one of the largest sellers of used books, movies, and music in the world. Half.com was acquired by eBay in 2000, and Josh remained with eBay for three years, running the Half.com business unit and growing eBay’s media marketplace to almost half a billion dollars in annual gross merchandise sales. In 2003, Josh helped to found TurnTide, an anti-spam company that created the world’s first anti-spam router. TurnTide was acquired by Symantec just six months later.

Josh co-founded First Round Capital in 2004 to reinvent seed-stage investing. Since that time, the firm has invested in over 300 emerging technology startups, becoming one of the most active venture capital firms in the country. Josh has consistently made the Forbes “Midas List” which ranks the top 100 tech investors, earning the number four spot in 2014. Josh was named one of the top ten “angel investors” in the United States by Newsweek, one of “Tech’s New Kingmakers” by Business 2.0, and a “Rising VC Star” by Fortune.

Josh earned a bachelor of science degree cum laude in entrepreneurial management and marketing from the Wharton School of the University of Pennsylvania.

Transcript

Note: This transcript was created using AI. Please excuse any errors.

Annie: I’m so excited to welcome my guest and my friend today, Josh Kopelman. Josh has been an active entrepreneur and investor in the internet industry for decades. Josh founded Half.com in July of 1999 and led it to become one of the largest sellers of used books, movies, and music in the world. Half.com was acquired by eBay in July 2000.

Annie: Josh co-founded First Round Capital in 2004 to reinvent seed-stage investing. Since that time, the firm has invested in over 300 emerging technology startups, becoming one of the most active venture capital firms in the country. Josh is a graduate of the University of Pennsylvania, where he earned a B.S. cum laude in entrepreneurial management and marketing from the Wharton School. He is the recipient of numerous honors and awards for his work in internet technology and investing. Josh was named one of the Top 10 Angel Investors in the United States by Newsweek, one of tech’s new kingmakers by Business 2.0, and a rising VC star by Fortune. He was also awarded Ernst & Young’s prestigious Entrepreneur of the Year Award for the Greater Philadelphia Region. Hi, Josh.

Josh: Hi, Annie.

Annie: Thank you so much for coming on the podcast. I’m so excited to chat with you. Just for listeners of the podcast, Josh and I have known each other for quite a long time, actually, over five years, if you can believe it. And we’ve worked together for the majority of that time. So normally when I have a guest on I’ll talk about how we met, but I thought it’d be fun in this case, because I actually haven’t heard it from your perspective, for me to hear from you how we met.

Josh: It actually ties to a podcast. It’s very meta. My partner, Brooke Burson, heard you talking on The Knowledge Project.

Annie: Shane Parrish’s podcast.

Josh: That’s right. And sent it around. And I’m not much of a podcast guy. I find that I process much quicker reading than I can listening. But I just started listening to it on my way home and I could not turn it off. So like the next day I had Thinking in Bets on my Kindle and read that whole thing. And as a venture capitalist, I’m in the decision-making business, and our returns are going to be heavily driven by the quality of the decisions we make and the ways we could sort of try to avoid or minimize biases. And so I wanted to reach out to you. So I found someone who knew you. I felt like I was cyber stalking you. I, you know, the fact that we both sort of live in the same neighborhood made it easier for us to grab breakfast. And I didn’t know when we started that we would be working together for five years. I just thought that I could learn a lot from you. And so I went into it, just, let’s grab breakfast, and one thing led to another, and you’ve had a profound impact on the way I and my firm thinks about making high quality decisions.

Annie: Oh, so my recollection is some rando reached out to me. And, no, you weren’t a rando to me, but we had breakfast. So actually relatively similar. I just actually didn’t know the beginning part.

Josh: And it was resulting in particular where I felt that our whole industry sort of, I don’t think, differentiates between the quality of a decision and the quality of the outcome and solely looks at the outcome to determine whether it was a good decision or not. And sort of hearing about that, then, you know, every other chapter I’m like underlining half the chapter, and that’s when I said, like, wow, we can learn a lot from Annie.

Annie: Oh, that’s awesome. Well, we’re going to get into that. I will say that working with you has made me a much better decision maker myself. Like, the practical application into the world of venture, I think, is really interesting. It’s why I stayed so engaged for so long. So I’m going to be very excited to talk about the work that we do together. But so, you mentioned, obviously you’re a venture capitalist, founding partner at First Round Capital investing in seed stage. For those who aren’t familiar, seed is very early in terms of being more on the institutional side. So there are certainly angel investors who come in and they write like a first check. But as far as once you get, you know, sort of what people understand as venture capital, seed is going to be the first place. And your firm was really instrumental in defining that as an official funding round in venture capital.

But when you’re funding at seed, there’s often no product. There’s just an idea. There certainly isn’t what we call product-market fit, where it’s the right product for the right market. You’re often investing in just, in a founder and an idea. And you don’t really know a lot else. So if we think about, you know, what’s going to be the place of the greatest uncertainty in the investment cycle, you can look at that versus, for example, what the work of somebody who is investing in public markets.

Josh: Or even a later stage investor, right? Later-stage investors have revenue to look at history, forecast, board decks. So much of what we do, I joke, like, we invest at the imagine if stage. I think actually the two most powerful words in humankind’s history is “imagine if.” Right? No great movement, no great concept, whether you’re a social movement, a form of government, whether you’re a company, whether it’s, imagine if, you know, we could have a reusable rocket, imagine if you could push a button and a car turns up, or imagine if all mankind is created equal. Like, every great concept starts with someone thinking about how it could be different.

And so, we invest purely at that stage. There really is very little often in terms of proof points. And even if there are, like, some markers, you don’t really have a product that has hit a consumer’s hand yet.

Annie: So, I want to really dive deep into kind of like the decision-making process at this early stage, at this imagine if stage. But I want to just sort of like rewind for a second because you yourself were an imagine if-er when you started.

Josh: Yes.

Annie: I’m using that as a term now. I love it. Because way back when, pretty soon out of finishing business school actually I think it was, was it during business school?

Josh: It was undergrad. I was a warden undergrad.

Annie: Undergrad? During undergrad?

Josh: Yeah, my sophomore, after my sophomore year, I co-founded my first company.

Annie: Right. So can you tell us a sort of a little bit about that first company? Why did you decide to become an entrepreneur? And then how did that lead into kind of getting on the other side of startups?

Josh: Yeah, I think I always had a fascination with problem solving through business. As a kid, I would always sort of, you would see a problem and say, “Oh, how could I make a business out of this?” Whether it was when those old big VHS camcorders came out that you would put on like, I started videotaping bat mitzvahs and birthday parties, you know, like imagine if I could take this VHS camera and make some money doing it.

And so the first company I co-founded was a company called Infonautics Corporation in 1991. And this was back before the web browser was invented. So you had Prodigy and CompuServe and AOL and dial-up service.

Annie: And obviously were like 11.

Josh: I was 18 or 19 at the time. And the imagine if was imagine if as a student, you didn’t need to go to a library, but you could actually search that library. And this was before any search engine existed, because there was no web browser. So we had a slightly different and ultimately wrong approach. But the approach that we had was, we would go license all of the great content. And we licensed the L.A. Times, New York Newsday, CliffsNotes, the encyclopedias, you know, hundreds of magazines. And what was interesting is none of them had it in a digital format. So we would get all of those magazines, send them off to the Philippines to be scanned and reviewed and digitized, and then we made it searchable. And with a product called Homework Helper on AOL, on CompuServe, on all the dial-up services. So I think we picked the right problem, right? There’s a lot of information that was trapped in libraries and in books and people would want to be able to search through it digitally. We just had the wrong solution. The company ended up going public, we did okay, but there were many other people there, you know, a few years later who said, imagine if you could search all of this content and they didn’t have to actually license all of it, scan it all, because it was already on the internet.

Annie: All right, so Infonautics, first company that goes public, the next company you found is Half.com?

Josh: That’s right. Yeah. I co-founded Half.com with someone who I’d worked with in Infonautics. Yeah, Half.com was, it’s funny because the original name that I thought of was Ebazon because I wanted to combine the best of eBay with the best of Amazon. But our lawyers told us even though eBay and Amazon compete, if there’s one thing they both would agree on, it would be suing me. So we built a different name, Half.com. But that was basically the concept, which is that eBay was an amazing marketplace for peer-to-peer commerce, but it had one dynamic, auction.

And auctions are great for limited, hard-to-find, scarce items, but like, when you go to like Old Navy or Gap to buy a pair of jeans or Lululemon, you don’t start bidding with the person next to you on those pair of jeans or those leggings or sweatpants, you just buy them at a fixed price. And so we said, what if we could combine the best of Amazon, which was a catalog, photos, reviews, a shopping cart, credit card purchase, with the best of eBay, which is instead of warehouses, we have the goods in people’s houses and people could be the underlying sellers. And we became one of the top 10 e-commerce sites when we launched in 1999, 2000. eBay quickly acquired us, and then Amazon launched Marketplace shortly thereafter.

Annie: That’s amazing. Obviously you’re an entrepreneur at heart. It sounds like you love the idea of what if, and then actually, or at least at that time, you loved the idea of what if, and then actually executing on it. Whether it was, what if I could videotape people’s bar mitzvah? You know, what if you actually could search anything that you would find in a library without actually having to go to a library, you could find it online, digitized, it would help you with whatever you were doing research on, reading magazines, so on and so forth.

So you’re like a real what if, but like a, what if, and I’m going to execute the what if person. So, how do you end up transitioning to the other side of the equation, which is investing in what if people?

Josh: Yeah, I think a few things. The first is in all of the companies I started, my favorite time was the first 18 to 24 months. So much gets baked during that time. It’s the hunt for product-market fit. You’re figuring out the problem you’re solving. You’re defining the solution. You’re figuring out your pricing, your positioning, your go-to-market. You’re building your culture. You’re hiring your core team. Like so much gets baked in the first 24 months. I created a Peter Pan type of job where I’d never have to grow up. I could just always help founders on their first 24 months of their journeys instead of just sort of you know, and all of these companies have grown, right? Half.com operated for, even after eBay acquired it, for over a dozen years afterwards.

I also felt that you know, if a founder wears a hundred different hats, you know, I could put together an incentive comp plan just as good as any other CEO, but I don’t think I added much value. And perhaps the areas where I felt that I had, like, any margin-value add were more around the challenges of those first 24 months. It’s a lot of invention combined with execution and the rapid iteration of feedback loop. And so by choosing to co-found a seed stage firm, which I did in 2004, so about 20 years ago, you know, we named it First Round to sort of force us to remember that there’s something special about those first 24 months.

And unlike the multi-stage firms, and unlike the firms that have many entry points, we think we have built a product and a practice that’s specifically focused on partnering with the founder at the imagine if stage and like helping them go from like imagine if to what’s next, right?

Like, it’s one thing to have the goal, but then how do you break it down to what do you need to do this week, this month, this quarter? And that’s kind of the practice that we’ve tried to build.

Annie: I want to sort of key in on something that you said because I, it was a conversation that I had with you in the beginning. I think, during our first meeting and also the second time we really talked on the phone where we decided that we were going to do a trial of working together because you like trials which I do too. So our first engagement was actually sort of hour by hour, which was like fun by fun, and then, once we did sort of figure out whether we liked each other, we decided to keep going.

But prior to talking to you, I’d actually talked to a few other pretty big firms. And it was interesting. One of the pushbacks that I got, and we had a little conversation around this as well, was that, you know, a lot of the work that I do is really trying to think about how do you close feedback loops in a way that you can get away from this resulting problem that you mentioned. So venture has a huge resulting problem, right? So if you happen to have invested in Uber, everybody thinks you’re awesome, but it’s a little hard to tell if you’re awesome, cause you kind of have to know why the person invested in Uber. Was it that like their college roommate just said, “Oh dude, you should write a check.” Right? Which I would point, I shouldn’t really think very highly of you that you happen to get into it. Or are you actually a really good decision maker and you happen to get an outsized outcome, but your decision process actually maximized for the possibility of getting an outsized outcome? And this is actually a really hard problem.

The more uncertainty that you inject in something, the more that particularly when power law applies, which in venture it does, so very small percentage of the investments are going to create the majority of the returns. Now luck plays a very big role particularly in the short run. So the pushback that I got was, okay, so you’re like the resulting lady and you really talk about closing feedback loops. So how are you supposed to help us? Because we don’t even know what the outcome is for 10 years often. So this was the most common push back that I got. We had a little discussion about this as well. And I would just kind of love to hear, because, you know, we ended up on the same wavelength around this issue. But I do think that this is a big problem, not just in venture, but in anything there’s a lot of luck involved at poker is like this as well, right?

How do you think about when you have these very long feedback loops, which is where venture and poker diverge, right? Because poker has shorter feedback loops. When you have these very long feedback loops, as you think about it, between the day you invest and the day the company either exits or you know it is dead, right? So that can be years and years and years and years. How do you think about overcoming and addressing that issue in this type of decision-making environment?

Josh: Yeah. So, it’s even harder than that because in many times you get early data that is wrong. Right? Like First Round has invested in over five companies that were worth over a billion dollars on their way to zero. And so I think that the way that we like to refer to it internally is that, you know, if a founder comes in with their imagine if, that’s kind of, you know, imagine if we could press a button and, you know, a car could appear anywhere in the world, you know, within five minutes. That’s like summiting Everest. And yeah, it’s really, you’re, you’re not going to summit Everest for 10 years. You’re also not going to summit Everest without hitting base camp. And I think what we try to figure out is what are all the steps along the way, and how can you measure and learn from those, right?

So, again, are there examples of companies that raise one round of financing and then go on to be worth billions of dollars? Sure, you could find examples of it, but in general, the vast majority of companies, after raising a seed round, raise an A round. So there’s some data there. There’s signal. You can say, how long did it take them to raise an A round? And is that predictive? If a company raises an A round within 6 months, Where you know, you can look at data, you can look at all the A rounds and all the C rounds and see, is time from C to A predictive? Is size of the A round predictive? Is quality of the firm that leads it predictive? And so, in terms of, does that give you signal of increased likelihood to get from base camp to the next camp to the next camp? So I think that too many people might use like IRR or multiples as a proxy on that. But we found that multiples are very misleading along the way. So instead, we try to find, you know, because by the way, we haven’t found that the multiple of an A round is the most strongest signal. In fact, it’s one of the weaker signals that we’ve found.

Annie: I think it’s also a very, what it’s the strongest signal of is what, what’s happening in the market in general, which is not helpful for knowing what’s happening with the company, but is it a hot market? Is it a, is it a contracting market, for example?

Josh: Right. So we’ve spent a lot of time trying to identify what are the things you could look at in months six to 18, what are the things you could look at 36 months out that could maybe give you some confidence or color where you don’t have to wait 10 years to collect a feedback loop?

You could begin to shrink those feedback loops. So, put it another way, like if X percent of all of our companies turn into billion-dollar outcomes, at the time we make the investment, like, if you asked me to say what are the odds that this one company would be a billion dollar outcome? I would say it’s X, because I don’t know. So if X is 6 percent or 12%, at the time of investment, it’s just random, because you don’t have the ability to differentiate.

Twelve years in, you might be able to sort of say there, I could look at that data and say within, with a 90 percent confidence level, I could tell you whether this is going to be a billion-dollar company or not. So like, the goal is to try to figure out how can you move that sooner? How can you move that from 12 years in, to 8 years in, to 5 years in, to 3 years in? And, now, you won’t have 90 percent confidence 3 years in, but maybe you could have 3x or 4x confidence at that point in time. You know, you could be meaningfully above base rate in terms of knowing that. So, those are the types of things that I think we’ve tried to do and you’ve also helped us to try to assess.

Annie: Yeah. So I think the way that I put it, in talking to some other firms before you and then talking to you was I kind of said two things. One is, you’re not Rip Van Winkle. And I think this is a really important concept in decision-making, particularly ones where the feedback loops are quite long and people kind of say, I’m going to go with my gut because I’m not even going to know what happens for 10 years.

And my response is always, you’re not Rip Van Winkle. It’s not like you make the investment and then you go to sleep for 10 years and then you wake up and you go, “Hey, what happened?” Right? Like the world doesn’t stop in between the day of the investment and the day of the exit. There’s all sorts of stuff that’s happening in between and that stuff is actually telling you things about what the quality of that investment is long before you would wake up 10 years later if you just went to sleep.

And I think what’s really important for people to understand the difference between necessary and sufficient. And I think this is a concept that’s really gotten, you know, enculturated at First Round is that what you’re looking for things that are necessary for the company to succeed. But note, those things clearly not sufficient, but obviously if it’s necessary, then the fact that you meet that thing that’s necessary means that the probability that it’s going to succeed has gone up. And if you’re better than average, at steed stage, when, when you’re just in the what if, if you’re better than average at predicting that this company has a better than average chance of hitting those necessary milestones, then this is actually going to help to clean up your decision process because, and I think this is like so key, right, is that so I remember having a conversation with you about whether you thought in an explicit way about the probability a company would fund it, say Series A, you know, at the time that you were investing. This is something that I think you were thinking about an implicit way, but not an explicit way.

Josh: Yeah, we hadn’t clarified it. You’re right.

Annie: And I think that what I said to you and you responded to immediately was, “The reason why I want you to make it explicit is because it’s already implicit, you’re already doing it anyway.” And because a company cannot succeed, I mean, again, there’s a rare case here and there, but mostly a company cannot get a billion dollar exit that does not fund at the next round, right? So you have seed and then you have series A. If it doesn’t fund at series A, it’s not going to go anywhere.

Josh: And there’s not just, you know, you learn a lot about other things. So, you know, we’ve, at First Round, we’ve, you know, developed a real framework for the four levels of product market fit from nascent, which is the imagine if stage, through developing strong and extreme. So how quickly does a company navigate hunt for product market fit to get there? There’s signal you get there. There’s signal you get about the founder and their resilience. are they solely just imagining or can they actually execute and what’s the cycle? So I think that trying to quantify or measure some of those things could provide, just even saying there are four levels of product market fit. We want to measure how quickly a company hits it. That now gives us a data artifact that could be valuable. We don’t always know at the beginning whether a data artifact will be valuable or predictive. And we’ve measured many, many things that, you know, have no correlation, but.

Annie: Some which were negatively correlated.

Josh: That’s right. So it’s not just like external funding. It’s things like, you know, it’s trying to understand you’re going to spend a lot more time with the founder after the investment than you do before. What are the signals that you can look for? You’re going to see that product get into the market. What are the things you can learn about?

Annie: And then, of course, you can measure those things, right? So, basically, what you can say is, in the companies that you invest in, are the companies hitting those milestones at a higher rate than the companies that you choose not to invest in, for example, right? Are they achieving level four product market fit more quickly than companies that you don’t invest in, right? So you can take the things that are signals that that company is going to be more likely to succeed. And then you can compare, here’s the things that we choose to invest in. Here’s the things we choose to pass on because you have both parts of those finally. You have the portfolio and the anti-portfolio, right? And then you can say, The things that we think are signals where we can start to close these feedback loops more quickly than 10 years from now, are the companies in the invested basket actually achieving those things more quickly? And then that tells you what’s the quality of my decision-making? I have certain things that I get to decide about, which is a go, no go question, right? Do I invest or not? And in the things that I choose to invest in, are they actually achieving those things more quickly than in the things that I don’t choose to invest in?

Josh: Yes. So, I mean, one of the things you’re talking about is how to use data. And so, you know, my second company, Half.com, we were the largest seller of used books, music, and movies on the internet 25 years ago or whatever it was. And I saw Amazon crush Borders Books. And what was interesting is Borders had more customers at the time. There were more people walking into Borders every day buying books than shopping on Amazon. And other than the selection and everything, like one of the things that, that I realized was Borders whenever I walked in had no idea who I was. So like, imagine if Amazon erased its customer database. At the end of every day they just said, Okay, we have, we wanted all of this about our customers. Let’s erase it. That is what Borders did like by default and I think that as VCs, we’ve seen through our companies, that some of the best companies are companies that really are instrumental to collect data and learn from it.

You could look at their KPI dashboards, you know, I haven’t seen an e-commerce company that doesn’t have an incredibly well-designed dashboard where they’re tracking conversion funnels and offers and margins. And yet, when I look at venture capitalists, I think most venture firms run like Borders Books. You know, a founder would come into a partner meeting. These partners would get together, talk about it, make a decision, and there’s no artifact, no data, nothing recorded. A company would hit product-market fit or not, and it would never be recorded. And so I feel like, again, I’m not saying to trust an algorithm. I’m not saying you want to outsource your decision-making to a data model. But I am saying that you can learn an incredible amount by tracking and collecting over a period of years and in our case, now almost 20 years. By collecting that data and learning from it you could figure out like, “hey, I thought X was important, but the data says it’s more important or not as important.” And it’s a way to sort of perhaps, like, make sure you’re constantly stress testing your thinking and learning. And so, like, as a firm, we are not building a model to outsource our decision-making to, but we are collecting data to try to evaluate the quality of our decisions and get better at it.

Annie: So I want to talk about this idea of like, how do you sort of sit in that middle between, I’m just going to go with it, right? I just kind of feel like I know what I’m doing, sort of period. And algorithmic decision-making? So I want to just sort of mention something that I heard from a VC, not one at First Round, which has really seared into my brain, which was, I asked about their decision-making process. Because I wanted to just understand like what was the process sort of as things move through the funnel, like, “How did you actually think about and model what a good decision maker was?” And the response was, “Oh, we just know a good founder when we see one.” And I said, “Well, what does that mean?” And they were like, “You know, we’ve just done this a long time. We just know a good founder when we see one.” So they couldn’t actually articulate like what they meant by that. And they were just like, hey my gut says this is great. Meanwhile, if you find out a 60-inch television is 20 dollars cheaper, but you got to drive all the way across town to go get it, people will actually do that even though it costs more in gas. So like, maybe don’t go with just what your gut tells you you’re supposed to do. But that’s basically what they were saying is that, “There is something special about me that I can see things that other people can’t see about founders. And that’s what our process is, is we just go with that.”

And I thought that was really interesting and you and I have had lots of conversations about this because my feeling about that was that may be true. Right. You may know a good founder when you see one, you’re probably better than a lot of other people at knowing good founders when you see one, but wouldn’t you want to make that explicit so you can figure out, of the sort of ways that you model agood founder, and we could think about what those are, what are those things are actually predictive and what aren’t actually predictive? And then you can track those things. And then other new people who come into the organization will understand what your model of a good founder is, and they’ll be able to up-level more quickly. And they’ll be able to, I mean, I thought there would be sort of all these good things that would come out of it. They were very resistant to that idea. I think partly because I think when you make this explicit, you do run the risk of finding out that you’re not better than the average bear, and I think that’s very scary for people.

One of the things I love about your firm is that people aren’t scared to find out the truth. Which I think is wonderful. But, what I want to sort of ask you about in this sort of world, and the way that you think about this at First Round is, you have sort of that extreme, which is sort of, I know I do it well, but I also don’t want to know for sure, so I’m going to sort of just say my gut is good at it versus doing it completely by algorithm. But you have humans involved. You have partners who are modeling the problem. They’re meeting with the founders. They’re hearing what the idea is. They’re deciding what the what if is. I think in this particular case, there is very little data at the time that you actually invest. So a lot of it is going to be qualitative judgment because it’s not like later sound stages where you can look at revenue, you can look at churn. You can look at run rate, you know, so on and so forth. Right? So you don’t have all that data and you are just really going by your opinion, but one of the things that First Round that happens is that those qualitative opinions that that first person I talked to was kind of talking about, get quantified in an explicit way.

So first of all, can you just talk a little bit about how you do that so that it kind of sits in between the world between algorithmic and gut. And then why that’s really useful for being able to close these feedback loops?

Josh: Sure. So, let’s imagine you know, a partner is meeting with a company and gets excited about the imagine if and the problem that they’re solving, and brings that founder into a partner meeting. So that founder would present in front of the whole partnership. Today, it’s pretty much by Zoom. And then, as soon as that founder leaves, it would be a conversation rather than a true dissertation but as soon as the founder leaves, we don’t talk about the opportunity. Instead, we all open up our laptops, and we’ve created a rubric of things that we think are important to evaluate and to discuss. So, we call it a pre-vote, and it might take 10, 15 minutes. So, like, on the team, we don’t just say, this is a great team, or this is a bad team. We ask, like, what’s the founder startup fit? Is it an N of 1 team, or do the skills not map to the opportunity? Like, what’s the team’s bias towards action? Do they get a lot done in a short period of time? Or do you have concerns about the pace of execution? How good is their storytelling? Are they, like, profit minded? Are they students of the market? Do they have market awareness? Do they, have they gone unreasonably deep? Do they have good product intuition? Can they articulate the chess game, the series of moves needed to build substantial business? And can they think clearly in conditionals? Do they have a killer instinct? And then, you know, just, are they intensely curious or overly confident? Are they truth seeking or are they salesy? Do they have, you know, do they have clear team spikes or is it unclear? You can see, like, we answer all of these questions and then we’ll do the same thing for evaluating the problem and the market and the product. And we also give free form text.

And then what ends up happening is all of that gets filtered and what we’re trying to find are areas where we could disagree. Because through disagreement, there’s something magical if a partner of mine, say Bill and I, have both been in a 45-minute conversation and I think the market is large and he thinks it’s small or I think the founder is profit minded and he doesn’t because wow, that’s really interesting. We both saw the same thing and have a different framework. So we almost sort the conversation instead of saying, let’s discuss where we agree. And, Annie, you know, we used to spend a lot of time talking about where we were sure we agreed.

Annie: Eighty percent! Eighty percent of the meeting was where you agree.

Josh: Now we just kind of, we actually find the biggest, like, gap and spend most of the time talking about that. Oftentimes those conversations help us make explicit things that are implicit. So if we’re talking about you know, I think the market is challenging and another partner thinks it’s tough. Why? And I can say, “Oh, it’s a hardware company and hardware companies do X.”

And then this person could say, “well, actually the reason I thought it would be easier is because, yes, they sell hardware, but it’s going to be, off the shelf hardware, that third party hardware that’s not custom made, and all the values and software,” and like, you could figure out, like, actually by having this rubric that you could use as the basis of a conversation, you could find those points of disagreement. And in fact, we used to have the point partner, the partner who was sponsoring the company, lead this conversation but we found that that was also, there was too much endowment bias there. Where, Annie if you said, “yeah, I have real concern about the founder’s x.” If I was the sponsor and I wanted to get a deal done, I would listen to what you have to say and instantly I would rebut it every single, “I hear you, but I thought X.” So we now have someone who is not an investing partner actually moderate the conversation. To try to remove some of the bias and the goal of a partner meeting is to make the best quality decision possible. Not like. If I bring a company in, a win is not getting to yes, and a loss is not no, a win is making a high quality decision.

And it’s only by identifying these things in advance, that you could then have the conversation to say, “A, do we all see it the same? And then B, for this specific opportunity, how should we weight these things? You know, if it’s a consumer product, you know, do we really want to understand product intuition more versus if it’s an enterprise sales SaaS play, do we want to understand go to market?” And then we can have those conversations. Once we’ve like reached some level of understanding of each of the attributes, we could collectively decide what’s most important.

Annie: What I love about this process is, you know, a couple of things. I mean, we compare it to the, I just know a good founder if I see one process, one of the things that I struggle with, with that type of process is how do you actually close the feedback loop, right? So I know that you invested in Uber, but was that because of luck? Was that not because like, if a company failed, was it bad luck for you? It’s hard for me to say, because I don’t actually know what it meant. Right? Like, so one of the things that I love about what you’re describing is that you’re breaking down that model into its component parts and making it explicit so that not only do I know what Josh means when he says he knows a good founder, he thinks it’s a good founder, but so does everybody else in the partnership so that there isn’t a miscommunication around that. Because what you think makes a good founder may be very different than what say Brett or Bill or anybody else in the partnership thinks is a good founder and to make those differences explicit I think is really important because you don’t need to share points of view because that’s why you have lots of people involved in the decision.

Josh: There’s also another benefit which is by forcing everyone to answer this before a conversation, we’re getting everybody’s perspective regardless of seniority, regardless of tenure and free from influence. So if I’m the most senior person, yes, I could speak last if I have the willpower to hold myself back. But in this case, if we have six partners, we’re getting six perspectives here. And there’s kind of no room to hide. You can’t just say, “Oh yeah, I thought Annie made a great point. I agree with what she said.” Because we’ve all had to make the points before the conversation even happens.

Annie: Yeah. And then I think the other thing that’s so great about it again is like, if you just sort of say, “Oh, you know, I really like this deal, sort of full stop period.” Right. And, the only thing that, you know, was invest or not invest. I think it becomes really, really hard to then figure out what’s the quality of your decision making in retrospect. Because first of all, the outcomes themselves are going to be driven a lot by luck, right? You’re trying to separate luck from skill here. You’re just trying to increase the probability of getting lucky, is one way that I like to think about it. And you don’t actually know, like, among all the different factors that you would weigh in investing in something, which were you the strongest on, which were you the weakest on, to what degree did you like it? Did you like it a lot, a little, not very much, so on and so forth?

Josh: Yeah, like you know, when Dave Baszucki came in to present Roblox, I barely remember that meeting now. It was so long ago, that my ability to sort of say, what did we like about Roblox from my memory, I think we’d all just, pattern fit and say today, ah, you know, I knew Dave was special, or I knew that, you know.

Annie: There’d be some pretty strong hindsight bias.

Josh: That’s exactly right. And so, the ability to sort of make explicitl the implicit, to try to force everyone to, like, just answer 30 questions, after you’ve had the opportunity to meet with the founder, and hear the way they present it, learn about them, learn about the problem they’re solving, gives you the ability, it gives you game tape, it gives you the ability to look back over time.

And by the way, not just the companies we funded, but the companies, the bad, I was going to say bad decisions, but I don’t know if there are bad decisions, but the companies where we said no and the company went on to be successful, you also have that. You could understand exactly, you could context load and understand exactly what you were thinking. And then as you, when you think about bringing new partners in, they now could look back at this game tape for every major outcome, either positive or negative and, and what were people thinking at the time.

And over time, if you collect enough data, you might learn that, like, does our assessment of bias to action on the first day we meet the founder, does that have any predictive ability beyond base camp? Does that predict the likelihood that they will summit Everest or not? And again, I’m not saying we, that, you outsource anything, but you might learn. And by the way, like, I, this process has been, it’s been humbling because there have been models and frameworks that I thought were important that now when I look back at how I rated things over a 10-year period of time, I could look back and say that maybe my framework is wrong, maybe, or maybe there is a better framework that we could garner more insight on.

Annie: One of the things I think, and this applies to decision-making outside of venture as well, and that’s one of the things I love about this problem is that so much of life is like this, right? If you go to college, are you going to know whether that was a good decision or not? Based on the outcome, I don’t know. You only get one run at it and maybe you’re 40 when you figure out whether you were successful in the way that you were hoping because of college, right? Like, so these are generally the types of problems that we’re dealing with a lot. And I think that, you know, one thing that I love about this process is just the amount of sunlight that’s being, you know, shined on the decision process, which not only allows you to be better decision makers in the moment, but allows you to close those feedback loops.

Cause, you know, as I think about that idea of like, I kind of go with my gut and this idea of sort of like the VC who just sort of knows it when they see it. This is the thing that always comes to mind. If you were investing in a hedge fund that did high-frequency trading, like options trading, and you said to them, so what’s the process for your options traders? Like how do they decide what options to trade? And their answer was, they just know a good trade when they see one. There isn’t a single person who would invest money in them. And the reason is that the frequency with which you get the answer, it’s such a tight and fast feedback loop that everybody knows that that’s absurd. Because of course, there’s lots and lots of data and it’s coming very, very quickly that you can crunch that should be able to help you figure out whether someone’s a good decision maker or not, or whether they have good process or not.

But yet in venture, when you give people the leeway, and that’s how I think about it, you’re giving them leeway that they can kind of hang themselves with, right? It’s actually a problem that now when you create these sort of what looks like, like a 10-year feedback loop where there’s so much uncertainty. Now, all of a sudden people are more likely sometimes to invest in someone who just says, “I’m a special VC. I’m someone who can see it.” And I think that that’s so interesting because, you know, at First Round, it’s this idea that these are isomorphs of each other. They’re the same problem. And it’s up to you to figure out how to bring process and discipline and de-biasing and reducing noise and creating good feedback loops in an environment that is, of course, more difficult to do that in. But if you’re not hungry to try to make that happen, then you’re going to miss out. You’re going to leave money on the table.

Josh: Totally. I think making the implicit explicit, when you say this is a good market or a bad market, or I love the problem this founder is solving or not, or this is a tar-pit industry or a great, like, you are making generalized statements, but the ability to actually unpack that is helpful. What are the characteristics of a good market? Why do you think it’s a good market? And then creating the data artifact over time is super valuable. And then I think the final thing is like trying to capture it with some level of exactitude.

I remember the exercise, Annie, that you did with us after attending a few of our partner meetings. You heard all of these phrases we use. “Oh, I’m super confident, I think it’s highly likely, I’m more, it’s more confident than not, most definitely, I’m nearly certain.” You know, all of these phrases to softly or strongly express confidence. And, you know, you asked us all, you captured, I think, a list of 20 to 30 of these, and you asked each partner to write down probabilistically when they like, what do they mean by each?

So when I say more likely than not, what do I mean? And I mean 51 percent because that’s more likely than not. But maybe another partner might mean 75 percent chance of it happening. And so what we saw was just massive dispersion. If you looked at the highest and the lowest for each of those 20 terms, I think on average, there was over 20 points of like dispersion that, you know, that highly confident for one person meant 66%, another one meant 95%.

Annie Worse yet, there were six of you, you didn’t agree on what always meant. That’s the one that’s always terrifying. What do you mean it’s not 100%? Well.

Josh: So, you know, just like those type of exercises to sort of force us in a conversation to try to make that be implicit that, Oh, I know what always means, or I know what highly confident means. Maybe if we just could express it in a numeric way you avoid the possible misinterpretation or misalignment. So, I just think trying to take the implicit, make it explicit, try to have a level of exactitude. Now, again, we’re wrong more than we’re right as VCs. We’re not saying that we’re trying to outsource judgment. We are in the judgment business. We’re just trying to learn and get better at how we collectively access that judgment and execute on it.

Annie: Yeah, I think, the idea is, you know, the two big ideas here, I think is anytime that you can make something that’s implicit, explicit, you’re better off both from your own perspective, right? Because you’re actually sort of defining what you mean by something, but also because then other people actually do know what you mean. And then they can tell you, they can give their opinion on that because the best de-biasing tool that any human being has on the planet is other people’s opinions. But they need to give that opinion before they know yours, which you also pointed out. And I think that that’s sort of big idea number one.

I think big idea number two comes from what you just said that of course what you’re doing is qualitative in nature, largely in the sense that you’re making subjective judgments about the quality of a deal when you see it. You’re not measuring revenue, you know, so on and so forth. Right? But even if you were measuring revenue, you would still be making a subjective judgment about what you think that meant.

I think that where people miss, where sort of the disconnect is, is just because something is a qualitative judgment, just because it’s a subjective judgment doesn’t mean that you can’t quantify it. So if I say something, if I say, I think something is highly likely, that’s a subjective judgment, right? That’s what I think. But if I say okay, instead of highly likely, if I say, I think it’s six 63% to happen, all I’ve done now is quantified my qualitative judgment. I’ve quantified my subjective judgment, and then now I can tell, like sometimes when I say highly likely, maybe, I mean 63%. Other times maybe, I mean 56%, but it’s a pretty big bucket, and there’s a big difference between 63 and 56. So this allows us to then see, Ooh, Josh is 68% and Annie’s 56%. Let’s have a conversation about why we disagree. Whereas if we were just saying, Oh, I think it’s highly likely we might fool ourselves into thinking that we have agreement when we don’t.

And then when you actually create that sort of, when you realize that you can quantify a qualitative judgment, not only does that drive better conversation and drive better decisions, but that’s what actually allows us to close the feedback loop so well. And I think that that’s a theme that I’ve heard running through what you’re talking about is this idea of not just implicit and explicit, but let’s actually be precise about what we’re saying, which means that you have to quantify those judgments. And that’s okay because you’re, it’s not algorithmic. It’s still your judgment. How could it be anything else? It’s a founder with a what if, right?

Josh: Exactly.

Annie: Imagine a society where the Alliance for Decision Education actually succeeds in its mission, which is to bring Decision Education, the kind of thing that is, has been so incredibly integrated into First Round for half a decade. Imagine that we’re able to bring this into K through 12.

Josh: So that’s your imagine if.

Annie: That’s my imagine if, right. Because we are definitely in the moonshot business over here. I like that. Now I’m going to use that. So that’s our imagine if. That’s our, what if, how do you think society would look different in two decades if we were successful in that mission?

Josh: Wow. I mean, I think it could countervail a lot of the challenges that we face right now in terms of assessing, understanding what influences you in making a decision, both assessing the quality of the information you’re basing a decision on, whether, you know, today, you know, you have a lot of people who are making, you know, personal, financial, medical, or other decisions off of TikTok and Instagram for, and look, there’s a lot of great content on TikTok and Instagram, and there’s a lot of bad content. So I think that, like, the ability to assess the veracity of information, understand how to weigh that information, understand that, you know, what is a probabilistic decision-making process to some degree and how uncertainty plays into that. Look, I think it will be a healthier democracy. People will make better-quality medical decisions, better-quality financial decisions, perhaps better personal decisions as to who their soulmate truly is. You know, I think that, it’s crazy to me that we make kids memorize the periodic table of elements, but we don’t help them understand how to truly evaluate, and not just critical thinking, but just critical decision-making and what are the ingredients of a good decision or a bad decision? What are sources of bias in a decision that they’re making? And so I think what the Alliance is doing, and it’s one of the reasons why First Round is excited to sponsor and underwrite the work in kind, is because what you’re doing is critically important, not just to society, but for a democracy to function.

Annie: Well, first of all, I’m just going to bring you around whenever anybody asks me so that you can just give that answer, because that was so much better than answers that I give. And also, thank you so much for sponsoring us. We’re obviously incredibly appreciative. All right. If people would like to learn more about your work or follow you on social media, where should they start?

Josh: So our website at firstround.com has a ton of information. We publish something called the First Round Review, which is our modestly aspirational goal of trying to create the Harvard Business Review for startups. We’ve been creating content now for over 10 years. And some of the best practices from some of the greatest startup operators are codified there.

I’m @joshk on Twitter. And you could always email me at josh.koppelman@firstround.com.

Annie: Awesome. Well, thank you so much for coming on the podcast, being an amazing guest, being so thoughtful, being an incredible thought partner for me.

Josh: Annie, thank you. You’ve really had a profound impact ever since I fanboyed you and, after hearing you’re on The Knowledge Project podcast you’ve really had a profound impact both on me and on how First Round thinks about our craft.

Annie: Well, I will say sort of to fanboy back that I think one of the hardest things is, you know, partly because of resulting, but by the time we got together, First Round was incredibly successful. You yourself have been incredibly successful at multiple things that you had done. Not just First Round Capital, but Half.com and so on and so forth. So I think it’s amazing when someone that successful is able to come in and say, I think I can do better. And I think it’s incredibly rare. And so I would just like to say that I think that you are way, way out at the tails for the way that you think about how much you’re willing to sort of rip apart and reconstruct the things that you do given how successful you’ve been because I think that most people aren’t willing to to actually do that. So it has been incredibly joyful for me as well. And I’m going to make sure that I write down, I’m going to have them transcribe your description of how society would change. So I really appreciate that. Thank you so much for being such an incredible guest. Thank you so much for sponsoring the Alliance and I will see you on the other side.

Josh: Thanks, Annie.