How to Get What You Want in Hidden Markets

Q&A with Judd Kessler, Professor at the Wharton School, and author of Lucky by Design: The Hidden Economics You Need to Get More of What You Want.

Judd Kessler is the inaugural Howard Marks Endowed Professor of Business Economics and Public Policy at the University of Pennsylvania’s Wharton School. At Harvard, Kessler trained with Nobel laureate Alvin E. Roth, one of the founders of market design, the area in which he has been conducting research for the past fifteen years. He is the author of Lucky by Design: The Hidden Economics You Need to Get More of What You Want, which came out this fall.

Lucky by Design is about hidden markets. What are hidden markets? They are markets that are not obvious, where price isn’t doing the work, you are still having to figure out how to allocate resources. And they are everywhere. Once you start noticing them, you can start figuring out how to play better in them.

If that all seems opaque, you’ll get it by the end of this Q&A. But before you read on, I highly recommend you watch this SNL video, “Big Dumb Line.” Not only is it hilarious, it gets you right to the conceptual heart of hidden markets. Check it out and then dive into this Q&A, one of my favorite conversations!

What is a hidden market?

Annie: You’re in an elevator and you have to pitch me Lucky by Design. What’s the tl;dr?

Judd Kessler: The tl;dr is that you are constantly playing in what I call hidden markets, markets that are allocating scarce resources that you want. Hidden markets operate by a set of rules that might not be obvious to you. The point of the book is to help readers to see that these markets exist, to figure out their rules, and then to develop strategy to get what you want from these markets.

Visible markets are the ones we think of, where the price is going to be the thing that helps us decide us as a society who gets what, where you use your limited financial resources to decide what’s worth it to you and what’s not. But once you’ve decided you want to buy something, it’s simple. You just buy it.

Annie: Just to contrast, can you give an example of a hidden market and how it’s different from a visible market, like a supermarket or the stock market?

Judd Kessler: Hidden markets are those where either there is no price or price is not actually doing the work. Getting a reservation at a hot restaurant is going to be particularly hard, whether you can afford to eat there or not. Getting a parking spot at Costco is not allocated by price.

Then, there are other hidden markets that we might even care more about, like dating markets or college admissions markets or labor markets. In a labor market, you have to apply for a job, you have to be hired. Price is there: you get hired and get paid a salary. But firms don’t just lower the salary until only one person wants the job. They have some matching process that decides who gets what.

Then, a lot of markets don’t have prices at all. Like markets for public school admissions, where you are not going to charge parents more to go to certain public schools. Everyone is going to get access to a public-school slot and you’re going to need some set of rules to decide who gets what. Dating markets don’t have prices determining who matches with whom. These kinds of markets play by different sets of rules. Success in them is going to require understanding those rules and then figuring out what strategy to play.

Swift-onomics

Annie: I just want to make sure I understand. If we are talking about a Taylor Swift concert ticket, there is a ticket price with a face value of, say, $99, which feels like the same as the supermarket. But not everyone who can afford a ticket can get a ticket. A lot more people want a ticket than there are available at face value. There are two ways to get a ticket. One way is that ticket sales are going to open at a certain time. You can wait in a line to get a ticket when sales open. That creates a hidden market. How do you navigate waiting in line with your tent? There is also a secondary market a lot of people want those tickets. Some people have access to sets of tickets that they could then sell on a secondary market and then you have to navigate that. Is that right?

Judd Kessler: Yes, and the secondary market is something I talk about in the book, but it basically comes up in response to the fact that there is a hidden market in the first place. The fact that Taylor Swift is not setting the price, in econ speak, “high enough” that the market clears, so that only one person wants to buy each ticket. That means there will be an incentive for speculators to come in and buy as many tickets as they can and resell them to you in a secondary market at higher prices. That’s a problem in hidden markets will often create: the speculators that come in and make it harder for fans to get the tickets.

Annie: The supermarket is much more straightforward because they’re just setting the clearing price. That’s it. We need the right number of people to want to buy this thing at this price. But Taylor Swift has different motivations in setting her price than the supermarket because of what you just said. She’s not setting it at the clearing price even though she could.

Judd Kessler: Yes, she could.

Annie: If she did, people would be like, “Ooh, that Taylor Swift. Why are her tickets $1,500 a piece? Does she want only rich people to come to the concert?”

Judd Kessler: Correct. It could be that there are situations where we see price below the clearing price. It could be that the seller is trying to create scarcity so that there will be more demand later. Think about a restaurant that could raise its price. They might not want to. They might like the line around the block. They might like the stories that it’s impossible to get tickets. I think about a fad that might be over soon, Labubu dolls. That’s a case where they could produce more or raise the price. But it’s hard for people to get their hands on them, and that’s why I’ve heard of them as opposed to just ignored them as something that other people were buying.

The logic for having prices that are low and creating hidden markets is to get attention for your product or drive future demand. Taylor Swift probably isn’t trying to develop more interest in her or more or knowledge of her, but I think there is this fear that she’ll come off as being greedy. If she’s charging the market clearing price, it’s going to be really high because a lot of people want to see her perform. That’s true of lots of live events, where the sellers of the tickets don’t want to be excluding people automatically based on the ticket price. They want to be inclusive, and Taylor, I think, wants her fans to be able to see the show for $99, but the hidden market makes it hard for that to actually happen.

Labubu mania

Annie: You mentioned Labubus. For those unfamiliar, think about when the big rage was Cabbage Patch Kids or Tickle Me Elmo or Beanie Babies. When hidden markets develop around such products, it feels like they aren’t stable. I understand that there are different reasons why you might not be setting the price at the clearing price. Like when we’re talking about Taylor Swift tickets, you’re pointing that out it could be because she doesn’t want to be seen as greedy. With Labubus, maybe it’s for attention. They think that more people will buy them if they think that they can’t get them. They’re creating excess demand of what the product actually would have a demand for. In those markets, would you predict they’re just always going to collapse?

Judd Kessler: I think, with a fad product, eventually the excess demand goes away. At some point people move on to the next thing. The excess demand is a short-term phenomenon, but it doesn’t make the hidden market any less important for the folks who want to buy the thing in that moment. A bunch of my friends have gotten it or nobody can get it and I’m really cool if I can get it. That creates this excess demand, this desire to get the thing. A year from now, will everybody who wants a Labubu get one? Will they lack the cachet? Will people not care anymore, the way that at some point people stopped caring about Beanie Babies as much? I think that’s going to happen. But in the meantime, we have a bunch of different sets of market rules that are doing the allocations for these products because there are too many people that want them at the price they currently retail for.

The book goes through many different sets of market rules. What’s funny is that for the Labubus, you can see different sellers using different hidden market rules to do the allocations. you get first-come-first-served races, where they do drops and everybody has to click as fast as they can to buy. If you’re not one of the first people to click, you don’t get to buy one.

There are places where they do random lotteries. When I was in London for a conference, Harrods had a Labubus lottery that you could enter to be one of the people who got a chance to buy one.

There are secondary markets that pop up where the people are buying Labubus to then resell.

You’re getting first-come-first-served lines, where people are waiting for a store to open to be one of the first to get in.

You can see the sellers who are facing this excess demand figuring out different sets of rules to actually resolve the scarcity.

The 3 E’s of hidden markets: equity, efficiency, ease

Annie: This brings me to this thing that you talk about in the book, the three E’s. Can you tell us what the three E’s are and how they apply to hidden markets?

Judd Kessler: The three E’s are equity, efficiency, and ease. The idea is that a good market, whether it’s visible or hidden, will achieve the three E’s. They will achieve equity, meaning that we’re being fair to market participants. People who we want to have an equal chance to get the thing do in fact have an equal chance to get it. We also like the markets being efficient. When I say efficient here, I mean giving the scarce resources to the people who want the thing the most. Then, we want it to be easy for market participants. We don’t care only about whether somebody gets a resource or not. We care about how hard it is for them to get it. If I have to wait in line for two days to get something, that’s an ordeal that I had to go through, and that’s socially wasteful.

One of the things that happens when we are thinking about visible markets that use price to allocate is that sometimes we are unhappy with those markets’ ability to achieve the three E’s. Usually, that is driven by concerns about the rich, wealthy, or high-income folks getting to buy everything. We just don’t think it’s fair. We don’t think it’s equitable that this group of people are able to get so many of the scarce resources.

Annie: How do these hidden markets do at achieving equity, efficiency, and ease?

Judd Kessler: Let me start with an example based on trying to get tickets to a live event. Before the internet era, that would involve something like standing in a potentially long line, the way we used to do it before the internet era.

Annie: Ah, a “big dumb line?”

Judd Kessler: The SNL video, exactly. The folks who were willing to stand in a physical line for a long time to get tickets, maybe they did want to go more than somebody who’s not willing to wait in line. Maybe that is a good measure of desire to see the show and that means that it performs well on efficiency. But it might not be equitable. Maybe there are people who have different amounts of spare time. If I have to work and can’t take off work to wait in line, I’ll be out of luck. If I don’t feel safe being on the street overnight, I’m not going to be able to spend all night on the street to buy tickets and somebody else might. That might not be equitable. Surely, it’s not easy because you have to wait in line.

Over time, that first-come-first-served line (from the pre-internet times), which is one of the sets of market rules that you can have in a hidden market, over time that changed from a line to a race. First, it was phone races. There was a period where you would call in when the tickets became available.

Annie: Yeah, the phone line would open up and it was just busy, busy, busy, busy, and you had to just keep dialing.

Judd Kessler: That was a mix of a race and a little bit of a lottery. There was a random element in the sense that they’re hanging up the phone with the previous call and taking the next one that comes through. Phone lines would get clogged in entire regions, and you wouldn’t be able to make phone calls because too many people were calling in to try to buy tickets. They were jamming the phone lines.

That was the way it worked for a while. Then, the internet came along and now it’s races, of whoever can click the fastest when Ticketmaster or whomever releases the seats. There’s innovation on that. Taylor Swift actually used something that was more like a lottery, the verified fan program that let people register in advance and then were lotteried in, but then, because the website couldn’t handle it, you ended up having to wait around on Ticketmaster’s website for your turn to buy. These are the market’s attempts to solve the problems of getting things allocated to folks and they often are not succeeding on the three E’s.

Annie: It sounds like particularly they fail on “easy.”

Judd Kessler: Yes, they fail on easy. I mean, there are ones that are easy or easier, but then they fail often on efficiency. There are a lot of lottery mechanisms. You can design one to be very easy. I’ve been trying to get tickets for the Shakespeare in the Park. The Delacorte Theatre has reopened in New York City, and they give out free tickets to see Shakespeare. Entering the lottery is very easy. I can go onto an app on my phone and enter the lottery, but it’s not necessarily going to reward people who want to see the show the most. It’s just going to pick from the people who entered on their phone. They also have a physical line where people can stand for hours, and those folks are able to get tickets in a different way. That one is not easy, but it rewards people who are willing to devote the most time to it. That scores better on efficiency.

Waiting in the Big Dumb Line

Annie: As you just explained, waiting in a big dumb line has had this history and this evolution. But at the same time, it feels like a growing cultural phenomenon. Why is it everywhere, so much so that Saturday Night Live is making a video about it? I’d love your thoughts on this.

Judd Kessler: I think the phenomenon of standing in line that has accelerated recently might have to do with the fact that now standing in line is a point of pride or something I can post about on social media.

Annie: It seems like there’s also some contagion to it, social proof. If these people are waiting online, then the thing must be worth it. I saw them posting about it on Instagram.

Judd Kessler: I think it’s both. As we talked about earlier, one of the reasons I might like to have people standing in line in front of my store is that it creates social proof for other potential buyers. “Look those cookies must be delicious. Why would people wait otherwise?” That’s good for the places that folks are standing in line. And it’s also the case that, if it is less painful or beneficial for you, it’s the point of pride. You’ve got bragging rights: you can post on social media about what you did and what you got.

That was also, if I remember, in the Saturday Night Live video. The point of doing it was just to be there and to be able to tell people that you went through that. That means there’s going to be a set of people who are willing to wait in line, who don’t find it painful. They’re getting some benefit out of it. It’s going to be tough if you just want the store’s cookie. If you don’t value standing in line or find it distasteful, it’s probably not going to be the time for you to go out and get it because you’re going to be competing with folks who are happy to go through the ordeal.

Annie: Can we get into some of the optimal strategies for playing in these hidden markets, like these lines?

Judd Kessler: In the ones with lines, the trick is deciding if the thing that you want to wait for is worth it. Is it worth waiting and getting there very early and spending a lot of time in the line? Once you’ve decided that, if you’ve decided yes, then there’s a question of when do you wait in line? The economic concept that I talk about there is opportunity cost. What are you giving up for standing in line? If you want to stand in line, what you’re giving up is ought not be that great.

You don’t want to stand in line when you’re going to get fired because of it. But if it’s standing in line or having a leisurely breakfast, maybe you don’t care that much. Maybe you grab something for breakfast that you can eat while standing in line. Then, it’s also about making the experience in line less painful.

One question as to the recent resurgence of lines as hidden markets is why people more comfortable standing in line now. Maybe the iPhone has made standing in line less painful. I can stand in the line and do most of what I would do at home. Instead of lying in bed and scrolling on my phone, I’m standing in line and scrolling on my phone.

3 kids, 1 treat – That’s a hidden market

Annie: I want to go deeper because I think that people don’t realize that hidden markets are everywhere. So far, we’ve talked mostly about things that you buy. But every single day, there are all sorts of resource allocation problems where people don’t realize that these are hidden markets.

I love how you ground this in talking about what happens when you’ve got three kids but only one treat. I don’t think that people think about that in the same way as a market, much less a hidden one. But as a parent, you’re dealing with that stuff all the time. How are you doing that? Can you talk about that experience that you have of thinking about how you allocate these things – scarce resources – to your children? And maybe give some other examples of everyday things so that people can really understand it.

Judd Kessler: This is great. There are lots of hidden markets that you control in ways that you might not be aware. I talk in the book about when all three of my kids want the same dessert. In the book, it’s Trader Joe’s “Hold the Cone” mini-ice cream cones. You have three kids who want this thing, and it’s not something that they can share.

So, what are the ways that you could resolve that scarcity? When you start thinking through, as a parent, what allocation rules you could use in this hidden market, you realize that you can’t achieve all three E’s at once.

You could do first-come-first-served, but that might lead to shoving matches in front of the freezer or to a race to finish dinner and say, “I’m ready for dessert.”

You could use a lottery, which is easy for them. It’s equitable in the sense that they all get an equal shot at it. But if the allocation rule doesn’t have a memory, it could end up picking the same winner multiple times. That’s harder on the market participants; my kids really don’t like that.

The way that we resolve this is something that takes into account the fact that we make this allocation decision on a regular basis. There are many nights where the kids are fighting over what to have for dessert. We do a version of taking turns where, basically, if you got the dessert tonight, then you won’t get it the next two times. That gives every kid something. It either gives them their first-choice dessert, or it gives them the knowledge that they’re one day closer to getting their first-choice dessert.

Allocating the scarce resource of your time and attention

Annie: I want to ask you about another common, probably overlooked, hidden market. Can you explain, from your chapter at the end of the book, about the allocation of the scarce resource of your time and attention?

Judd Kessler: A lot of people want your time and attention. Like at work, people are going to be emailing you, they’re going to be asking you to work on projects, they’re going to be asking you to take meetings and you have to decide how to allocate the scarce resource and you have to pick the rules that you’re going to use to decide who gets your time and attention.

Thinking about the three E’s is helpful in getting you to realize whether you’re using your time and attention efficiently. Are you giving it to the people who actually value it the most or are you giving it to the people who are the most annoying, who are emailing you multiple times, continually pushing for your time? Is that how you want to be allocating it? I realized in writing the book and thinking about the three E’s that I was using recurring meetings a lot to decide what I would be doing on any given week. Reflecting on that made me realize that that is not an efficient way of using my time.

I was using what is called first-in-time, first-in-right allocation rules. This is how historically water from the Colorado River was allocated to states. The rule is whoever was the first to tap the river gets full access to their water rights. If there’s a drought, California, which was the first to take water from the Colorado River, would get all their allocation and states that tapped the river later, like Arizona, would have to cut back. For a long time, those were the rules, but it doesn’t make any sense. In 1901, California starts diverting water. Phoenix, Arizona was barely a city in 1901. Now it has over a million and a half people. The fact that we were, for a long time, setting the rights to the scarce resource based on some race that was run 120 years ago, that doesn’t make sense.

But that’s what I’m doing whenever I devote my time to a recurring meeting and blocking it off on my calendar this week because I put it there six months or a year ago. That can’t be efficient and it’s probably not equitable because folks who want my time who came along later have second-tier access. Thinking more about the three E’s in these hidden markets I control turned out to help me be better about my time and attention.

The college admissions market

Annie: You earlier mentioned, as an example of a hidden market, college admissions. But we haven’t talked about that market yet, and it’s something that so many people experience, but don’t really think about from a strategy standpoint. How should we be playing the college admissions market?

Judd Kessler: One thing that should be explicit but might not be is that schools, even very selective schools, even the top-tier schools, have very low admissions rates. They take very few of the applicants. They basically all care about their yield, about the fraction of folks who are admitted, who then matriculate. It’s a point of pride. It’s also how you maintain the elite nature of the school. The school must be good if everybody who gets admitted decides they want to go. One of the ways that preference manifests is that schools will reward you with a higher chance of admission if you apply in a way that makes it more likely that you will commit to going.

Annie: Like early decision, early admission.

Judd Kessler: Exactly. Basically, when you apply for early decision, you are committing to go. If you admit me, I will come. It’s binding. This is something the schools like and they reward folks who apply for early decision. Knowing that as an applicant allows you to think more carefully about the strategy you want to play. One of the strategies that comes up a lot in these hidden markets is what I call “Settling for Silver,” a silver medal as opposed to a gold medal at, say, the Olympics. Sometimes, the optimal thing for this strategy is to act as if your first choice is something that actually is not your first choice. The reason you would do that, pretending your silver-medal choice is actually the thing you like best, is because it may be a way to get something where there’s less competition against you.

In the case of college admissions, here’s what this could look like. Instead of applying for early decision to your absolute favorite school, which might be a total reach school, you do it for your second or third choice. The idea is that you’re more likely to get in those schools if you apply for early decision, they are likely to have a bit less competition, and they are less of a reach than your number one choice.

Annie: You’re saying the expected value would be higher. Your chances of getting into Harvard are very, very low. Even with the nudge from early decision, it is still going to be very low probability that you can get in. But at your second- or third-choice school, if you add in that extra nudge, you may actually get into a situation where you’re a pretty big favorite to get accepted. But if you were to apply to Harvard for early decision and you only have one bullet …

Judd Kessler: Exactly.

Annie: You take one shot at that, but then you might end up going to your 10th favorite school because you gave up that nudge for the Harvard application. You have to recognize it’s that balancing out the regret of, “but maybe I could have gotten into Harvard,” with, but what are you giving up? It’s back to opportunity costs. What are you giving up for that one crazy-reach shot in terms of where you’re going to end up?

Judd Kessler: I think our natural instinct, when we have an opportunity like this, is to say, “I’m just going to go for the thing I want the most, I’m going to apply to Harvard because that’s my first choice.” But it might not be the most strategic. It might not be the optimal decision to make. It might not be the right one because you are wasting your one early-decision application and going from a 1% chance of admission to, say, a 2% chance of admission. That’s not the best use of that early decision because, as you point out, if you applied for early decision to your second or third-choice school, you might have gotten in. But it you used your early decision on your first choice and failed, you’re less likely to get into your second or third choice because now you’ll be applying for regular decision. You may then be down to your safety school, your tenth or eleventh choice, that you can get into without early decision. In these kinds of markets, where you’re facing a lot of competition, the sensible thing might be to play it safe.

What the French Laundry can teach us about settling for silver.

Annie: Can you talk a little bit about what you did about trying to get reservations at the famous Napa Valley restaurant, The French Laundry, and then your realization later about the mistake that you had made?

Judd Kessler: When I said that the settling-for-silver strategy comes up a lot, one of the places where you could imagine playing it is when you’re in a race but you’re not racing just for one thing. Everyone’s not trying to click for the same object, but where you have an option of what to race for. The French Laundry is a very exclusive restaurant in the sense of there are only 60 seats, only 17 tables, and even though it’s very expensive to eat there, there’s excess demand. There are many more people who want to get a reservation at The French Laundry than they have available tables. They release all of the reservations for a given month on the first day of the preceding month. If you want a reservation sometime in July, you have to go on June 1st and click at 10:00 AM Pacific Time for the reservation that you want.

In the book I tell the story about trying to get a reservation for my wife’s 40th birthday and my instinct was to go for the thing that I thought she would want the most, which was a 7:30 PM reservation on the Saturday night that she was going to be in Napa Valley. That strategy led me to click for that time, probably the same time as many, many other people who also wanted a very desirable reservation time. I did not get it. When the page refreshed, I saw that 4 PM and 4:30 PM were still available for that same Saturday. But when I went to click on them, they were also gone.

The thing that I learned was I had stupidly played Go for Gold as my strategy. I went for the thing that I wanted the most. What was obvious from how it played out was that if I had instead settled for silver, I would have been able to get one of those earlier reservations. This dynamic plays out a lot that we often think about what we want most, and we don’t think enough about what other people in the market are going for.

If you are thinking more strategically, if you’re recognizing it’s a hidden market, recognizing that most people are going to be going for gold and not thinking an extra step ahead, you can game the system. You can win by playing a slightly cleverer strategy. The same thing is true in the college admissions. A similar thing I should say is true on the dating market, which I also talk about a bit in the book.

The dating market and the labor market

Annie: That was what I wanted to ask you about as the last question, the dating market. Actually, the dating market and the labor market because you mention in the book that they have a lot in common. What should your considerations be in those markets?

Judd Kessler: The thing with dating is not to settle but to focus on the things that you idiosyncratically value that other people might not value. There are things about potential partners that are going to particularly appeal to you. They have the same hobbies as you. They have a look that you find attractive that might not be the thing that everybody finds attractive. You might like a quirky sense of humor that this person has. In these markets, focusing on those things allows you to end up with matches that might be better and last longer because you’re not courting somebody who’s getting 12,000 other suitors because everybody agrees that they are the most attractive or have the highest social status. Focusing on traits that you value most that other people value less might be where you’re going to win in hidden markets.

Annie: I’m thinking now about the dating apps. You’re saying try to figure out the things that you want in a partner that are different, that would stand out from the crowd. You really want someone who plays Dungeons and Dragons or something, and not very many people like that. If you index on that, you’re going to have a better chance in that market. But I assume that the reverse is true, that if you’re trying to sell yourself in the dating market, you should also think about what those really unique things are about you and emphasize those.

Judd Kessler: Yeah. In the dating market, the thing that is limited there is your ability to interact with a lot of people. The dating apps have made it so that there are so many people that you might have conversations with on the app that you might meet in real life. The goal is to find somebody that’s going to be a good fit for you, both someone you find desirable, and who will also find you desirable. The trick, I think, is finding somebody that’s going to be a good match for you. What you’re speaking to is what you should reveal about yourself and when you should reveal it. If what you’re going for is a long-term match, you really want to make sure that your true self is coming out and coming out early.

Annie: And what is the common thread between dating and labor markets?

Judd Kessler: A lot of the themes are the same. When I was interviewing for jobs to be an assistant professor, the faculty would ask me something like, “What are you going to want to study over the next few years?” There was a temptation in that environment to tilt your answers to be more like what you think they want to hear.

Maybe I should talk about doing research that’s more similar to the research that they do, to make me a more appealing candidate in this labor market. That might succeed in getting me a job offer at a school, or at a firm. But if what I really care about is my long-term happiness, that means getting tenure and staying at the school or firm. It’s probably a bad strategy to pretend that I want to study something that I don’t actually want to study. It’s easy to fake it for 30 minutes, but it’s hard to fake it for a decade. I’m going to either have to reveal later that this is not what I want to study, and then maybe they won’t promote me or give me tenure, or I’m going to have to do this work that I’m actually not that interested in and maybe I won’t be as successful, not as motivated, or maybe I’ll be miserable for the decade where I’m trying to get tenure.

The idea in dating and labor markets is the importance of being authentic about who you are. In both markets, that will help you find better matches. It’s going to be true when you’re applying for jobs. It’s going to be true when you’re going on dates. You don’t want to have to be faking it for an extended period of time. It’s better to know early whether a person is not happy with you. Don’t pretend you want to have kids if you don’t, because that’s not going to be a successful match. Sussing that out quickly is going to be good for everybody.

Annie: That’s good advice for dating, for jobs, and probably for a lot of other markets. Thank you so much for your time.

Judd Kessler: I had a lot of fun. This was great from my perspective. Your questions got me thinking about stuff about stuff I hadn’t thought about before, certainly not in other podcasts or Q&As.

Annie: That’s nice of you to say. It’s been a pleasure.