Unit 4 > 4.0 Introduction to Unit 4 > 4.0.2 Unit 3 Debate Debrief
- Where do the best insights come from? Do they come from lab studies? Or do they come from field studies? We had several experts, again, on the panel, who talked about their opinions.
- He said, well-designed lab studies have clear advantages over field experiments.
- Owain started off by saying that he firmly believes that field studies add more insight than lab studies.
- Allison’s comments, again, favored word lab studies over the field, because the lab gives you a bigger sense of control.
- Professor Nina Mazar, another colleague of mine, also favored- well, actually favored the field instead of the lab, because she was talking more about studying impact of our interventions in the real world.
- Gita Johar, who is an associate dean at Columbia University, favored the lab.
- So we had three people who favored the lab, two people who favored the field.
- If we look at what reasons they gave, the people who favored the lab were all about control, control, and control.
- Whereas, people that favored the field talked about relevance.
- Why would you use a field study? When would you use a lab study? And so as you can tell, I’ve been thinking about this problem for a long time.
- Phase four, we want to go out and test that nudge in the field.
- So if you’re looking at a phase of discovery, then field settings might actually be very good.
- Lab settings help to confirm what you find in the field.
- If you can do that both in the field and the lab, that is actually a stronger demonstration of your effect.
- When you’re testing theory, things become a little bit trickier.
- You could never get that in the field, because in the field, as we’ve seen, many things co-vary at the same time.
- So it’s impossible to uniquely isolate one factor at the time in the field.
- So if you’re looking at theory testing, the lab is a much safer place.
- You can apply very strong tests of theories in the lab that you can never do in the field.
- Designing nudges is, again, a stage that is particularly well-suited for the lab.
- It would simply be very expensive to run them in the field.
- So designing nudges are, again, best done in a lab.
- Once you’re down to one or two little interventions that you want to test, that’s when you want to go to the field.
- When we think about what all of our panelists were saying, both Allison and Itamar and also Gita Johar were essentially making the point that labs are good for testing theory.
- If you go back to their commentary, they were really talking about testing theory.
- You can actually test all of that very well in the field, because now, we’re in the efficacy side of the process.
- The key question is, what is your goal? Is your goal to test theory? Is your goal to document a new phenomenon? Or is it to test the efficacy of a nudge.
- Depending on your goal, your answer is either the lab or the field.
- They’re papers that show a theory that’s developed in the lab, but then they show the implication of the theory in the real world.
- There are others that start off with the field study.
- They document the phenomena, bring it to the lab, and then start testing out and teasing apart the theory.
- In sum, I don’t think we can ever live in the world where you only do field experiments or you only do lab.
Unit 4 > 4.1 Advanced Topics in Experimentation > 4.1.1 What is the Value of Advertising Content?
- Unlike some the experiments we did earlier, some field trials, which are also called Randomized Controlled Trials, or RCTs, use a really large number of factors, each with multiple levels.
- The goal of this paper is to test the effectiveness of advertising content on the take-up of a loan offer in a field experiment and to estimate demand sensitivity to advertising content, compare it with price elasticity and the demand sensitivity to deadlines.
- Now, in essence, what these authors we’re trying to do is they were trying to send out solicitations for a loan product in which they manipulated both the features of the loan as well as advertising content.
- The advertisement varied on ten factors, both advertising factors and loan features.
- A second factor was the number of example loans that was shown in the solicitation.
- Suggested loan uses were either not specified or were specified.
- The authors use a probit type of regression to look at the effect of each of these factors on the take-up, or the likelihood that the loan was actually accepted by the recipient.
- It was negative because as the interest rate went up, the likelihood of accepting the loan went down.
- This is perfectly consistent with economic theory, with the rational decision-making that as you pay more for the loan, you are less likely to want it.
- Simply putting a female photo increased the likelihood that the loan was taken up.
- Providing only one example of a loan increased the likelihood that the loan was taken up.
- Finally, mentioning no specific uses of the loan also increased the likelihood that the loan was taken up.
- So while the first of these factors is what one would call a rational reason for influencing the take-up rate, the other three factors were simply factors that were part of advertising content that also had significant effects on the take-up rates.
- So what was the conclusion? Across the eight advertising features that were tested, three variations that we just saw here have a substantial impact on applications for the loan offer.
- Including a photo of an attractive woman, showing one loan example, and not mentioning the specific use of the loan increased the probability of applying for that loan before the deadline.
Unit 4 > 4.1 Advanced Topics in Experimentation > 4.1.3 An Experimental Organization
- What does it really take for your organization to become an experimental organization? What are the skills needed, and what sort of capabilities do you need to develop? Before we answer that question, let’s spend a minute trying to ask a more simple question.
- So what does it need for you to become an organization that is experimental in nature? The first skill that you need is to have the ability to generate small sub-samples of consumers, of citizens, or of agents that you can experiment on.
- It is important to be able to generate a control condition as well as many treatment conditions that you need to test a particular idea.
- What that means is that you need to have the ability to deliver a different intervention to each of those segments separately and effectively.
- It is important to keep in mind that you need to have the ability to generate these samples and to treat those samples differently.
- The second key indicator is you need to have the ability to deliver simple interventions, low cost interventions, and quick interventions.
- You need to have a process in place that decides what exactly the intervention is going to be, and those interventions need to be effective and efficient in order for you to be able to experiment effectively.
- Third, you need to have a system for rapid feedback and rapid analysis of the data.
- The moment you generate an experiment and you treat different consumer bases to different products or advertising messages, you need to be able to quickly understand what the effect of that cause was on whatever it is that you’re trying to measure.
- The moment you have the ability to collect the quick feedback, you want to also make sure that you can analyze the data effectively and make sure that the analysis is then delivered back to people who might need to design a follow-up experiment quickly and efficiently.
- The first- and we’ve briefly touched on that- is the need for processes.
- The second set of skills that are needed is, essentially, a culture of empiricism.
- You need to embrace the fact that data tells you what the truth is as opposed to theory.
- So in order to have a comprehensive theory of decision making, we would need to have a theory that embraces almost everything we see in the world around us.
Unit 4 > 4.2 Understanding Intuition and Judgment > 4.2.1 Decision Analysis and Intuition
- We said that the utility of a given option or a given product is a function of the combined utility that is delivered by each of its attributes.
- In particular, if you can decompose a product into multiple attributes and you can assess the importance of each of those attributes to your decision, then the utility is the sum of w, which is the weighting that you place on a given attribute, multiplied by the utility of that attribute, added across all of the attributes.
- A simpler version of that same model is the second equation that you see in this chart, where you simply multiply the weighting of a given attribute by the value of that attribute, wi multiplied by xi, and add that up across all of the attributes.
- How does a consumer, for example, choose between two products, product a and product b? How does a policymaker decide which of two policy options he or she should choose? Here’s what they need to do.
- They need to decompose that policy or that product into as many as relevant attributes as they think are important.
- Then they need to come up with an importance weighting for each of those attributes.
- Finally, they need to evaluate each attribute on a scale, multiply the weight by the value of the attribute, add up the total to give them the utility or value of every given option, and then pick the option that actually provides the highest value.
- What is intuition? And I’m going to read for you a definition of intuition that Robin Hogarth first proposed in his book.
- Sometimes we go to a store, and we simply know what we’re going to by just looking at products.
- So what is intuition, and where does it come from? The first thing I want to keep in mind is that intuition is not necessarily emotion.
- When you have a good intuition, it doesn’t mean that you’re just acting on an emotional response.
- It’s not that they have a mathematical equation in their head. But every time they see a certain cluster of symptoms happening at the same time, they know it’s got to be disease a or disease b or disease c. That is intuition.
- That is a learned set of intuition, because what the expert has done is they have accumulated their knowledge over a period of time.
- So when I go to a store and pick up a product, like a quilt or a product of art, I don’t necessarily spend the time decomposing that product into its attributes, because I know what I like when I see it.
- You can think of intuition as being something that arises from expertise, it arises from a vast experience of dealing with similar tasks in the past.
- What does that mean? It means they have a good intuition about how much they know.
- What you do is you look at your past experiences, your past decisions, and use those decisions to formulate a model of relevant attributes, and the importance of those attributes.
- On the other hand, intuitive processes also use history, but they use history to develop this engine that we call intuition.
- Think of intuition as a regression equation sitting in your head, which is quick and which lets you make decisions in the snap of a finger.
Unit 4 > 4.2 Understanding Intuition and Judgment > 4.2.3 Models of Intuitive Judgments
- DILIP SOMAN: In many situations in life, we need to make predictions or judgments about things that are going to happen in the future or things that we can never observe.
- You need to make a judgment about the quality of the performance that this person is going to give you over the next three years on the basis of evidence that you can see in front of you now.
- Unless that performance actually happens, all you are left with is simply your judgment.
- In all these cases, there is some truth, But all you can observe are pieces of evidence that you believe will predict that truth.
- This is what is called the lens model, and it comes to us from a part of psychology known as social judgment theory.
- The social judgment theory says something very simple.
- It actually says that when we try and make judgments about the truth, and again, the truth is something that we don’t observe or that will take time to happen, we rely on what are called cues, which are pieces of information in the environment that we believe will predict the truth.
- Based on those cues, we somehow combine them to form a judgment.
- Once you have a set of cues, all you then need to do is to look at these cues and form a judgment about the truth.
- Let’s think about an example of a situation where you have a recruiter who’s going to try and make judgments about the performance of a given employee over the next three years.
- If you had true data on the performance and you measured the correlation between the judgment that the recruiter made today and the actual performance, that correlation is a measure of what we call the performance of the judge.
- For a moment, let’s focus on a situation where we don’t yet know the truth.
- For every application, you have data on these three cues, and you form some judgment.
- Let’s imagine that you could run a regression where you looked at your judgment, the recruiter’s judgment, as a function of the three cues across these 200 data samples.
- It’s an experiment where you are essentially replicating yourself multiple situations where the cues are different, and you’re forming a judgment.
- It might actually say that judgment is some a, a constant, plus some b1 multiplied by the IQ score plus b2 multiplied by experience, plus b3 multiplied by education.
- If you did a correlation between the model’s prediction of what you would have predicted and the truth, assuming, of course, now you had the truth, it turns out that that correlation is better than, in many cases, the correlation between the judge’s prediction and the truth.
- Finally, they compared the r squared, the correlation of the model’s prediction, with the truth as well as the manager’s or the expert’s prediction with the truth.
- In one study where experts were asked to predict the GPA, the Grade Point Average, of a number of students in Illinois, the correlation between the experts’ prediction and the truth was 0.33.
- The correlation between the model of the expert and the truth was 0.5.
- This process of taking a regression model to capture your own intuition is called judgment bootstrapping.
- That’s the basis of judgment bootstrapping.
- Intuition can be decomposed by simply looking at situations where you have a decision or a judgment that you make repetitive times.
- The resulting regression equation will give you a model that explains how you make decisions and judgments.
Unit 4 > 4.2 Understanding Intuition and Judgment > 4.2.5 Educating Intuition
- Why would we actually look at our own judgments and use regression to model them? This at least three reasons why this can be beneficial.
- The first is it gives you insight into your own decision making policy and your own judgment policy.
- It’s almost like opening up your head and figuring out how you make choices and how you make decisions.
- Perhaps once you have insight into that process, you can then start making changes and corrections to your own judgment policy.
- Perhaps you thought you were weighing price to a great degree in your decisions, but the data tells you that you don’t.
- So simply looking a your own equation gives you A, a better insight, and B, avenues for improving the quality of your decision making.
- Second, you can actually use a model like this to track changes in your own decision making policy as a function of time.
- Third, you can use the model of other people that you consider as good judges as a benchmark into your own decision making.
- What you could do is you could try and convince that physician to run a judgment bootstrap of their own clinical diagnosis.
- You could then potentially compare your own judgment policy with the policy of this expert.
- Decision analysis is interesting because it not only lets you understand your own decision making process, but it also lets you benchmark and improve that process with time.
- As we look in week five and week six of this course, we actually look at some very specific ways in which you could use this insight in improving your own decision making.
Unit 4 > 4.3 Consumption Vocabulary > 4.3.1 Consumption Vocabulary
- DILIP SOMAN: It’s always fun to enjoy a good glass of wine.
- While I’m working on my wine, I have a simple question for you.
- What is common to wine, quilts, and classical music? Obviously, on the surface, it seems like these are three completely unrelated categories.
- Most people can tell you that they like a given wine versus the other one, that they like a given piece of music, that they love the pattern on a quilt.
- If you ask me why I like this glass of wine, I’m going to say because it tastes good.
- What I’m not going to be able to tell you is that my preference for this glass of wine is a function of the complexity, the bouquet, the aroma, the acidity, the tannins, all of that lovely stuff that you read about in Wine Spectator magazines.
- If I only knew what all of those things meant, and if I were able to identify each of those terms with a specific feature of this glass of wine, a few things could happen.
- Two, I might be able to learn more about my own preferences by trying wines that are constant on every single dimension except one.
- You can actually decompose the liking for a quilt along a number of different dimensions.
- These are actually real pieces of data from expert quilters.
- These are all the attributes of quilts that you and I would probably not think about, but the expert knows that they exist.
- Suppose you had five or 10 such attributes and you created a large number of quilts by looking at different combinations of these attributes.
- Let’s imagine you came up with 150 different patterns of quilts.
- Now what you have is the participant’s liking for each of these quilts, which you have from the expert is the evaluation of each of those quilts on those different attributes.
- What you’ve just done is you have allowed this individual to figure out why they like certain quilts and why they don’t.
- If you could actually start decomposing the preferences for fine wine or classical music or quilts or objects of art along these dimensions, we can now start achieving some of the ideas that we spoke about at the beginning of this module.
- First, people in her experiments were taught how to appreciate a quilt by being given the right vocabulary.
- Because they now had a hook on which to anchor the utility on, they were able to come up with much more fine-tuned preferences for those quilts.
- By providing people with the consumption vocabulary, what you can actually do is to take a process that is largely intuitive and convert that into an analytical process because you now give people to the right terminology, and therefore the right meter, on which they can assess their own utility for this lovely glass of wine.
Unit 4 > 4.4 Debate > 4.4.1 Debate 4
- So should behavioral economics have a grand unified theory? Or is the best we can do to have a culture of experimentation? I say that we can’t be very effective without a theory.
- A famous psychologist, Kurt Lewin, said there’s nothing so practical as a good theory.
- Without a theory, we don’t know what we should be experimenting about.
- So a theory basically helps give us focus and helps us interpret what’s coming back.
- I don’t think it’s the case for Behavioral Economics that there’s going to be one theory.
- So we’re going to have a set of smaller theories that cover certain phenomena, like prospect theory, like theories about hyperbolic discounting intertemporal choice, et cetera.
- If we don’t have a theory and we’re just experimenting, then we’re really not doing what we need to as academics.
- So the theory really helps guide us in what to experiment about.
- RORY SUTHERLAND: Can we ever have a kind of grand unifying theory that covers human behavior? Well, probably not.
- Most real-world human behavior doesn’t really have a right answer.
- Now, just because we will never have a perfect theory, that doesn’t mean we should stop trying.
- If you look at a much simpler field than human behavior, certainly much simpler than mass human behavior, which is meteorology, they accepted once they understood complexity theory, that perfect predictability weather is impossible beyond a certain point.
- I think it’s recorded in the very, very good book, actually, The Signal and the Noise by Nate Sliver, but if you look at meteorology, it’s now- and this is a very poor use of English- twice as good as it was 20 years ago.
- I think in the mid-’90s you would find that they could predict the path of an incoming hurricane with a kind of accuracy to within a width of a corridor of about 400 miles- now it’s down to 200 miles.
- I think a four-day forecast now is about as accurate as a two-day forecast was in 1994.
- I think that if we could only understand and predict human behavior 5-10% better than we do at present, I think the economic gains and the social gains could be spectacular, and an exponential.
- Why do I think it’s never possibly to be perfect? Well, even if human behavior is simple, the environment isn’t.
- So the idea that you can have this grand unifying theory seems to be pretty unlikely.
- It’s worth remembering that most progress doesn’t happen because someone comes up with a brilliant perfect theory which applies to everything.
- Because in academia, you really engaged a massive attempt to be right, to come up with some fantastic theory.
- DARREN DAHL: So the question on whether we can achieve a grand theory or a central framework to describe decision-making.
- There’s so many different ways to think about decision-making, and so many different contexts, situations, drivers, motivations that make up the pie, that I think it’s going to be difficult to provide one overarching framework.
- I do believe that providing some simple parsimonious grand theories is probably going to be difficult to do.
- It becomes very difficult to truly track what would be one simple theory or one simple answer.
- I think we’re going to continue to add bits and pieces as we move forward that further our understanding and give us a broader depth when it comes to decision-making.
- I don’t think that we’re ever going to be done.
- CHRISTOPHER HSEE: Well, about grand theory, I think this question, per se, maybe reflects some bias.
- Why do we need a grand theory? I think that pursuit for grand theory among scholars is somewhat like children pursue a fairy land, or people pursuing one god.
- So I think the fact that we have, empirically, a cumulative different principle, a cumulative different piece of knowledge.
- I think this is just illusions that there should be one grand, unifying theory.
- I think the pursuit of one grand theory may be a bias in and of itself.
- CHENBO ZHONG: What I feel most excited about, and I’m leaning towards the position, is actually we have a culture of experimentation where we can actually explore and investigate the variety of human behaviors.
- I think theory is important whenever we try to do hypothesis testing.
- The question here is whether we can have a grand or unified theory about human behavior.
- There are grand, unified theories perhaps about universe- maybe we’re going towards there.
- Looking at the history of economics, there certainly has a embryo of a unified theory.
- So I think a lot of the debate between this is not just happening within economics or between traditional economic theory and behavioral economics, but also happening in other fields of social science in general.
- Psychology has been criticized as being a field without a unified or a grand theory.
- I think looking at compare, so for example economics and psychology, I don’t think we can make a judgment about which approach is better or worse.
- I think both of them probably can contribute to our understanding of human psychology and behavior in different ways.
