Topic 1: Methods > Readings and Lectures > Introduction Video
- This session of the course focuses on case study methods.
- What they’re good for, maybe what they’re less good for.
Topic 1: Methods > Readings and Lectures > Science of Delivery Cases Video
- How do you read these case studies? Let me point out a few things that might help.
- The case studies do talk about political will, but in this course, we’re really focused on a different stage of decision making.
- There will also be in each case a vision, a policy, a decision about what the reform team wants to do, presumably in consultation with others.
- In all cases, it involves looking at how reform teams have implemented those interventions.
- So the cases are focused on implementation and on delivery.
- The case studies talk about political will, they’ll talk about the vision, they’ll talk about the intervention, but their focus is on implementation and delivery.
- SOUND] The beginning of every case highlights a development challenge.
- It may be, “We want to improve incomes across the whole country.” That’s a big development challenge.
- A case could be about streamlining or it’s a case about rewarding performance.
- That’s a focus on an intervention and that too can be a kind of development challenge.
- The beginning of a case, the very opening paragraphs will usually highlight a development challenge, and the case returns to that subject at the end.
- Did they meet the challenge, or fail to meet the challenge? Or, as in most instances, meet it only partially? The body of the case will focus on the steps taken to implement a particular vision, implement a particular set of interventions or a given intervention.
- Now, what do we mean by delivery challenges? To understand the case, you may have to have a little bit of knowledge about the development challenge, although I think usually there’s sufficient explanation in the case itself.
- The delivery challenge is what you need to pay particular attention to.
- These are the political economy challenges that the civil servants trying to carry out the actions, are going to encounter, are likely to encounter, or encounter quite unexpectedly.
- So part of what you want to do when you read a case, is to spot what those challenges are, or ahead of time, think, what challenges are they likely to encounter? So, a delivery challenge is a challenge that arises in the execution of a policy or an intervention.
- A development challenge is the big, overarching, impact that you want to have, the big overarching objective goal.
- Now, the ability to achieve implementation of the steps taken to address the development challenge is affected by context.
- Some of the delivery challenges will arise from characteristics of the context.
- It could be any number of things and each case will have a number of ways in which the context, the sociopolitical context, the environmental context, the weather context, as weather features in a number of our cases, may influence the execution, the implementation, and the actual delivery.
- So we’re going to ask you to pay attention to all of those things as you read through the case study.
- Now, when you read through a case or when you’re writing a case, we invite you to put yourselves in the shoes of the reform leader or reform team.
- We want you, when you begin a case, to think about the challenges that a public servant who’s tasked with carrying out a vision is going to encounter.
- What challenges will they encounter and what can you actually do about them? So, you might want to, at the very beginning, put yourself in this person’s shoes or in the shoes of the team members, and think all right, what are the likely challenges that I’m going to encounter? What do I think i might do before actually reading the case? And then as you read the case, see what they did, and whether you agree or whether you think you have a better idea.
- So you’re going to anticipate, what problems are likely to arise? Given the context, given the particular challenges of the subject creates.
- If your challenge is to reduce delay, there are certain things we might anticipate would be problems in trying to reduce delay.
- When you’re reading a case or writing a case, put yourself in the shoes of the reform leader.
- Usually, right up in the beginning of a case, there’s a statement about the opening, the moment that was created to introduce reform, to change the system.
- How did the change actually take place? And we often talk about cases when we’re writing them as the hows of development.
- In the getting down to work part of each case, we try to break this down.
- Now, if you’re a reformer yourself, and you’re trying to think, okay, could I copy what they did in this case? The sequence may be especially important.
- There are usually other things happening in the course of a case that might influence these results.
- I’m gonna encourage you to plunge into your first case study.
- Read, with respect to the, or an eye to the development challenge, the implementation steps taken, the results, the alternative plausible explanations for those results.
- The things that you might have done differently to solve the particular development challenges that appeared along the way, or rather, the particular delivery challenges that appeared along the way.
Topic 1: Methods > Readings and Lectures > Case Studies Methods Video
- In this course, we want to spend a little time talking about using case studies to build general knowledge.
- Social scientists have a conversation about the relationship between quantitative methods and qualitative methods, or case study methods.
- We don’t want you to get bogged down in this [discussion], because this course is mainly about researching and writing a case study.
- One is on identifying causal mechanisms using case studies.
- The second is using case studies to understand outliers in quantitative research.
- A third is to use case studies to trace a process of strategic interaction.
- Process tracing case studies are one of the few ways we have of understanding that phenomenon.
- Finally, I want to say a few words about using case studies to test theories.
- Cases are not always the best method for testing theories but there are some circumstances in which they’re the only way we can think about this.
- These are just some general ways in which case study methods and quantitative methods differ.
- In my perspective, as I think the perspective of everybody associated with this initiative, qualitative research, case study research, and quantitative research.
- Case studies are helpful for identifying causal relationships, causal mechanisms.
- Quantitative research may be better than case studies for some sorts of purposes.
- Let me just take a minute to introduce three important ways in which case studies can make a difference.
- Now, when we do a regression analysis on a lot of different cases, a large-N data set, when we use quantitive methods- we’re trying to assess a correlation first of all, an association between some variables that we think have some causal influence and an outcome.
- To dig in with case studies or interviews to try to figure out what this would be.
- Case studies are one method for trying to dig out what that causal storyline would be.
- Interview based case studies where we go out and we talk to people, might help with that.
- One use of case studies is to try to identify the real causal mechanisms behind an explanation.
- That’s the causal variable, so we look at a lot of cases where there is merit pay and some where there’s not merit pay.
- We’d expect most of the cases where there’s merit pay to land in this quadrant.
- The most of those cases would be down here [on chart] but of course in real life that’s not always true.
- What a case study does is allow us to go in and try to find out what else was driving the result, in addition to merit pay or no merit pay.
- A third use of case studies that I want to point to-because it’s very central to what we’re doing in the science of delivery- is to understand social phenomena that involve bargaining, strategic interaction, persuasion, negotiation, cutting deals, forming coalitions.
- So when we have strategic interaction we really want to dig in with case studies to try to show the causal relationships between steps.
- Therefore a case study is a good vehicle for understanding whether our theory about how the bargaining, the negotiating, takes place really reflects the reality.
- So we use case studies to capture the strategic structure of events or situations where the outcome is the product of bargaining, persuasion, negotiation, deal cutting.
- Process tracing case studies usually have a number of components that might not be present in other kinds of case studies.
- The formal or informal rules will also make a difference and those may shift in the course of a case.
- So we actually do want to pay attention to these things in a process tracing case study, including a science of delivery case study.
- We want to use these case studies to see what happens when these kinds of elements change.
- Now, very often people say, case studies aren’t good for theory testing and I think, in some measure that’s true.
- In those cases each instance there will be maybe one, or two, or three cases.
- It’s very hard to get large numbers of carefully matched cases in order to test at theory.
- So in these instances, very often a case study can give us a much more compelling account of causal effects than any kind of statistical analysis.
- So wherever the number of cases is small, we probably want to use case study methods.
- Now, we’ll be talking about a number of other ways in which case studies are useful.
- You’re about to hear from a number of social scientists and policymakers speak about case study methods.
- The case alerts us to how a small change in something might have altered the outcome.
- Case studies are a much better vehicle for doing that than any other method.
Topic 1: Methods > Readings and Lectures > Story of a Case Study Video
- One would expect that countries that are middle income or are approaching middle income status to have reasonably good schools and measured by attendance, measured by the presence of sophisticated technology in classrooms, measured by the level of qualifications of teachers.
- On standardized tests that are used to measure the performance of those schools, and how well kids actually learn inside those schools, they’re way below where they really should be.
- So there’s a wide acknowledgement in the Middle East that their schools are doing very well in the terms of logistical issues.
- So we were trying to explain both this broad phenomenon of why it was that students weren’t performing well, but also trying to recognize that actually even if the average school isn’t doing well, there are some schools that are actually performing amazingly well.
- So what we were trying to do was to be cognizant of the distribution, the array of performing schools from the really good to the mediocre to the awful.
- We wanted to have a much richer engagement with those schools that were performing really well.
- So we spent time, a week, living in a village and trying to understand in more detail why this one particular school worked- a public school that wasn’t in any particularly rich suburb or anything.
- What were the characteristics of both the internal dynamics of the school and the school’s relationship with the broader public administration that was enabling it to, generate students that were performing at the at the top level of their own country.
- The standard way in which one might think a highly effective school is working in a context of otherwise rather indifferent performance is that there is a heroic individual, a sort of a celebrity, or a superstar kind of quality to the people functioning there.
- Most of the time we think a good school is all a function of the good teachers or the good principal.
Topic 1: Methods > Readings and Lectures > Case Study Methods and Common Mistakes
- One of the big challenges with this work is figuring out how to exploit the comparative advantage of case study work.
- So it’s really important, I think, with case study work that we see this case as an example of something.
- It has to be a case of something and the “Of something” part needs, usually, other forms of data or just lots and lots of other cases to help us to situate that case in the most appropriate way.
- So in the case studies that we did in our work in the Middle East, for example.
- You just don’t have enough cases or enough instances or enough data points to actually say something meaningful about that particular place or that particular sector.
- If you want to get the inch wide mile deep then you’ve got to work differently, you know? Ethnographic kind of work or somewhere between/ There are trade-offs between depth and breadth.
- You find that case studies are able to do both of functions if they’re done well.
- They can situate themselves within this broader statistical array, but they’re also able to bring together an array of different actor’s and processes and organizational structures in ways that help really to interrogate that case and situate it with in that broader landscape.
- There are several qualities that I think a good case study write up conveys.
- The first of these is that it is clear what it is a case of! A case is a case of something, and The “Of something” part is often assumed or not taken as seriously as it should be.
- The outliers can be very useful in terms of providing insights, but one needs to know what one is looking at with respect to other cases like it or comparable to it in that same political space.
- Then the third characteristic I think, is that there, is that the write up of a case needs to be, balancing the trade-offs and they’re very real trade-offs between speaking to the details and the particular problems associated with a particular case.
- The characteristics that define that case which are great interest and, and probably most justified to those people that are actually then in a position.
- To respond to what the case study itself has found versus the much larger body of readers who may know nothing about the country context, the organizational details.
- They’re looking to these other cases for insights on what might or might not work.
- One of the key lessons or at least the key vindications, I guess, that came out of our work was we recognized very early on that having 16 people in the field, doing this work was gonna be wide open for having 16 different ways of doing this work.
- Being able to get good at being able to vernacularize questions because a lot of times we’re working in communities that were speaking.
- In this case Indonesian or the languages that need to be translated once sometimes twice in order to be able to get it phrased you know in the right kind of way.
- You work iteratively with the raw material to fashion it to the, the, context and the people and the control idiosyncracies that you’re working with and there’s no short-changing that process.
- Some of the common mistakes that case study writers or researchers make, of, are various different kinds.
- One is that they commit to the standard problem that’s always leveled against case studies is they select on the dependent variable.
- Being very clear, as I said earlier, about situating your case in a context and in a context of other data and the, the context of the array of different outcomes that comparable cases display, I think is really essential.
- If that, if that’s not clear then a case is just cute or it is very particular and idiosyncratic but it is not a case of anything.
- If people aren’t clear about what it is a case of, then that can diminish the usefulness of a case.
- I think your go, a case study becomes useful both to the particular people involved in the case that a, a broader audience when the analytical categories within which the discussion evolves are clear.
- Because that’s, it’s the categories that enable people to connect the individual case to, to an, a array of different problems that might be occurring somewhere else.
- Then the third problem is that there’s just this, which is an extension of those, is just balancing the trade-off between the details of the case and the, and the generalities.
- The temptation is always to having spent a lot of time in the field and coming to have quite a deep personal investment even in a particular case that you want to convey all the detail with it, that you’ve discovered.
- That is the really hard part and only doing the cases in low times going thru the intuitive process of riddling your cases down to the core essentials is really important.
- We did that with our work we probably went through ten drafts of our cases before we will have them at a level where you’re satisfied with for presenting in a, in a major report.
Topic 1: Methods > Readings and Lectures > Andrew Bennett Case Study Methods Video
- One of the things that case studies can try to help you get hold of is, is complexity.
- So doing case studies allows you at least to try to unpack that complexity after the fact when you couldn’t have ever anticipated all of it before the fact.
- What it also does is that it gives you the ability to draw inferences from individual cases.
- In addition to making inferences from individual cases sometimes explaining the individual case is important.
- Sometimes it’s important for policy, But it’s also important for theory development to understand, for example, deviant cases that didn’t fit theoretical expectations.
- Does that mean there was a left out variable or several? Or some interaction effect we hadn’t anticipated? Or some path dependent element or other forms of complexity? And so, looking at individual cases like a detective and working forward from theories and hypotheses and backward from evidence or clues, just like a detective works from suspecions and from clues, can give you a lot of leverage on case studies.
- When you actually look at his monetary history in the United States it’s very much like case study work.
- He does a lot of what I would call process tracing and he does a very credible, more than credible, historical job of looking at this case.
- It’s actually several cases, different periods of US monetary history.
- There are lots of people doing really interesting work and doing qualitative work there, at that intersection, who are using case study methods.
- It’s trying to understand and explain the outcome in a individual case.
- It’s what we call a within-case method of analysis, right? So you’re looking at an individual case.
- You may have chosen that case for different theory building reasons, right? It could be a deviant or outlier and you’re trying to understand why this case not fit the theories that we know about.
- Or it may be that it’s a case that you think is a most likely case for theory if this theory’s going be true anywhere, it should be true here and if it’s not, then that’s really interesting and surprising and we ask the question why not.
- So a “Most likely case” that doesn’t fit or a “Least likely” one where the theory does fit- that’s interesting too.
- I know in some of the World Bank language that they like to talk about theories of change, you know, so what you’re doing here is you are going within the case.
- If the theory of change is true, if it explains the outcome of this case, what else should I see? What would be true about this case? Let me go over and check or validate whether that’s true about the case or not.
- That’s the deductive side to it and there’s a inductive side to it where either if you have a deviant case and you’re not sure or you don’t have strong candidate theories about why the case turned out the way it did, You sort of work backwards from the outcome and just try starting with an event history to see what might possibly explain the outcome.
- What happens proximate to the outcome? What were the the actors thinking and saying to each other? Now, even if you know a case reasonably well or you worked on it as a policymaker or you if studied a little bit as a researcher, there are going be tons of details in the case about the way things happened.
- By the end of your study, if you’ve done it right, you’ll probably know more about how that process unfolded than any one individual, even the people involved in it.
- You can have ten candidate explanations in one case but you may have enough different types of evidence to invalidate all but one of those explanations-or you may not.
- Right there you’ve knocked out everything except for one, right? So it proceeds by different logic than say frequent to statistical analysis, right? [pause] Example-I have a statistically significant number of clues that fit my hypothesis, except the murderer was not in the state when it happened.
- Now, whether it’s powerful or not depends on whether you have the right kind of evidence.
- That is evidence that’s could only be true if this explanation is true, couldn’t be true if these other explanations explanations are true, right? I’ve written some about how this falls [UNKNOWN] logic, right? So [UNKNOWN] you just try to have a prior expectation of the likely alternative theories or explanations of a case.
- That’s what makes this within case analysis so powerful is this Bayesian logic.
- Then you can actually make inferences from single cases.
- Now whether they generalize or not that’s another question, right? And, and they may, or they may not, right? The example I use is let’s give two, two contrasting examples here, one is Charles Darwin, right? Charles Darwin goes to the Galapagos Islands.
- His father’s disappointed in his son, cuz what can you possibly learn from a few bird species on this island, right? And Darwin comes up with a theory that once he understands the mechanism, right? That theory should apply to all living things and that’s a huge generalization, right? And that generalization, the logic of that generalization doesn’t have to do with the number of cases or the number of bird species.
- This is one thing about process tracing: It gets you closer to the causal mechanisms in the case.
- Does it generalize? Well, [LAUGH] not unless there’s a lot of intermarriage in this particular voting community, right? So sometimes that understanding of the mechanism tells you should that be a mechanism that’s highly general or that only applies to this one kind of bizarre case? And I used to use the sister-in-law example as a hypothetical.
- LAUGH] So it actually happens, right? So odd cases happen once in a while but they don’t necessarily generalize, right? And so generalization from cases is contingent.
- To have a finding in a case, even if it’s very convincing as an explanation of that case, does not automatically generalize.
Topic 1: Methods > Readings and Lectures > Standards for Process Tracing Video
- One is “Did the person come up with a a really good list of alternative explanations?” Did they, did they cast the net widely for alternative explanations? Did they think of structural explanations, agent-based explanations, explanations that have to do with material power, functional efficiency and transactions costs, legitimacy, social relations and so on? And in fact, elsewhere I’ve developed a checklist of theories about social life that’s sort of a good test.
- I find with my students often they’ll leave out huge categories of social explanation that they haven’t thought of.
- I can say, “Here’s another explanation you haven’t thought of” 19 00:00:58,320 -> 00:01:00,890 But, but let’s take an example here.
- Now, use of child soldiers is such an abhorrent idea that it it’s hard to do anything but focus on a normative [values] explanation.
- LAUGH] Right? So if my stopping rule for my favorite explanation is, “I won’t stop until I find some evidence,” and my stopping rule for the alternative is “I’ll stop as soon as I find the first piece of contrary evidence,” then, we all know what’s going to happen, right? I’m going to end up confirming my favorite explanation.
- Ask “What would be true if my explanation is true?” And “what would be true if the alternative explanation is true?” And people often don’t do that fairly, right? Then the next step is they have to be pretty relentless in gathering evidence, in asking the question.
- “What else should be true if this explanation is true?” And then going and finding the evidence.
- Right? I tell my students that sometimes we political scientists have to be historians.
- Right? If there’s one person, one archive, one piece of interview that’s critical to our alternative explanations, as political scientists, we’re going to expend a lot of effort to go there and get that.
- Right? Even more than a historian might, we see that evidence as a critical piece of this whole tapestry of a historical event.
- For us, if we think in Bayesian terms, that that’s the critical piece of evidence, we’re going to go to great lengths to get it, right? So we have to be relentless in gathering the evidence, doing the interviews, all that.
- Bad case selection, for example- where there’s either some selection effect that’s going on that they haven’t recognized.
- So people tend to overlook these negative cases or cases where economic growth could have happened but it didn’t.
- One of the frequent problems I see is that people don’t recognize the possibility of endogeneity.
- That can also manifest itself as selection bias or selection effects, right? So for example, is it that arms races lead to war, or is that actors see a war coming and then they engage in an arms race? Or is it that sometimes it’s one directions and sometimes it is the other, right? And often people don’t recognize that potential for endogeneity.
- Right? They don’t even recognize it as a possibility.
- Right? So I always tell my students, look, whatever your argument is, turn it around.
- See if there is any plausible way that you can argue it exactly the opposite of what you were doing, right.
- There have been some studies in the newspaper that say the more frequently people have sex, the longer they live, right? [LAUGH] Well okay, is it that sex is like exercise? Then that’s making their heart stronger? Or is it that people who are healthier and are going to live longer anyway are having sex more often? And I don’t know.
- This kind of process which in this case would be spuriousness, right? I think it’s very pervasive and a lot times students just don’t even see it, you know? And that, and that’s a real threat to the research design.
