Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > What is Data?
- So for me, I knew that if I just went to a gym and tried to walk into my local neighborhood gym, that I didn’t have what it took to motivate myself to get on the treadmill or to lift weights.
- CrossFit is a kind of exercise that involves gymnastics and weight lifting and a lot of natural movements, like push-ups and pull-ups and sit-ups.
- He stopped me, and he said, you know, I want to schedule every meeting so that we can talk about your progress.
- So from his perspective and observing my practice, I wasn’t making any progress.
- From my perspective, I was doing a lot and actually was making some big improvements.
- So as an improvement advisor, I wanted to bring data to the conversation.
- I wanted to show up and be able to look at what I was doing from a broader perspective, so we could really tell a whole story.
- So I had been tracking data on things like my attendance, things like my body weight, using the scale that was hooked up to the internet that would track my weight every day.
- I looked at things like my travel schedule and times that I was away from the gym and other details, like when I missed a class, but I made it up.
- I brought it in an annotated run chart, display of data over time, but annotated with all kinds of details about my travel and things that were happening, so that we can really see the sort of blended picture of what was happening.
- It helped us to have a much deeper conversation about where I was in relation to where I want to be and all of the factors that were associated with the process of me becoming physically fit.
- When we look at data through the eyes of improvement science, it becomes something that’s really empowering.
- When I bring in the data that show various aspects that influence the way that my health and my fitness occur, he gets to see a bigger picture of where I’m coming from and where I’m at.
- It gives us an ability for us to start to think about what are the things that we can do to change and potentially improve the results that I’m trying to accomplish.
- Data is also a blend of qualitative and quantitative.
- So things that we observe and can categorize and see versus things that are very specific data oriented pieces, like counts and measures that we described earlier.
- In the sessions to follow, we’ll be learning about collecting data for improvement and how to display it in a way that enables analysis that builds knowledge and also helps to tell a story.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Creating a Data Plan
- DAVID M. WILLIAMS: So once you’ve decided it’s important to collect data, which I hope you have after the last few sessions, you need to think about how you can do it in such a way that it’s part of your daily life and part of your work.
- How can this data be useful? How do we use this data in such a way that it helps and informs our improvement? So let me tell you about a restaurant chain in Austin, Texas.
- Also embedded in her process, she is collecting data as part of her daily activities and her current process of managing the flow of the line.
- So it’s really easy for her to just capture data on her clipboard and capture it in real time.
- So keeping the line moving and the wait predictable was important to making it reliable, especially when they had a first-time customer and wanted to take the extra time to ensure a positive first experience, knowing that the best indicator of a return customer is a happy customer.
- So when you’re thinking about data, ask yourself, how do I get that data, and then how can I get just enough to learn.
- You also need to be thoughtful and intentional about the plan you develop to collect the data.
- Actually really think about how you’re going to get that data and who’s going to do it and what process they’re going to follow.
- For any process, you’re collecting the data for a number of reasons, including to understand the variation or the behavior, to monitor the process over time, to see if it’s consistent or if it’s changing, to see the effect of changes as we test alterations to the process to see if they’re actually resulting in the improvement we’re trying to achieve.
- In quality improvement, there are different types of data or measures that we collect.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Finding a Family of Measures
- DAVID WILLIAMS: Before we jump in any project, we need to think about the key, vital few measures that will help you realize whether you’re making a difference.
- It’s important to capture several different types of measures.
- So why is that? Well, one thing that’s important to realize is that no one measure is going to give you all the information that you need.
- We need to a family of measures that are going to enable us to look at a problem or process from a number of different perspectives.
- So we’ve talked about different types of measures previously, like outcomes and process and balancing measures.
- These are all part of a family of measures that help us to be able to get a full picture of what we’re trying to improve.
- Process measures are actually the voice of the process itself.
- How is the system performing to achieve that outcome? And then finally, there’s something called balancing measures.
- Balancing measures may not be something that you’re as familiar with.
- They’re there the measure you need to look at in the system that gives you some different dimensions.
- These are measures that help us to avoid doing things that are side effects of our improvement.
- So from an outcome perspective, as you see at the top, we had an outcome measure that was to look at my weight and also to look at my level of fitness.
- I might want to measure the number of times that I’m able to complete a workout at a certain level that shows that I’m actually making progress.
- So I’m going to measure those and try to work on those process measures to improve my outcome measures.
- So I might track some balancing measures like days between getting hurt.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Graphing Data over Time
- DAVID WILLIAMS: So we talked about data and collecting data, but how do we now convert data into something that we can visually see and enables us to build knowledge and learn.
- We can start doing that even with our first data point for one of our measures and add a data point onto a line chart.
- Over time as we collect new bits of data we can add each one of those data points to the line chart.
- Once we get up to 12 data points we actually can add a median and the median enables us to be able to just apply four simple rules that help us to differentiate when things are actually part of the common variation within the system, and when there’s something special or something attributable and there’s a signal of change that we want to explore.
- We could make it in any spreadsheet program, or we actually just do it on a piece of paper, which sometimes is the best way to learn and to really be connected with your data.
- Along the bottom is the x-axis, and this shows the time series in which you’re tracking your data.
- They can see it and they can spend their time learning about the data, but not having to try to figure out what it means.
- One thing that people often worry about when we’re talking about data and talking about run charts, is that the qualitative aspects are going to be lost in the quantitative data that we’re looking at.
- This transforms from a median to a mean and then adds two other lines, an upper control limit and a lower control limit, that help us to apply some additional rules that are more sophisticated and enable us to identify when there are signals of change, or things that are special cause versus common cause variation in our process.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Constructing a Run Chart
- One of the things that we found that is an interesting obstacle for people to begin to use these tools and to collect data is that we actually don’t know how to build a run chart.
- Now you can build a run chart by hand, and I actually strongly encourage that, because it’s really useful to sit down and have that relationship with putting your own data in and knowing where the changes are occurring as you’re plotting your dots on a piece of paper.
- Then in column two, next to the day when you’ve collected data, you’ll enter vertically your data for each of those data points.
- So once we reach 12 data points, this is where we can add the median and it enables us to apply those run chart rules, that we talked about in a previous session, to help interpret the difference between special cause and common cause- identify signals of a nonrandom behavior in our data.
- So to build that median, what we need to do is select the cell underneath the word “Median” that you’ve put in as a header in column three and put in this simple formula that will highlight your data from column two.
- Now grab the corner of the C3 cell and drag it down until the median copies next to all of your data.
- Now select all three columns of data, including the headings, and in your menu bar find the place where it says Insert or Build A Line Chart and select that.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Improvement In Action: The Dimock Center
- Dr. Holly Oh is the chief medical officer here, and she’s going to tell us about how these actual tools are applied to a project that she worked on with her colleagues around pediatric asthma.
- HOLLY OH: From a big picture standpoint, asthma is important in the pediatric world because it really is one of the most prevalent chronic diseases of kids and adolescents, and the symptoms that can come with asthma- wheezing, asthma attacks, and so forth- are entirely preventable.
- What we decided to focus on was trying to implement a more reliable preventive care asthma pathway, and our particular aim was to try to reduce the number of unplanned acute care visits to our health center over a 10 month period of time.
- As we began the work and thought about our aim, we first began to think about, well, what are the major drivers? The areas that we came up with were, first of all, how can we better classify the severity of our kids with asthma? So to make it kind of short and sweet is, how do we who we’re going to be working with? Second was, as we know what the evidence based treatment guidelines are around asthma, how can we make sure the medication treatment portion of that was going well in our clinic? Third was focusing on education.
- So how can we actually ensure that our staff is most reliably and most effectively helping to educate kids and families about asthma care.
- How can we help activate kids and their families to take control of their asthma, take care of their asthma.
- We decided to focus on this asthma action plan to see if we could improve it to make it work better for our families and staff.
- So what we actually asked our nurses to do was, if everybody could take that existing template and just focus on reaching one or two families with asthma a day basically over about a two week period of time, we had all this feedback on the current asthma action plan- what was useful, what was not useful, what worked, what didn’t work.
- We went to patients and families just one a day with each of the nurses, and again, over about two or three weeks’ time, we basically gathered more feedback on the prototype and then took that feedback and fed it back into the next iteration.
- What you’ll see is over the course of the 10 months we started out where only about 44% of our patients were getting this asthma action plan reliably in the old system, but over time, with the multiple iterations of improving the form as an education tool as well as a self management tool for the patients, as well as understanding better how the asthma action plan fit into the workflow of the clinic and making sure that that workflow was more reliable and standardized as well.
- One main thing that we actually learned and realized was asthma is a seasonal disease.
- So what we recognised was, even though we might not have chosen the most appropriate outcome measure for our aim for a 10 month scoped period of time, because it didn’t allow for the seasonal variation, I think as we got together as an improvement team to talk about it we realized it was OK. It was still a good learning experience, and really in the end, we believed that the process measures that we were following for the work on the drivers, the severity classification, the prescribing of the right preventive medications as well as the asthma action plan work- those were the right process measures and drivers to be working on that would overall lead to improvement in asthma outcomes.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Improvement in Action: A Closer Look
- We just had a great experience where Holly Oh took us around The Dimock Center, and talked a lot about the work that they’ve done on applying improvement science to a very practical problem.
- What stage in the project was that? HOLLY OH: That was kind of our discovery and information gathering stage, that I mentioned before.
- That’s why you kind of have the 44% that you see and the wide variability.
- That whole middle section of the control chart reflects the time when we were doing the continual learning and the continual tweaking of the prototype to a second prototype, third one, taking in the things that we were learning as we were using it, the feedback that we would get from families about what sections were useful and not so useful, also feedback from the staff, as a teaching tool, what was useful, what was not useful, what was working, what was not working.
- Then we were kind of create the second iteration, the third iteration, and then continue deploying it, and continue to get continuous feedback, and feeding it back into another iteration of the form.
- Because then of a sudden, on the right hand side of the chart, that’s sort of the third phase, you really see the data become much more predictable and reliable, and sort of settle around a particular level of performance.
- As we were getting good at that, and as we were kind of in that phase, when we were doing iterations then changes of the drafts of the Asthma Action Plan, that’s when we also begin to branch out and really look at, well, where does this actual tool fit into the rest of the system, fit into the rest of the clinical workflows.
- So that part sort of towards the end of the control chart, where things really tighten up and become a lot more reliable, is when we began to look at- how do we actually, kind of, tighten up how we use this form- not just the form itself, but how we use the form with which families, with which children, in what way, so that we can actually make the context and the system that the Action Plan was using a much more reliable system as well.
- DAVE WILLIAMS: I’m just curious, kind of as a closing question- and I know this was the beginning of the project, the part that we’re talking about, and all the elements.
- How did this contribute to the staff’s interest, and the will around this project, and continuing to do the improvement work? HOLLY OH: Right.
- I think, just by starting out in small steps and small chunks, bringing in people who brought excitement to the work, getting some results, even just in those first few iterations, allowed people to get excited about it and brought more team members.
- Really, by the end of the project, it was not just excitement and getting reinvigorated about asthma care work.
- You know, I really appreciate you, Holly, for inviting us to come to The Dimock Center, and to hear about your story of using improvement science to actually make changes in your organization.
- I think it’s a really powerful, as people are trying to figure out how to take the theory and apply to practice.
- It’s really exciting to be here with Dr. Oh, and to learn more about the work at the Dimock Center.
Lesson 3 – Introduction to Measurement for Improvement > Lesson 3 Lectures > Your Turn
- DON GOLDMANN: You’ve been learning about run charts in this last lesson.
- For me, this is going to be a much longer-term project, and I’m going to be working on this for at least a year.
- I think we can go over the structure of the run chart and how I’ll be tracking what I’m doing.
- I’m going to pick just one of the elements of my personal improvement project, and that’s the time I spend on that elliptical machine each time I go to the gym.
- So the title of this run chart is “Time Spent on Elliptical Machine.
- You have the y-axis, and that’s going to be the number of minutes I spend on the machine.
- I’ve populated this run chart with a couple of data points because I’ve already started.
- Of course, there are other things I’m tracking as well on separate charts or tallies.
- Did I get to that gym twice a week? Am I up to my level 12? And have I stretched twice a week for 15 minutes? Now, to reach your project aim, you may have more than one measure you want to track.
- Accordingly, you may end up with more than one run chart.
- Let’s just focus on the one run chart, your first run chart, and make sure we get that one right.
