Evidence-Based Baseball and the Social Service Enterprise

hanging out shoesMajor League Baseball emailed me this week to let me know that my renewal subscription to mlb.com is ready, so I can watch up to 200 spring training games and 2340 regular season games. It is typical that they told me about the upcoming season in numbers. Baseball may be art. It may be sport. But baseball is also statistics. Before Nate Silver started forcasting U.S. elections, he was a bonafide baseball statistician.

Baseball managers manage by statistics. Should I pinch hit for lefthanded hitter Skip Schumaker late in the game against a left handed pitcher? Let's look at the evidence. What are his batting statistics against left handers? Against this pitcher? With runners in scoring position?

Baseball is an evidence-based sport. Teams subscribe to statistics systems and employ their own statisticians and programmers to make sure that their general managers and managers have the best numbers to make the best decisions. Yes, a manager gets to play hunches, but they are weighed against the evidence, evidence that is accessed in real time.

Early on, baseball was ruled by the simple count statistic. How many hits did this player have? How many starts did this pitcher make. These statistics can be misleading. A player can get more hits by rarely taking a base on balls. A bad pitcher can make a lot of starts, but pitch few innings. The ERA was introduced in 1912 to give a better account of a pitcher's effectiveness. The save was introduced in the 1970s. In the past 20 years, a host of young Nate Silvers have brought us a new set of metrics to the game.

One statistic du jour is the Wins Above Replacement or WAR, which purports to predict the number of wins a player was worth to his team over a 162 game season, above the typical replacement player. It is most valued by the executives who have to decide what to offer players to play for their team. Last year’s rookie phenom Mike Trout led the majors with a WAR of 10. He was thought to be worth 10 wins to his team over the typical center fielder who could have replaced him.

How about the typical manager in a social service agency? Is she managing by evidence? Does she have real time numbers to make real time decisions?  How about the administrator in a social service agency. Does she have last year’s stats to make this year’s decisions? Does she have a metric to tell her who her best supervisor or worker is? Does she have any numbers that tell her anything at all? Is she managing by 2012 metrics, 1911 metrics or no metrics at all?

The social service world is just as complex as baseball, but we measure fewer things and we report out even fewer.  In the social services, we have been ruled by the most basic of statistics: the count. How many people did this agency serve last year? How many people entered a nursing home last year for the first time? How many children died at the hands of a relative last year?

These counts mean as much as judging the best pitcher by the number of starts made.  (BTW, Don Sutton made the third most starts ever and no one thinks of Don Sutton as one of the best pitchers of all time).

Now imagine a world where social services decision makers had the kinds of metrics baseball decision makers have. Imagine, for example, a typical senior care manager charged with safely arranging services to keep older folks out of nursing homes. Let’s say a “win” in this situation is a senior who is safely maintained in her home for a year. If you supervised such people, wouldn’t you like to know the WAR of each of your workers, the number of senior service wins each worker gave you last year over a typical replacement worker? As her supervisor, you might have your own WAR statistic. Due to your supervisory skills, how many “wins” above your typical replacement were you worth?

In the social service world, people like me are advocating for the balanced scorecard approach, a move toward the baseball statistics of the 1920s. We want social service programs to pick five or so indicators that together, show us how an agency is performing.  I want to be able to peruse agency statistics on the web the way I looked through baseball statistics in the Sunday newspaper as a kid.

A balanced scorecard for the senior services agency might include these five indicators:

  • how many families were served;
  • of those served, what percentage went into nursing home care;
  • of those served, what percentage thought that they were helped;
  • what percentage of these families would recommend the agency to a friend in need of senior services;
  • what percentage of employees were satisfied or very satisfied with their jobs.

As a general manager in baseball will tell you, one season is one season. They want to know how a player has performed year-to-year to help them predict potential of young players and whether a player is on the upswing or downswing of his career. So, ideally, we would want to see these five indicators charted over time. With five important indicators measured the same way over time, an agency has some idea of where it stands, whether it is gaining or losing clients, whether they are doing better or worse at keeping clients out of nursing homes, and whether consumer and worker satisfaction is increasing or decreasing.

But what about that individual worker? Few social service agencies are using worker metrics. What should we measure for her? What percentage of her clients went into nursing home care? What percentage of her family clients would recommend her service? This is, of course, more controversial, akin to measuring how well an individual teacher's students perform on standardized tests.

So the next time you watch a baseball game and see a player's or team's stats flashed on the bottom of the screen, think about us in the social services world. What stats should we be flashing? What is the most advanced or useful metric you’ve seen used in the social services? Who is our Nate Silver?

 

 

 

 

 

 


Leave a Reply

Your email address will not be published. Required fields are marked *