A young man is walking along a beach where thousands of starfish have washed ashore. Further along he sees an old man, walking slowly and stooping often, picking up one starfish after another and tossing each one into the ocean.
"Why are you throwing starfish into the ocean?" he asks.
"Because the sun is up and the tide is going out and if I don't throw them they will die."
"But don't you realize there are miles of beach and starfish all along it? You can't possibly save them all, you can't even save one-tenth of them. In fact, even if you work all day, your efforts won't make any difference at all."
The old man listens and then bends down to pick up another starfish and throws it into the sea. "It made a difference to that one."
Everyone knows the story.
It's a great story about perseverance and compassion, about the small but tangible impacts we can have even when faced with what seems like an insurmountable problem. In the difficult world of delivering social interventions, it’s stories and ideas like this that get people out of bed and into work every day—the knowledge that whatever else is happening in the world, we can make a positive change for someone.
As an evaluator, I count myself among those whose work is dedicated toward improving the lives of others. While evaluation may seem a few steps removed from the difficult and important work of direct service delivery, if you speak to evaluators you will find that most of us are not in it just because we love numbers and metrics so very much. Sure, we may enjoy a good spreadsheet, but we do what we do because we sincerely believe that our work is valuable and necessary and makes a difference to the organizations and people we support, and therefore to the organizations and people they support. 'Helping people who help people' is my professional motto for a reason.
So what about starfish?
Evaluation is a weird job. I firmly believe that evaluation is a fundamentally optimistic venture, even though one of the central questions of our profession is, 'Okay, but what if it's not working?' Evaluators (and those who think evaluatively) believe so strongly in the capacity of social initiatives to produce real change that we are dedicated to testing and challenging that capacity, confident that if we haven't figured out how to make it work yet, we can and will, or we will find a way that's better: more beneficial, more transformative, or more sustainable. (And 'we' here refers to everyone invested in the change, not just us picky data-crunchers.)
In the story there is a skeptic and a believer, but to do evaluation you must be both. The pure skeptic astutely observes the gap between the scope of the issue and the actions that are being taken to address it and responds by giving up. In contrast, the true believer and the hero of the story changes the scope of the problem—he is not saving all starfish, he is saving that starfish—and so persists.
But the single-starfish approach to social change has been criticized before, and legitimately. It can lead to short-sighted and misguided interventions which lack insight and fail to allocate resources wisely, and, more importantly, which fail to challenge the sustaining structures of social conundrums. In a world of complex, urgent, multifaceted and dynamic problems and crises, more and more we are being called on to develop solutions that span communities, institutions, sectors, and other traditional boundaries, meet multiple needs, and do so within the ever-limited resources, budgets, and staff available.
In the face of this complexity, there is something especially human and necessary about reassuring ourselves of the tangible impacts we can have through our individual efforts, and something unassailably pragmatic about approaching the daunting task of social change one starfish at a time. More than anything, I think the value of this approach is in its sustainability. What faster way to burn out and lose hope than to look at a beach covered in starfish and think, 'How am I going to make a dent in this?' And what is more sustaining than remembering those singular moments when you know you were able to reach out and change someone's life for the better? But at the end of day, more of those positive changes is what we're all aiming for, not just more memorable ones.
Skeptical believers
So what do we do when we're seeking profound, transformative changes that are also within our grasp? This is where we can turn to evaluation, refuge of the skeptical believers, to start bridging the gap between what seems within our control and what seems well beyond it. The evaluation-informed approach to the starfish problem is neither to give up before getting started nor to persist doggedly without pause or reflection. Being evaluative means stopping to ask the questions, Do I know what I'm trying to do? and Is what I'm doing the best way to make that happen? An evaluator walking along the beach might have asked why there were so many starfish there in the first place, or questioned what ultimately happened to the ones that were thrown back in.
We often associate evaluation with post-intervention processes, with measurement and trying to line up a complicated and messy reality with approximated indicators and a plethora of survey responses. (Evaluating social change is about as tricky and frustrating as creating social change, mostly for the same reasons.) In the usual approach, evaluation is the thing you do after the thing that you did to see if you did it okay or not, often after it's too late to do anything about it.
Evaluation is about not just about answering questions, it's about making sure we're asking the right ones in the first place.
But evaluation is about not just about answering questions, it's about making sure we're asking the right ones in the first place. Evaluation requires that we think ahead, plan our actions, try to foresee challenges, take stock of resources, consider alternatives, and figure out how we're going to know if all the starfish-throwing made a difference in the end. After all, believing an intervention works isn't the same as knowing it works. Evaluation is more than tallying up your starfish count at the end of the fiscal year (or asking the starfish how satisfied they were with the force and frequency with which they were chucked into the ocean). It's a reflective process full of tools for better understanding how your programs and systems are working, making informed decisions about what to do next, and providing accountability and transparency to others concerned with the general welfare of starfish (not to mention the starfish themselves, who are notably and unrealistically passive in this metaphor). And it's more useful the sooner we bring it into the change process.
It may seem like a luxury in world too many beaches, too many starfish, and not enough people strolling down the sand with patience and a good arm, especially while the sun keeps rising and the tide keeps going out. It also may seem defeatist, assuming we might fail before we even get started. But failure is only failure when we don't use it as an opportunity to learn and grow. Evaluation doesn't mean never throwing a starfish. Sometimes it means throwing it with better aim.
* This version of the starfish story was adapted from http://www.esc16.net/users/0020/FACES/Starfish%20Story.pdf. The story itself is also an adaptation from an essay by Loren Eiseley, 'The Star Thrower'.