At Vulcan, we emphasize data to understand how our work is impacting the communities and customers we work with. Using data most effectively, however, requires more than collecting information in Excel spreadsheets. To be as clear as possible about our work’s impact, we build measurement and learning frameworks to inform how we collect, process, and evaluate data.
I was introduced to the field of Measurement and Evaluation in graduate school, but my interest in philanthropy and mission-driven work started much earlier. From a young age, I always knew that I wanted a job where I could help people. My mother and father were family therapists working with vulnerable populations, and our family was surrounded by other like-minded people who wanted to be part of a community and make the world a better place in whatever way they could. I thought that my contribution would be through direct service like my parents, and I dove head-first into working at a high school drop-out prevention program and eventually to being a case manager for homeless youth. I enjoyed the challenges and deeply personal rewards of direct service, but I also struggled with the design flaws that I observed in systems that were meant to help people. Why did we continue to implement the same programming when we didn't know if it was actually leading to better outcomes? My experience in direct service convinced me that if we are serious about solving persistent, wicked problems such as poverty and inequality, then we need to be just as serious about the solutions we design and implement to address those problems.
I share this story because it's important to remember our reason for doing the work that we do. At Vulcan, we work to tackle some of the world's toughest problems through technology, grants, storytelling, and advocacy. Part of my role as our Measurement and Evaluation Manager is to help our technology product teams define the problems they are trying to solve, the change they would like to achieve, and to map how use of the product will help end-users create that change (read more about Impact Metrics here). These conversations are the precursor to the Product Success Framework, a visual roadmap that we developed to outline our hypothesis for how use of the product will contribute to long-term impact outcomes. This is coupled with a Metrics Framework that we use to measure and report on progress at different stages in the product lifecycle:
Stages in the product lifecycle. (Note - Sustainability plans may involve monetization of the product, fundraising, open source software, or a combination of options.
The process of creating the Product Success and Metrics Frameworks also serves to drive alignment among team members and with our leadership by helping us articulate our assumptions and our definition of success. We hold that if we are achieving our goals in product development, usage, and sustainability, then our product is delivering on its promise as a tool for impact. However, we also monitor our Impact metrics and milestones because we know that our products are only part of the solution to a larger problem. If we do not observe a corresponding improvement in our impact metrics, we can refer to our Product Success Framework and see if there is something else we can do through grants, advocacy, or other solutions to support those front line implementers. By situating product success within the larger context of the problem we are trying to solve, we not only begin to understand our role in tackling a complex problem - we anchor our work to the impact goals that inspired our work in the first place.
Working to solve complex and deeply entrenched problems requires commitment, humility, and cooperation. I work in Measurement and Evaluation because I care deeply about improving outcomes for people, for animals, and for our planet. While challenging, the ability to engage in this type of work is a privilege and responsibility, and when I find myself feeling discouraged or frustrated, I try to remember the tremendous opportunity that is before us. Similarly, when we center our products within the context of the larger problem they were designed to address, we are able to see the bigger picture. This not only enables us to test our hypothesis for how change occurs and to learn from our partners and end-users, but also grounds us in our impact mission. And we remember why we are here.