Dude, what’s with this notion that nonprofits don’t have clear outcomes?

LinkedInGoogle+Share

hedgehog-468228_960_720 Hi everyone, this post will likely be my last coherent one for a while, because my second baby is due to arrive next Tuesday, March 15th (Eeeeeeek!) I plan to keep up with my weekly writing schedule, because I have my priorities. But that means the next 20 posts or so will reflect the hallucinogenic, meandering thoughts of a sleep-deprived father of a toddler and a newborn. Grammar and spelling may be questionable, and there will probably be a lot of baby-related analogies, such as “Restricting funding is like using duct tape as a diaper; sure, it makes you feel clever, but—OMG, please please just go to sleep, Daddy is so tired!”

For some wacky reason that I can’t comprehend, there seems to be this pervasive notion that nonprofits don’t have clear outcomes. In the past few months, I’ve heard this several times in various places. At a leadership seminar last June, for example, a colleague from the business sector said, “Nonprofits are just so squishy on outcomes.” I think squishy was her exact word. Or maybe slippery. Or fishy? Whatever, it was not complimentary. I got so annoyed I had to look at several pictures of baby animals on my phone to calm down.

Meanwhile, the comments on my post “Hey, you want nonprofits to act more like businesses? Then treat us like businesses” included this:

“I am often disconcerted by the lack of clear outcomes. In any for-profit business, you don’t launch without expected outcomes/metrics – qualitative and/or quantitative. These usually evolve and change along the way, but there is always a way for investors to gauge ROI and impact. Many nonprofits still don’t have these outcomes defined nor do they have consistent processes to evaluate and evolve. In the absence of a clearly discernible way to determine ‘success,’ funders are forced to ask for breakdowns or make (often detrimental) assumptions.”

Hearing these comments is really baffling, like seeing a dog walk backward, or trying vegan coconut “bacon” for the first time. Considering that most nonprofits write grants, and all grants force us to spell out outcomes, and many require us to create a logic model, which is a condensed chart of outcomes, I am confused why anyone thinks we don’t have clear outcomes. I mean, here’s my org’s evaluation map (created with help of our friends at TrueBearing). You may not agree with our outcomes, but to say we don’t have any clear ones is disingenuous and insulting. You don’t have to like my wild mushroom and chocolate chip risotto, but don’t you dare say I never cook!

This is actually kind of alarming. Maybe we nonprofits don’t do a very good job communicating with people outside our field, and that’s why there are these rampant notions and beliefs out there among the other sectors. Who knows what other ridiculous and damaging narratives are being reinforced in people’s minds? The quack notion that we should keep overhead below 15%? The misguided belief that nonprofits can be self-sufficient and “sustainable”?

So let’s talk about nonprofit outcomes then, because we need to dispel this bizarre and unfounded belief many of our friends tend to have. Here are several challenges with outcomes and evaluation, as pointed out by many of my nonprofit colleagues; this may help explain the confusion:

Nonprofit outcomes are complex: While most for-profits’ outcomes basically boil down to whether they made more money than they spent, we nonprofits don’t have that luxury. We are dealing with complex issues such as mental health, homelessness, education equity, domestic violence, human trafficking, community building, environmental justice, food security, access to art, etc.

Nonprofit outcomes are often not linear or predictable: A kid drops out halfway through a mentorship program. That nonprofit probably can’t count that toward a successful outcome. But who the heck knows how much impact his six months in the program had on him? Sometimes our outcomes take years to manifest. Heck, that kid, even though he dropped out, may just grow up and enroll his own kid in a similar program because he was exposed to the concept of mentorship.

Process is just as important as end results: This may be a strange concept for many people outside our sector, but HOW we reach outcomes is often as important as the outcomes themselves. To do it in such a way where we follow our values of equity and community and ensure the right people are leading and participating and in culturally competent ways adds another layer to the outcomes.

Return on Investment (ROI) is difficult to calculate: You can tell an investor that their dollars will yield a 5-to-1 return or whatever, but go ahead and try to figure pieces-of-the-puzzle-592798_960_720out ROI on providing mental health services or getting citizens to vote or helping close the academic achievement gap. Sure, it can be done, but that brings us to another challenge:

Evaluation takes time and money: Because of all the complexity, evaluation of nonprofits is incredibly time-and-resource-intensive in order to do a thorough and scientifically valid job. And there are ethical issues to contend with, such as the requirement for control groups, which means services would have to be denied to those who really need it. Unfortunately, funding for this area is skimpy and rarely sufficient, because it’s far sexier to fund the program itself and not stupid, frivolous things like evaluation.

The Danger with the Obsession with ROI

The biggest issue that I see with this confusion about nonprofits’ outcomes is the misalignment of values between the nonprofit sector and people demanding clearer and better outcomes. When most people say “nonprofits don’t have clear outcomes,” I think what they really mean is “Nonprofits don’t have outcomes that I like or find valuable.” A while ago I wrote this post, “Nonprofits’ ultimate outcome: Bringing unicorns back to the world,” about the damaging influence of this ROI-obsessed thinking, and about the intrinsic value of serving people. A colleague echoes the same frustrations:

“It irritated the crap out of me that it wasn’t enough to provide food, transportation, housekeeping, yardwork, companionship, and other services for frail and disabled seniors. I had to provide ‘outcomes.’ We de-slimed 60 seniors’ decks, stairs, and walkways each winter – but that wouldn’t be enough for the likes of you. No, I’d have to devote a third of our operating budget for a study to prove to you that this work would mean fewer falls broken hips and therefore lower medical bills to the government health care systems and better quality of life […] Can’t you accept that it’s a good thing for seniors not to fear that they’ll fall on their slippery walkways? I can tell you how many were done. That they can get to medical appointments? I can tell you how many miles they were driven. That they have a friend in the community? I can tell you how many hours of companionship. Why isn’t that good enough for you?”

This dissonance between our services and goals and what many people consider “good, clear outcomes” is dangerous and something we need to take seriously. It seems like we all have a lot of work to do to counter these damaging narratives about nonprofits, since it affects organizations’ funding and flexibility to do their work. Here are a few things we need to do:

Nonprofits: Speak up and communicate your organizations’ goals more clearly and overtly; chances are you know them very well, but your donors and funders and community may not. Also, push back on the people who say ridiculous things. When you hear “Nonprofit outcomes are squishy and fishy,” take a deep breath, smile, and ask some questions, such as “Would you mind elaborating on what you mean?” and “I’d love to hear your thoughts on, and maybe some examples of, good outcomes.” Chances are, they may just be repeating these lines, just like they repeat other lines like “nonprofits spend too much on overhead” without having much experience or knowledge of our sector. Use it as an opportunity to learn and to enlighten.

Funders: Increase funding for evaluation as well as planning around evaluation. It’s complex, so it’s expensive. You want nonprofits to be better at differentiating between outputs vs. outcomes, do a longitudinal study, disaggregate data into income level and zip codes and age and ethnicity? Then provide funds so we can get it done. If you require it, make sure you are helping to pay for it. While we’re at it, how about increasing funding for communication? That will help us provide clearer and more frequent messaging around our outcomes, which will help us increase donations and do our work better.

Everyone else: Listen, nonprofits have outcomes, all right? And we care about outcomes, and in fact, we’d rather people just allow us to focus on developing and achieving outcomes, instead of spending time on ridiculous things like figuring out which funder is paying for which part of the rent and who’s paying for toilet paper. Our outcomes are complex, usually difficult to measure, and maybe not even communicated all that well sometimes, because we have limited resources and we’re busy providing services. And yes, some of us suck at creating outcomes. And overall, you may not like our outcomes. But stop saying we don’t have any. It’s irritating and untrue. Please do us nonprofits a favor: If you want to know what a nonprofit’s outcomes are—ask.

***

I have a couple of exciting announcements!!! First, upon some colleagues’ requests, I’ve created a Facebook Group called “Nonprofit Happy Hour.” It’s a public forum for nonprofit professionals to provide advice, support to one another, and share jokes about unicorns and hummus. Please go join the group and actively participate. And tell your friends about it. Also, if you are or were an ED, there is a special confidential ED/CEO-only group (“ED Happy Hour”) where you can get advice from your colleagues.

Also, hilarious nonprofit guru Joan Garry just launched a podcast series, and I’m on the 4th episode, to be released tomorrow, 3/8/16. I don’t remember what I actually said, since hearing my own voice on tape creeps me out. Probably something about tequila. Check it out.

***

Make Mondays suck a little less. Get a notice each Monday morning when a new post arrives. Subscribe to NWB by scrolling to the top right of this page and enter in your email address. Also, join the NWB Facebook community for daily hilarity. 

And don’t forget NWB’s poetry contest, sponsored by Nonstop. You can win up to $750 in cold hard cash!

  • bigchicken170

    How does one request access to the ED’s Facebook Page?

    • Brad O’Donnell

      Very, very politely.

      • http://nonprofitiwithballs.com/ Vu Le

        That also helps 🙂

    • http://nonprofitiwithballs.com/ Vu Le

      we’ve been having some technical glitches with the ED Happy Hour page. I think it’s fixed now. Please try again.

  • Allison Mayfield Herrin

    Yes please! Would love to join ED Happy hour

    • http://nonprofitiwithballs.com/ Vu Le

      Allison, we’ve been having some technical glitches with the ED Happy Hour page. I think it’s fixed now. Please try again

  • Ann Tydeman-Solomon

    Another great post – thanks. Best wishes for lots of wonderful moments with new baby and big brother, interspersed with all those sleep deprived zombie moments!

    • http://nonprofitiwithballs.com/ Vu Le

      Thanks, Ann 🙂

  • sbc111

    A good post indeed. While the premise of that post is crucial as are outcomes – the fact is that it varies with the social sector the nonprofit thrives in. I think it was in one of your earlier posts that claimed that most (90% ?) of the grants go to well-established (or politically connected) organizations which raise outcomes to be questionable. Equally important is that certain social sectors like public health & healthcare truly have complex factors (categorized as #SDOH – Social Determinants Of Health). These can be more effectively addressed by a ‘Collective Impact’ approach. That is, a coordinated effort of multiple organizations attempting to bring out a more effective outcome to SDOH. Unfortunately, health disparities still persist – I put it at #PDOH – Policy Determinants Of Health as a contributing factor also. An example of ‘Collective Impact’ in Public Health _
    http://www.amchp.org/AboutAMCHP/Newsletters/Pulse/Archive/2012/NovDec2012/Pages/Feature5.aspx

    • http://nonprofitiwithballs.com/ Vu Le

      Thanks, sbc111. While I support collective approach, it does have some weaknesses that should be addressed, including engagement and funding of communities of color

      • sbc111

        Actually, it is in the communities of color where ‘CI’ is more needed, especially in the arena of health disparities. In quite a few cases, the fundamentals are missing – e.g., Race, Ethnicity & Language (R|E|L) data collection & utilization in the healthcare context (see my link below). Another fundamental problem is poor governance itself which invariably leads to poor compliance that results in poor outcomes. ‘CI’ brings in others with common goals and hopefully, with more transparency. Here’s my legislative testimony from last week’s CT Legislative Bill HB 5451 which also states about measurements (via R|E|L data) and health disparities _ https://www.cga.ct.gov/asp/menu/CommDocTmyBillAllComm.asp?bill=HB-05451&doc_year=2016

  • Matthew Turner

    The senior services example above made me think about another way one can demonstrate the burden of evaluation and reporting to non-profits. Non-profits are often required to show advanced metrics and long-term outcomes. Not just “did you provide a service” but “who got this service and how will this service make a better person and a better society?” Now, let’s do the same for a for-profit. How about a corner grocery store. So, the store sold x number of candy bars this week. In a simple for-profit model, the candy bars cost the store some amount, they sell them for more than that, and make a profit. Now let’s add the non profit requirements:

    How many unique customers purchased a candy bar this week?

    What gender, age, and ethnicity were these customers?

    Is that the first candy bar that they purchased this week, or have there been many candy bars?

    Did they buy candy bars in any other stores this week?

    Is this candy bar increasing your customer’s risk for diabetes or obesity?

    Are you destroying the physical health and moral fiber of your community by selling candy bars!? Why aren’t you selling more bananas?

    (and, you know, the list goes on).

    If the corner store was required to gather and report all of this information, not only will it increase the effort it will take to sell the candy bar (thereby decreasing profit or increasing cost), but it sure will make it difficult for the poor customer who now has to fill out a few form and just wanted to buy a candy bar.

    • http://nonprofitiwithballs.com/ Vu Le

      Matthew, that is a brilliant analogy! Also, “Please make sure none of my money goes into staff time. I only want to fund the bars themselves, not staff who sell candy bars.”

      • Becca

        Similarly, pretend for a second that these are ice cream candy bars: “I only want to pay for the bars themselves, and we’ll make you jump through hoops to pay for any portion of the refrigeration cost. And forget about paying for the annual food safety inspection.”

    • S NV Nonprofit Info Ctr

      Matthew, what an excellent and simple analogy. Can I “borrow” it for some of my nonprofit workshops at http://www.lvccld.org/library/special/#grants ? ~ Shelly

      • Matthew Turner

        sure thing.

  • Becca

    Vu, congratulations to you and your family on your upcoming new addition!!!

  • Kate Lockard Snyder

    YES! Oh my gosh, your senior services example was great. I used to feel that way all the time when I worked for a human services organization. Among other things, we ran a soup kitchen. It made me absolutely insane that it wasn’t enough to say “people got a good meal” and then say how many were served. How is that not an acceptable outcome?

  • Sophia Katt

    Maybe if all nonprofits had staff with these skills outcomes would not be so hard: http://jstahl.org/archives/2014/07/02/nonprofit-tech-skills/

    And Jon lives here in Seattle, too. Reachable.

  • Joel R Putnam

    Devil’s advocate here, do you think maybe an alternative translation of “squishy” might be “complicated, nonlinear, not representative of process and expensive to measure”? It seems likely that the disconnect is not whether or not they value what nonprofits do but that they don’t see a simple number and wouldn’t entirely trust it if they did (and nor should they in most cases, as you can’t reduce our work that easily). If you (as they do) live in a culture that values and trusts numbers more than anecdotes, you can see that the discomfort might not be because they don’t value your work or mission, they’re just used to figuring out how to make things work better using tools that aren’t realistically available here.

  • S NV Nonprofit Info Ctr

    As a fellow vegan, I just LOVE coconut “bacon”. Much healthier than Bac-Os. Now I’ll go back to reading the rest of the story (past the vegan bacon line), as Paul Harvey would say.

  • http://socialkeenan.wordpress.com Keenan Wellar

    This is certainly very accurate as pertains to ignorant outsiders with a vague notion of how charitable organizations operate and contribute to society and communities.

    An amazing outcome in my sector could be that someone went three months without hurting themselves or others. The combination of art and science involved in that outcome and a presentation about the it’s value will be hard to quantify in the same way that one would present on coffee sales. Coffee sales are important. So too is saving a life. But these are not the in the same category of “business” and it’s simply ridiculous to suggest that they should account for their expenditures in the same way.

    On the other hand – as an insider – to me the bigger problem is that we DO have a significant percentage of charitable organizations that do not have an outcome or impact focus – some of these actually look more “like a business” and would rank well on one of those useless charity evaluator websites. They just happen to be doing a crappy job – if one agrees that the primary measurement in the charity sector is how effectively we deliver positive social change.

    Some charities quite literally have a mission statement such as “To run day programs.” So they account for the number of participants and the revenues less expenses and maybe a little survey proving that that participants are “happy” with the program.

    In that situation criticism about lack of outcomes is fair. Because a proper analysis may well show that a day program delivers lousy outcomes at a high cost. But far too many charities just “do what they do” and if they do it in the black and have some data to prove they are “doing something” then often they will go unchallenged. And we should challenge this, even as we stand together against misguided mythologies about our inability to “run it like a business.”

    • S NV Nonprofit Info Ctr

      I can’t tell you how many people come to our info center to learn how to “become a nonprofit”. When I ask them what the mission of the org. is/will be, about 45% of the time I actually get the reply “to help/save people/animals/places” (ME: umm, can we be a bit more specific here?) and/or “to get grants” (ME: ain’t gonna happen, no way, no how).
      As for evaluation, I teach a class every few years on designing program evaluations. One time, I asked the attendees, “What do you think one of your goals should be?” One person said “We didn’t help anyone this year” For once, I was speechless.
      I’m surprised I don’t purchase new wigs on a monthly basis LOL.

      • http://socialkeenan.wordpress.com Keenan Wellar

        Exactly. Just as there is no one to say that you cannot open a restaurant juts because you’ve never cooked anything, it takes $200 and a bit of paperwork to start a non-profit. There are people who start non-profits in probably the same numbers as people who should not start restaurants.

  • Rachel Rosenbaum

    Hi Vu, thank you so much for this important post. I have a background working for nonprofit arts organizations, and measuring outcomes is a topic I often wrestle with. How do new or emerging arts organizations communicate the data-driven outcomes funders often want, especially in situations where qualitative outcomes seem much more appropriate? It’s challenging to speak out about the outcomes we know best, while also trying to cater to the numerical language funders care about most.

    I really appreciate your observation that we need to talk about evaluation in a way that engages funders and empowers nonprofit leaders at the same time. The nonprofit sector provides so many different types of services, answers, relief, and value to our complicated world. What gave anyone the idea that our evaluation methods were going to be simple and standardized?

  • Sara

    Thank you SO MUCH for sharing this blog post with the nonprofit community! I’m currently a Development Intern who is designing a grant evaluation system for a civil & human rights organization, and I’ll definitely be referencing this blog to motivate my colleagues to loudly/proudly share our outcomes with prospective funders 🙂

  • CRS: Evaluation into Action

    I particularly appreciated your comment about funders
    funding what they want. That evaluation requires resources. I appreciate
    funders who view evaluation as a separate line item, not something to just make
    work within a current budget.

    However, outcomes being clearly created and articulated has
    to come from a place where it is valued in the organizational culture.
    Historically, funders mandate evaluation; therefore, it is reluctantly done
    because it’s a culture of compliance. (i.e., you must eat your vegetables)

    If funders and nonprofits partner, and shift evaluative
    thinking to valuing using data to make decisions (and funders provide resources
    to do so). Then, the culture becomes one of learning.

    How does this relate to outcomes?

    Outcomes are just one part of the evaluation process.
    Really, the cornerstone of what data are collected.

    A common question from nonprofits – “What do I ask in a
    survey?”

    The response – “Well, what do you expect to change?”

    The outcomes drive the content of all data collection tools.
    They are change statements.

    In the senior services example, I agree. You can assume if
    walkways aren’t slippery, then seniors are less likely to fall. In this case,
    an outcome might be “to increase the safety of seniors” and measuring it is
    really outputs with an assumption statement. The number of decks cleaned off
    and made safe…well you get the idea…because I feel like I’m stepping on a large
    soap box now.

    The bottom line – outcomes are important. They are complex,
    and important for any nonprofit to internally agree on what exactly they’re
    trying to change. They are also unique to each organization, which is why the
    ROI can be like fitting a square peg in a round hole.

    If you’re wondering, yes, I’m an evaluation consultant (guilty)
    with lots of passion about this topic. Thank you for opening up the conversation.

  • Alex Ong

    Helping to eliminate this stigma is a challenge the sector needs to collectively face to change the perception that outcomes/data/metrics are not important to understanding an organization’s success. To this end, I recommend a simple exercise: create an infographic. There are free platforms (Canva, Piktochart) that make it easy to design one that can be proudly shared beyond one’s refrigerator. You just need to come up with the info part. If you need inspiration google “nonprofit infographic” or search the same thing in Pinterest. The process of gathering the data, in itself, can be beneficial in understanding what metrics are important to an organization.

    If it is your first go at an infographic, I suggest not trying to create something so brilliant that you expect the world to react with “oohs and ahhs”. Focus on creating something to be shared internally and eliminate that pressure of satisfying outside requirements. Create a graphic representing 2015 accomplishments. What are you proud of? Raised X amount of dollars? Gained X amount of volunteer hours? Drank X amount of cups of coffee? After creating your infographic, you may discover that it is brilliant and should be shared with the rest of the free world. And even if you decide to relegate it to the virtual circular file, build on your experience and create the next on with more purpose and benefit in mind.

    Create one every month, every other month, or every quarter, so that it becomes habitual and is perceived as important. If it becomes addicting, seek immediate help.