Transparency Talk

« Building a Community for Excellence in Evaluation | Main | Why Glasspockets? An Interview with Foundation Center President Brad Smith »

Foundations Fail at Failing
January 18, 2011

Michael Remaley is the director of Public Policy Communicators NYC and president of HAMILL REMALEY breakthrough communications.

"If you hit the bull's eye every time,
you've set the target too close."

I thought of this, one of my favorite aphorisms, at the Communications Network's annual conference last September when the Hewlett Foundation's Communications Director Eric Brown talked about his organization's "failed grantmaking" contest.  Hewlett's smart internal exercise forces each department to name one grant from its portfolio that did not meet expectations, think through and explain what went wrong and help the entire organization learn from its failure. 

This is a learning exercise that more foundations should consider adopting. But more than that, it is an important example of how Hewlett's leadership has set the tone for candor about the unavoidable truth of philanthropic experimentation: failure is part of the equation. 

It is no coincidence that Hewlett is also one of the few foundations that has talked publicly about initiatives that didn't live up to expectations. It is also no coincidence that Hewlett's profile on Glasspockets gives a good indication of its commitment to transparency.  I would assert that Hewlett's reputation for being one of the most innovative, thoughtful, and effective foundations is directly related to its transparency, willingness to publicly question its strategies, and forthrightness in discussing the limitations of its successes. And that reputation further enhances its ability to exert influence and make change.

The hard sciences learned the importance of sharing candid assessments of "failed" experiments centuries ago. In fact, scientists seem to treasure results that do not meet expected outcomes even more highly than those that confirm what is already believed to be true. 

I am hardly the first person to call upon foundations to talk more openly about failure, experimentation, and unexpected outcomes. (See list below.) Hewlett's Paul Brest seems to have really kickstarted the conversation in 2007 by writing and talking about his foundation's experiences. That was followed by Robert Giloth and Susan Gewirtz's seminal 2008 piece in Foundation Review, "Philanthropy and Mistakes: An Untapped Resource." Many others, including Bob Hughes, Larry Blumenthal, Edward Pauly, Grant Oliphant, and Sean Stannard-Stockton, have added important insights about the need for foundations to be more open about their lessons learned.  The conversation about failure and experimentation seemed to grow and deepen over the past three years.  So you might think that foundations would be making major changes in how they communicate about failure.  You would be wrong.

Foundations give a lot of lip service to supporting "experimentation" in social sciences. But you almost never hear them talking about outcomes that failed to meet expectations, and even more rarely, those that call their basic strategies into question. If foundations want to be real leaders in advancing social change, they must move past the endless happy-talk that makes every grant sound like a success. Instead, they should use their web sites to detail how they are evaluating their work and what they've learned from unexpected outcomes. 

A foundation sharing its experiences with grants gone wrong is still very much the exception.  Anyone who is on the receiving end of foundation annual reports and newsletters knows this is true.  But to substantiate my assertion, I decided to do a little systematic poking around.

I figured the 21 largest supporters of the Center for Effective Philanthropy (most of which are also supporters of Grantmakers for Effective Organizations) would be the foundations most attuned to the value of self-reflection, evaluation, and sharing results that defy expectations, and also those that would have budgets big enough to support substantial evaluation efforts. I spent many hours exploring the nooks of crannies of these foundations' web sites.  I looked at numerous publications and evaluation sections of the sites, and I searched each site on the terms failure, failed, unmet expectations, unmet objective, unmet goal, experimentation, mistake, lessons learned, and assessment.

What I found was that few foundations make it easy to learn from projects that didn't go as spectacularly as planned, let alone talk frankly about what has been learned from the shortcomings of foundation strategy or execution.   Many of the 21 foundations I examined made no mention at all of evaluation criteria and organizational outcomes, even though their association with CEP and GEO implies that they demand that kind of forthrightness from grantees. The majority of the foundation sites I examined had a few project evaluation reports scattered among other foundation supported research – and many of those evaluation reports were laudatory with pablum like "real collaboration is a challenge" tacked on at the end. 

Some of the best exceptions were Robert Wood Johnson Foundation, the William and Flora Hewlett Foundation, and the Wallace Foundation. Each of those foundations not only makes it easy to find many project evaluations that are balanced in presenting positive and negative outcomes along with what was learned through the process, but also present self-critical examinations of foundation strategy and progress as whole. It is also not a coincidence that each of those foundations' profiles on Glasspockets indicates a commitment to transparency demonstrated by making public an assessment of overall foundation performance.

But perhaps the best example – the foundation that gets the Gold Star for Succeeding in Failing – is the James Irvine Foundation. The evaluation section of its site describes their approach to evaluating grantee success and links to all of its individual evaluations of initiatives. It also links to a Foundation Assessment section that has foundation annual progress reports for the last four years.  These progress reports are exceptionally detailed and well-documented, as well as frank about successes and failures.  Irvine has also produced "Insights: Lessons Learned" publications with candid assessments of their experiences with collaborations and other grantmaking practices. A search of the Irvine site on "lessons learned" produces lots of useful and interesting evaluative information and insightful critical analysis.

We are all members of the social science community and contributors to the social experiment that is American philanthropy. We now have enough examples of foundations talking humbly about their shortcomings to know that such candor only accelerates social progress and enhances the reputations of those philanthropic leaders. We've seen no evidence that talking forthrightly about the real-world circumstances leading to failure damages nonprofits or the foundations involved, so I wonder why foundations seem so reluctant to take on this leadership role.

What has your organization learned from experiments that didn't meet expectations?

Selected Readings:
A Chronology of the Dialogue on Failure
and Experimentation in Philanthropy

— Michael Remaley

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Bravo to Michael Remaley and the truly transparent organizations he champions! We need much more accountability from those who invest other people's money in nonprofit efforts, and we need to be willing to use the truth we find to guide future work rather than just as a marketing tool to seek more funding. For further thoughts on how accountability is a good thing for nonprofits - including foundations - who are willing to engage in it, see:

http://www.peppergrasspartnerships.com/resources/partnerships-accountability-and-consequences/

Sheila Marsh


Bravo to Michael Remaley and the truly transparent organizations he champions! We need much more accountability from those who invest other people's money in nonprofit efforts, and we need to be willing to use the truth we find to guide future work rather than just as a marketing tool to seek more funding. For further thoughts on how accountability is a good thing for nonprofits - including foundations - who are willing to engage in it, see:

www.peppergrasspartnerships.com/resources/partnerships-accountability-and-consequences/

Sheila Marsh

This topic has long been an area of interest among the membership of Grantmakers for Effective Organizations. Some of the questions our members grapple with include, how do we apply what we've learned in a useful way? How do we encourage candor from our grantees? How do we speak publicly about failures without putting our grantees in compromising positions? In 2011, GEO plans to engage a cohort of our members in peer learning around the topic of leveraging failures. We will share the results of what we learn from this cohort. Funders interested in learning more about this work can contact Leonor Alfonso at alfonso@geofunders.org.

One simple way to do this is to pose a simple ratio for each foundation of successes. this eliminates the stigma of publicly highlighting failures. A 100% success ratio, would naturally be suspect, similar to winning a democratic election with 99.9% of the vote.

A simple success ratio would at least allow for the acknowledgement of failure and begin frank discussions. It is a simple question all donors should ask. What is your success ratio? the follow on questions from there would lead to interesting opportunities for learning and understanding. Failure isn't all bad if it is learned from or acknowledged as research into the untested.

sharing those results can help everyone understand what succeeds better.

I'm a strong supporter of "open source" grant reporting. I think the whole NGO community would benefit greatly if final reports were much more broadly disseminated online so that researchers and others could readily find similar projects, talk to the practitioners about what worked and what didn't and explore the impact of projects and people over time. Imagine being able to track particular leadership across different organizations within a community over time. Or to look for cumulative impacts of different projects. A treasure trove for researchers which now is hidden away in file cabinets.

Sylvia,

Great to hear from you. Thanks so much for sharing the Gaining Perspective document. I am hoping to be doing some follow up research on this topic and your piece will be a very nice addition to the information I'm gathering.

best, -michael

For another view about talking about "foundation failure," check out this video interview with Grant Oliphant, president of the Pittsburgh Foundation: http://bit.ly/5bYsMA

The reports mentioned in this post were among models we examined when the Northwest Area Foundation initiated an effort to reflect on and experiences from a decade of work. We hope our just-released report "Gaining Perspective: Lessons Learned from One Foundation's Exploratory Decade" encourages conversation and the sharing of both stumbles and achievements among fellow funders. The full report can be downloaded at http://www.nwaf.org/content/Lessons.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

About Transparency Talk

  • Transparency Talk, the Glasspockets blog, is a platform for candid and constructive conversation about foundation transparency and accountability. In this space, the Foundation Center highlights strategies, findings, and best practices on the web and in foundations–illuminating the importance of having "glass pockets."

    The views expressed in this blog do not necessarily reflect the views of the Foundation Center.

    Questions and comments may be
    directed to:

    Janet Camarena
    Director, San Francisco Office
    The Foundation Center

    If you are interested in being a
    guest contributor, contact:
    glasspockets@foundationcenter.org

Subscribe to Transparency Talk