Transparency Talk

Category: "Survey" (5 posts)

Creating a Culture of Learning: An Interview with Yvonne Belanger, Director of Evaluation & Learning, Barr Foundation
November 8, 2018

Yvonne Belanger is the director of learning & evaluation at the Barr Foundation and leads Barr's efforts to gauge its impact and support ongoing learning among staff, grantees, and the fields in which they work.

Recently, Janet Camarena, director of transparency initiatives for Foundation Center, interviewed Belanger about how creating a culture of learning and openness can improve philanthropy. This post is part of the Glasspockets’ #OpenForGood series in partnership with the Fund for Shared Insight. The series explores new tools, promising practices, and inspiring examples showing how some foundations are opening up the knowledge that they are learning for the benefit of the larger philanthropic sector. Contribute your comments on each post and share the series using #OpenForGood.


YvonneGlassPockets: More and more foundations seem to be hiring staff with titles having to do with evaluation and learning. You’ve been in this role at the Barr Foundation for just about a year, having come over from a similar role at the Bill & Melinda Gates Foundation. Why do you think roles like this are on the rise in philanthropy, and what are your aspirations for how greater capacity for evaluation and learning can benefit the field?

Yvonne Belanger: I think the spread of these roles in strategic philanthropy comes from increasing recognition that building a stronger learning function is a strategic investment, and it requires dedicated expertise and leadership. My hope is that strong evaluation and learning capacity at Barr (and across the philanthropic sector generally) will enable better decisions and accelerate the pace of social change to make the world more equitable and just.

GP: What have been your priorities in this first year and what is your approach to learning? More specifically, what is Barr’s learning process like, what sources do you learn from, how do you use the learnings to inform your work?

YB: At Barr, we are committed to learning from our efforts and continuously improving. Our programmatic work benefits from many sources of knowledge to inform strategy including landscape scans, academic research, ongoing conversations with grantees and formal site visits, and program evaluations to name a few. During this first year, I have been working with Barr’s program teams to assess their needs, to sketch out a trajectory for the next few years, and to launch evaluation projects across our strategies to enhance our strategic learning. Learning is not limited to evaluating the work of our programs, but also includes getting feedback from our partners. Recently, we were fortunate to hear from grantees via our Grantee Perception Report survey, including specific feedback on our learning and evaluation practices. As we reflected on their responses in relation to Barr’s values and examples of strong practice among our peers, we saw several ways we could improve.

GP: What kinds of improvements are you making as a result of feedback you received?

YB: We identified three opportunities for improvement: to make evaluation more useful, to be clearer about how Barr defines success and measures progress, and to be more transparent with our learning.

  • Make evaluations more collaborative and beneficial to our partners. We heard from our grantees that participating in evaluations funded by Barr hasn’t always felt useful or applicable to their work. We are adopting approaches to evaluation that prioritize grantee input and benefit. For example, in our Creative Commonwealth Initiative, a partnership with five community foundations to strengthen arts and creativity across Massachusetts, we included the grantees early in the evaluation design phase. With their input, we modified and prioritized evaluation questions and incorporated flexible technical assistance to build their capacity for data and measurement. In our Education Program, the early phase of our Engage New England evaluation is focused on sharing learning with grantees and the partners supporting their work to make implementation of these new school models stronger.
  • Be clearer about how we measure outcomes. Our grantees want to understand how Barr assesses progress. In September, we published a grantee guide to outputs and outcomes to clarify what we are looking for from grantees and to support them in developing a strong proposal. Currently, our program teams are clarifying progress measures for our strategies, and we plan to make that information more accessible to our grantees.
  • Share what we learn. To quote your recent GrantCraft Open for Good report, “Knowledge has the power to spark change, but only if it is shared.” To maximize Barr’s impact, we aim to be #OpenForGood and produce and share insights that help our grantees, practitioners, policymakers, and others. To this end, we are proactively sharing information about evaluation work in progress, such as the evaluation questions we are exploring, and when the field can expect results. Our Barr Fellows program evaluation is one example of this practice. We are also building a new knowledge center for Barr to highlight and share research and reports from our partners, and make these reports easier for practitioners and policymakers to find and re-share.

GP: Clearly all of this takes time and resources to do well. What benefits can you point to of investing in learning and knowledge sharing?

YB: Our new Impact & Learning page reflects our aspiration that by sharing work in progress and lessons learned, we hope to influence nonprofits and other funders, advance field knowledge, inform policy, and elevate community expertise. When you are working on changing complex systems, there are almost never silver bullets. To make headway on difficult social problems we need to view them from multiple perspectives and build learning over time by analyzing the successes – and the failures - of many different efforts and approaches.

GP: Barr’s president, Jim Canales, is featured in a video clip on the Impact & Learning page talking about the important role philanthropy plays as a source of “risk capital” to test emerging and untested solutions, some of which may not work or fail, and that the field should see these as learning opportunities. And, of course, these struggles and failures could be great lessons for philanthropy as a whole. How do you balance this tension at Barr, between a desire to provide “risk capital,” the desire to open up what you are learning, and reputational concerns about sharing evaluations of initiatives that didn’t produce the desired results?

YB: It’s unusual for Foundations to be open about how they define success, and admissions of failure are notably rare. I think foundations are often just as concerned about their grantees’ reputation and credibility as their own. At Barr we do aspire to be more transparent, including when things that haven’t worked or our efforts have fallen short of our goals. To paraphrase Jim Canales, risk isn’t an end in itself, but a foundation should be willing to take risks in order to see impact. Factors that influence impact or the pace of change are often ones that funders often have control over, such as the amount of risk we were willing to take, or the conceptualization and design of an initiative. When a funder can reflect openly about these issues, these usually generate valuable lessons for philanthropy and reflect the kind of risks we should be able to take more often.

GP: Now that you are entering your second year in this role, where are the next directions you hope to take Barr’s evaluation and learning efforts?

YB: In addition to continuing and sustaining robust evaluation for major initiatives across our program areas, and sharing what we’re learning as we go, we have two new areas of focus in 2019 – people and practices. We will have an internal staff development series to cultivate mindsets, skills, and shared habits that support learning, and we will also be working to strengthen our practices around strategy measurement so that we can be clearer both internally and externally about how we measure progress and impact. Ultimately, we believe these efforts will make our strategies stronger, will improve our ability to learn with and from our grantees, and will lead to greater impact.

 

Transparency Chat: CEP On Sharing What Matters
March 2, 2016

CEP_Ellie-ButeauEllie Buteau, Ph.D., is the vice president of research at the Center for Effective Philanthropy (CEP), which received a grant from the Fund for Shared Insight (FSI). FSI is a multi-year collaborative effort among funders that pools financial and other resources to make grants to improve philanthropy. Transparency Talk is featuring grantees in the FSI openness portfolio. Janet Camarena, Foundation Center’s director of transparency, and Ms. Buteau discussed the findings of CEP's new report, "Sharing What Matters: Foundation Transparency."

Janet Camarena:  I'm going to start with what jumped out at me as surprising. The report lists time and inconsistencies across staff members as the most common barriers to greater foundation transparency.  Only 6% responded to your survey that a lack of commitment to transparency was a barrier and a full 24% responded that there was nothing specific that limited their foundation's transparency. Could this be because those surveyed are already predisposed to pushing the effectiveness envelope? Can you talk a little bit about the survey sample and how representative it might be? 

Ellie Buteau:  Yes, definitely. Response bias is always a top-of-mind question when we conduct a survey. The main bias we wondered about for this study was whether or not foundations that are already working on, and care about, transparency were more likely to respond. Unfortunately, we have no way of reliably measuring that. We did have data about a few other variables that were important to compare, including assets, giving, geographic location, etc. The main difference we saw was that foundations that have used one of CEP’s assessments (such as our Grantee Perception Report) in the past were more likely to respond to the survey. This is something we find in most of our survey samples. It doesn’t mean that foundations that haven’t used our assessments aren’t responding, but they are doing so at a lower rate. It could indicate, though, that foundations interested in gathering feedback on their performance were more likely to respond. We have more information about what we tested for response bias on page 45 of the report. 

JC:  I found it a little troubling that only 45% of CEOs of independent foundations view the general public as a relevant stakeholder group for their transparency efforts, yet the premise of philanthropy is that it is dedicated to serving the public good.  Did you also find this surprising? And any thoughts on the disconnect there?  

CEP_Foundation-Transparency_coverEB:  I did not find that surprising, and I’m not sure our data indicates that there is a disconnect between how foundations are thinking about certain aspects of transparency and serving the public good. If foundations are focused on being open with the nonprofits they fund and the nonprofits that may want funding from them in the future, that does seem like a pretty direct connection to serving the public good. After all, those are the organizations through which foundations are able to serve the public.

I think sometimes conversations about transparency suggest foundations should make sure they are sharing information with anyone and everyone. But that doesn’t seem like the most effective or efficient use of foundation resources. If people want to know what foundations are up to, most of the foundations of the size included in our study have websites or publicly available annual reports. Where I see real opportunity for foundations to do more is in sharing information about what does and doesn’t work in addressing the tough challenges they’re working to address. While that information itself may not be of interest to the general public, it can be applied in ways that benefit the general public.

JC:  Since the report points out that the philanthropy field is weak when it comes to sharing lessons learned and assessments of foundation performance, and since it also correlates stronger grantee-grantmaker relationships among foundations who have a tendency to be more transparent, will you be advocating that those who use your Grantee Perception Reports and other survey products share them?    Why or why not?

EB:  It’s up to foundations that use our Grantee Perception Report to decide whether to share their results publicly. Many, in fact, do, and almost all at least share a summary of what they learned. You can find on our website a list of those foundations that have made their GPRs public (scroll down on this page). I think it’s great when foundations are open in this way. But I don’t think that a foundation publicly sharing its GPR results is necessarily indicative of it doing more to respond to feedback or having strong relationships with its grantees.

JC:  Of the websites you examined, only 5% shared any information about lessons learned when things didn't go as planned.  Often this is because grantmakers fear harming the reputation of grantees or casting their work in a negative light.  Can you talk about how those grantmakers that were opening up this side of the work tackled that issue.

EB:  In the report, we share some examples of foundations being open about when things didn’t pan out as hoped. Those foundations do not name names of specific grantee organizations or tie results back to any individual organization. They seemed to share their lessons in a more general way, but still communicated enough specificity that others could learn from their experiences. I think their examples show that it’s possible to strike this balance.


JC:
 One of the struggles with the field and transparency is, of course, that there is no one-size-fits-all solution. However, once you start looking under the hood of foundation websites, patterns of emerging and best practices often surface.  Can you point to one or two transparency examples you uncovered that you wish others in the field would emulate?

EB:  Here is where we had a finding that did surprise me. I thought that perhaps the more information foundations shared on their websites, the more transparent they’d be seen to be by grantees. It turned out that was not borne out in the data. I think this is really important to consider: that the amount of information shared isn’t directly tied to perceptions of transparency. In my own experience, that makes sense. Sometimes, even when I know that a foundation has shared information about what it’s learned, I’ve had difficulty figuring out where to find that on a foundation’s website because there is so much other information on the site. I think what I’d suggest is that a focus be on how their websites can most effectively be used as a tool for sharing information that matters.  

 JC:  The last time CEP issued a report on transparency, it led to changes in the kinds of questions you include in your Grantee Perception Survey, which now includes questions specific to assessing perceptions about foundation transparency.  How will what you learned from this report impact your own work in the future? 

EB:  This research has given us a better understanding of how foundation CEOs, themselves, are thinking about transparency. It turns out there is a lot of agreement about what transparency means, so this research really validates the importance of the questions we added to our grantee survey a few years back. Transparency, especially about the substance of foundations’ work, is considered crucial by both grantees and foundation CEOs. Foundations and grantees are more aligned than they may realize when it comes to the information they think is important for foundations to share. Now it’s about foundations implementing — and really doing it well. Our research suggests they are doing well in some areas but not in others. We will build off of the findings in this study as we continue our research on other related topics. For example, we recently fielded a survey on evaluation practices at foundations, in partnership with the Center for Evaluation Innovation, and are seeing findings in that study that further build upon what we published in this report.

Grantmaker Transparency: The Dawn of a New Age in Philanthropy
November 16, 2015

(Aaron Lester is demand generation manager at Fluxx.  This blog post first ran in PhilanTopic.)

Aaron_lester_for_PhilanTopic"People tend to be private about love and money, and in philanthropy, it's both," says Janet Camarena, director of transparency initiatives at Foundation Center.

It's only natural that, traditionally, philanthropy has unfolded behind closed doors. On the one hand, the freedom to make personal funding choices gives grantmakers the ability to stay above the fray, uninfluenced by both market and political pressures. On the other hand, it doesn't allow the public to understand, learn from, or think critically about philanthropy.

"Giving and charitable acts are such private, emotional transactions," says Suki O'Kane, director of administration at the Walter and Elise Haas Fund. "How do you come from such strong traditions of privacy and intimacy, and bring that out into the open?"

Where do things stand?

Indeed ­– how do we as a sector make the switch from a traditionally opaque business model to an enterprise that embraces more transparency? It all comes down to the following questions: What am I funding? Why am I funding what I'm funding? Is my funding making an impact? And perhaps most importantly, how do we improve?

How do we as a (philanthropic) sector make the switch from a traditionally opaque business model to an enterprise that embraces more transparency?

There is good news: transparency in philanthropy is happening, there's no denying it. In fact, it's well under way, with large foundations like Gates, Ford, and Getty, sharing their endeavors with the public, surveying their grantees (and sharing the results), and creating searchable grants databases. Still, transparency can be difficult.

As a grantmaker, you know that sometimes your investments fail, sometimes grantees don't perform the way you expected, and sometimes, despite your best intentions, you can't pull off a new initiative or program. "Philanthropy isn't venture capital," says Christine Maulhardt, director of communications and public affairs at the Blue Shield of California Foundation. "Big losses aren't typical in our sector. We want everything to work out perfectly."

Regardless of the perceived risks, transparency in philanthropy is here to stay. And yes, it can be scary and hard to figure out how to get started. But the rewards for embracing transparency far outweigh the risk of turning your back on it.

Time for Transparency ImageWhere are we headed?

 

As we look to the (not so distant) future, we're particularly excited about the potential for grantmakers and grantees alike to have the ability to track incoming evaluation data, to understand in real time their organization's short- and long-term impact, and to be able to respond to that data and take action to ensure continued progress.

In the past, there was no common language used to talk about impact evaluation. Now, for the first time, technology can help create that common language. It is possible for foundations to not only track their own progress toward a goal, but also to compare results with other groups working toward the same end. The intelligence learned creates a greater potential for real needle-moving impact.

Becoming Transparent: Best Practices

If your foundation is just beginning the journey toward greater transparency, Camarena has suggestions for working in league with your peers. First, there's no need to be revolutionary. "Rather than creating something custom for your foundation, really look across the field to some standard practices," she says. "When it comes to creating the application process, look at grants management systems that exist already, and look at taxonomy so that you're not inventing a language that won't make sense field-wide." Her key takeaways:

  • Look to other foundations for standard practices on transparency; don't reinvent the wheel
  • Take advantage of modern grants management systems to help guide your application process and to create a common taxonomy.
  • Join a regional association of grantmakers so you can network with your peers and share ideas, successes, failures, and best practices. If you're using a grantmaking solution, join the community of users.
  • Participate in field-wide movements like the Who Has Glasspockets initiative and Foundation Center's Get on the Map campaign.

As daunting as it may be to open your foundation's doors to the public, transparency has far more benefits than drawbacks. Not only will you be moving in step with a growing movement, you'll also be in great company. It's time we started to share the why and how of our giving. All of us stand to benefit.

--Aaron Lester

Last Call for Glasspockets Survey Feedback
September 16, 2015

Do you have a moment to help us make Glasspockets better? We are about to close our feedback period for a short survey to assess ways to improve Foundation Center’s Glasspockets web site, services, and social media presence. We invite you to contribute to our thinking around new directions for our work, and how we can improve our web site and social media engagement to better engage and inform our audiences toward the goal of encouraging greater foundation transparency.

You can access the survey here until September 25th. We look forward to your feedback!

Participate in the 2015 Glasspockets User Survey
June 11, 2015

Do you have a moment to help us make Glasspockets better? We are conducting a short survey to assess ways to improve Foundation Center’s Glasspockets web site, services, and social media presence. As a Glasspockets community member, we invite you to contribute to our thinking around new directions for our work, and how we can improve our web site and social media engagement to better engage and inform our audiences toward the goal of encouraging greater foundation transparency.

You can access the survey here. We look forward to your feedback. 

Share This Blog

  • Share This

Subscribe to Transparency Talk

  • Enter your email address:

About Transparency Talk

  • Transparency Talk, the Glasspockets blog, is a platform for candid and constructive conversation about foundation transparency and accountability. In this space, Foundation Center highlights strategies, findings, and best practices on the web and in foundations–illuminating the importance of having "glass pockets."

    The views expressed in this blog do not necessarily reflect the views of the Foundation Center.

    Questions and comments may be
    directed to:

    Janet Camarena
    Director, Transparency Initiatives
    Foundation Center

    If you are interested in being a
    guest contributor, contact:
    glasspockets@foundationcenter.org

Categories