Transparency Talk

Category: "Feedback" (58 posts)

How "Going Public" Improves Evaluations
October 17, 2017

Edward Pauly is director of research and evaluation at The Wallace Foundation. This post is part of the Glasspockets #OpenForGood series in partnership with the Fund for Shared Insight. The series explores new research and tools, promising practices, and inspiring examples showing how some foundations are opening up the knowledge that they are learning for the benefit of the larger philanthropic sector. Contribute your comments on each post and share the series using #OpenForGood.

ED_finalAs foundations strive to be #OpenForGood and share key lessons from their grantees' work, a frequent question that arises is how foundations can balance the value of openness with concerns about potential risks.

Concerns about risk are particularly charged when it comes to evaluations. Those concerns include: possible reputational damage to grantees from a critical or less-than-positive evaluation; internal foundation staff disagreements with evaluators about the accomplishments and challenges of grantees they know well; and evaluators’ delays and complicated interpretations.

It therefore may seem counterintuitive to embrace – as The Wallace Foundation has – the idea of making evaluations public and distributing them widely. And one of the key reasons may be surprising: To get better and more useful evaluations.

The Wallace Foundation has found that high-quality evaluations – by which we mean independent, commissioned research that tackles questions that are important to the field – are often a powerful tool for improving policy and practice. We have also found that evaluations are notably improved in quality and utility by being publicly distributed.

Incentives for High Quality

A key reason is that the incentives of a public report for the author are aligned with quality in several ways:

  • Evaluation research teams know that when their reports are public and widely distributed, they will be closely scrutinized and their reputation is on the line. Therefore, they do their highest quality work when it’s public.  In our experience, non-public reports are more likely than public reports to be weak in data use, loose in their analysis, and even a bit sloppy in their writing.  It is also noteworthy that some of the best evaluation teams insist on publishing their reports.
  • Evaluators also recognize that they benefit from the visibility of their public reports because visibility brings them more research opportunities – but only if their work is excellent, accessible and useful.
  • We see evaluators perk up when they focus on the audience their reports will reach. Gathering data and writing for a broad audience of practitioners and policymakers incentivizes evaluators to seek out and carefully consider the concerns of the audience: What information does the audience need in order to judge the value of the project being evaluated? What evidence will the intended audience find useful? How should the evaluation report be written so it will be accessible to the audience?

Making evaluations public is a classic case of a virtuous circle: public scrutiny creates incentives for high quality, accessibility and utility; high quality reports lead to expanded, engaged audiences – and the circle turns again, as large audiences use evaluation lessons to strengthen their own work, and demand more high-quality evaluations. To achieve these benefits, it’s obviously essential for grantmakers to communicate upfront and thoroughly with grantees about the goals of a public evaluation report -- goals of sharing lessons that can benefit the entire field, presented in a way that avoids any hint of punitive or harsh messaging.

“What is it that you don’t know, that if you knew it, would enable you to make important progress in your own work?”

Asking the Right Questions

A key difference between evaluations commissioned for internal use and evaluations designed to produce public reports for a broad audience lies in the questions they ask. Of course, for any evaluation or applied research project, a crucial precursor to success is getting the questions right. In many cases, internally-focused evaluations quite reasonably ask questions about the lessons for the foundation as a grantmaker. Evaluations for a broad audience of practitioners and policymakers, including the grantees themselves, typically ask a broader set of questions, often emphasizing lessons for the field on how an innovative program can be successfully implemented, what outcomes are likely, and what policies are likely to be supportive.

In shaping these efforts at Wallace as part of the overall design of initiatives, we have found that one of the most valuable initial steps is to ask field leaders: What is it that you don’t know, that if you knew it, would enable you to make important progress in your own work? This kind of listening can help a foundation get the questions right for an evaluation whose findings will be valued, and used, by field leaders and practitioners.

Knowledge at Work

For example, school district leaders interested in Wallace-supported “principal pipelines” that could help ensure a reliable supply of effective principals, wanted to know the costs of starting such pipelines and maintaining them over time. The result was a widely-used RAND report that we commissioned, “What It Takes to Operate and Maintain Principal Pipelines: Costs and Other Resources.” RAND found that costs are less than one half of 1% of districts’ expenditures; the report also explained what drives costs, and provided a very practical checklist of the components of a pipeline that readers can customize and adapt to meet their local needs.

Other examples that show how high-quality public evaluations can help grantees and the field include:

Being #OpenForGood does not happen overnight, and managing an evaluation planned for wide public distribution isn’t easy. The challenges start with getting the question right – and then selecting a high-performing evaluation team; allocating adequate resources for the evaluation; connecting the evaluators with grantees and obtaining relevant data; managing the inevitable and unpredictable bumps in the road; reviewing the draft report for accuracy and tone; allowing time for grantees to fact-check it; and preparing with grantees and the research team for the public release. Difficulties, like rocks on a path, crop up in each stage in the journey. Wallace has encountered all of these difficulties, and we don’t always navigate them successfully. (Delays are a persistent issue for us.)

Since we believe that the knowledge we produce is a public good, it follows that the payoff of publishing useful evaluation reports is worth it. Interest from the field is evidenced by 750,000 downloads last year from www.wallacefoundation.org, and a highly engaged public discourse about what works, what doesn’t, why, and how – rather than the silence that often greets many internally-focused evaluations.

--Edward Pauly

No Moat Philanthropy Part 5: The Downsides & Why It’s Worth It
October 6, 2017

Jen Ford Reedy is President of the Bush Foundation. On the occasion of her fifth anniversary leading the foundation, she reflects on efforts undertaken to make the Bush Foundation more permeable. Because the strategies and tactics she shares can be inspiring and helpful for any grantmaker exploring ways to open up their grantmaking, we have devoted this blog space all week to the series. This is the final post in the five-part series.

Reedyjenniferford-croppedEverything we do is a trade-off. Spending time and money on the activities described in this No Moat Philanthropy series means time and money not invested in something else. Here are some of the downsides of the trade-offs we have made:

It takes some operating expense.  It requires real staff time for us to do office hours in western North Dakota and to reformat grant reports to be shared online and to do every other activity described in these posts. We believe there is lots of opportunity to advance our mission in the “how” of grantmaking and weigh that as an investment alongside others. In our case, we did not have an increase in staff costs or operating expenses as we made this shift. We just reprioritized.

It can be bureaucratic.  Having open programs and having community members involved in processes requires some structure and rules and standardization in a way that can feel stifling. Philanthropy feels more artful and inspired when you can be creative and move quickly. To be equitably accessible and to improve the chance we are funding the best idea, we are committed to making this trade-off. (While, of course, being as artful and creative as possible within the structures we set!)

“We believe our effectiveness is fundamentally tied to our ability to influence and be influenced by others.”

Lots of applications means lots of turndowns.  Conventional wisdom in philanthropy is to try to limit unsuccessful applications – reducing the amount of effort nonprofits invest with no return. This is an important consideration and it is why many foundations have very narrow guidelines and/or don’t accept unsolicited proposals. The flip side, however, is that the more we all narrow our funding apertures, the harder it is for organizations to get great ideas funded. We’ve decided to run counter to conventional wisdom and give lots of organizations a shot at funding. Of course, we don’t want to waste their time. We have three strategies to try to mitigate this waste: (1) through our hotlines we try to coach unlikely grantees out of the process. (In our experience, nonprofits will often apply anyway – which suggests to us that they value having a shot – even if the odds are long.); (2) we try to make the process worth it. Our surveys suggest that applicants who do the programs with the biggest pools get something out of the process – (and we learn from the applicants even if they are not funded.); and (3) we try to make the first stage of our processes as simple as possible so folks are not wasting too much effort.

Relationships are hard!  Thinking of ourselves as being in relationship with people in the region is not simple. There are lots of them! And it can be super frustrating if a Bush staff member gives advice on a hotline that seems to be contradicted by the feedback when an application is declined. We’ve had to invest money and time in developing our CRM capacity and habits. We have a lot more work to do on this front. We will never not have a lot more work to do on our intercultural competence and our efforts to practice inclusion. Truly including people with different perspectives can make decisions harder as it makes decisions better.  The early returns on our efforts have been encouraging and we are committed to continuing the work to be more fully in relationship with more people in the communities we serve.

Conclusion

Overall, we believe a No Moat Philanthropy approach has made us more effective. When we are intentional about having impact through how we do our work — building relationships, inspiring action, spreading optimism — then we increase the positive impact we have in the region.

We believe our effectiveness is fundamentally tied to our ability to influence and be influenced by others, which demands trust, reciprocity and a genuine openness to the ideas of others. It requires understanding perspectives other than our own. It requires permeability.

While we arrived at this approach largely because of our place-based sensibility and strategic orientation toward people (see learning paper: “The Bush Approach”), the same principles can apply to a national or international foundation focused on particular issues. The definition of community is different, but the potential value of permeability within that community is the same.

--Jen Ford Reedy

No Moat Philanthropy Part 4: Beyond the Transactional
October 5, 2017

Jen Ford Reedy is President of the Bush Foundation. On the occasion of her fifth anniversary leading the foundation, she reflects on efforts undertaken to make the Bush Foundation more permeable. Because the strategies and tactics she shares can be inspiring and helpful for any grantmaker exploring ways to open up their grantmaking, we are devoting our blog space all week to the series. This is the fourth post in the five-part series.

Reedyjenniferford-croppedWe have a grantmaking model that is based on the belief that, if we do it right, we will create more good by what we inspire than by what we directly fund. Principle #4 and #5 of No Moat Philanthropy are directly related to this, how connecting and sharing with others can advance your foundation’s mission.

Principle #4: Value every interaction as an opportunity to advance your mission

Our tagline and our strategy are one and the same: We invest in great ideas and the people who power them. We know that the only way anything happens is through people. Any place or field, therefore, is limited by the ambitions and the skills of the people in it.

The Bush Fellowship has been a flagship program of the Foundation for decades. We hear repeatedly from Bush Fellows that the experience changed what they thought was possible in their life and career. With the Bush Fellows program as our source code, we’ve been working for the past five years to ensure that all of our programs have the same effect. How can we encourage people to think bigger and think differently? How can we be a force for optimism?

This notion of a foundation being a force for optimism is not an obvious one. After all, we mostly tell people no. Last year, 95 percent of people who applied for the Bush Fellowship did not receive one. We’ve worked diligently to make sure all applicant interactions with us are helpful and encouraging, regardless of grant or fellowship outcome. And our surveys suggest the work is paying off. For example, 79 percent of declined Bush Fellowship applicants said the process increased their beliefs that they can accomplish “a lot.”

“If we do grantmaking right, we will create more good by what we inspire than by what we directly fund.”

To have this impact with each applicant, we:

Operate hotlines to speak with Bush staff. For our open programs, we have established hotlines for potential applicants. We will speak with people as many times as they desire to provide coaching on their idea or proposal. For applicants, this is a way to clearly understand what we are looking for and to vet ideas with us. For Bush staff, this is a way to provide coaching and encouragement to strengthen proposals and to influence activities beyond those we fund.

Give feedback about declined applications. We offer feedback to declined applicants for our major grant and fellowship programs because we see this as another valuable opportunity to provide coaching and encouragement. We have also witnessed applicants using the feedback to improve their plans and proposals, which benefits both them and us. This two-way dialogue also allows applicants to share how we can improve the process for them.

Find ways to support declined applicants. In the course of our processes, we learn about far more amazing people and organizations than we can actually fund. Therefore, we try to find ways to be useful to more than just the limited number of accepted applicants. For example, we consider declined Bush Fellowship finalists to be part of our “Bush Network” and invite them to bushCONNECT. We also provide declined Bush Prize finalists with a $10,000 grant. In our hiring process, we offer unsuccessful finalists the chance to meet with our hiring consultant for an individual coaching session. In addition, across all our programs and operations, we try to craft our applications and our processes so that the experience of applying adds value to an applicant’s thinking and planning.

Every interaction is an opportunity to influence and be influenced.  Every interaction is an opportunity for shared learning. And that brings me to our fifth and final principle…

Bush-altlogo-color Principle #5: Share as you go.

In the past five years, we’ve been working to get more of what we are thinking — and learning — out to the community. This has required adjusting our standards and prioritizing just getting something out, even if it is not glossy and beautiful. It has required a new, shared understanding with grantees and Fellows that their reports and reflections will be public, so as many people as possible can benefit from their experience. It has required designing our internal work — like strategy documents for the Board — with external audiences in mind so they are ready to share.

We believe that if we do it right, we can have as much and potentially more impact from sharing the stories and spreading the lessons from our grantees and Fellows as from the investments themselves. This belief is at the heart of all our communications (see learning paper: “Communications as Program”) and is also reinforced with specific tactics such as:

“We potentially have more impact from sharing the stories and spreading the lessons from our grantees and Fellows.”

Post grantee reports on our website. We introduced “Learning Logs” to make grant reports public, and we hope, to give them life and utility beyond our walls. We refer prospective applicants to relevant learning logs as they craft their proposals, and we hear from applicants that they have indeed learned from them. Grantees and Fellows also share that they read one another’s Learning Logs as a way to get new ideas for overcoming barriers.

Share lessons along the way. We are publishing learning papers (like this one) as we believe we have something useful to share. We intended this to lower the bar of who, when and how we share. Our learning papers are not beautiful. Most of them are not based on statistically significant evaluation methodologies. They simply document a staff effort to process something we are working on and to share our reflections.

Tie evaluation to audience analysis. We invest heavily in external evaluations of our work, but in doing so we have found that the end-product is often only useful to our staff and key stakeholders. Consequently, we introduced a different approach to thinking about evaluation with a sharing mindset. We use a framework to identify the audiences who might care about or benefit from the lessons of an evaluation, what questions are relevant to each group, and what form or output would be most useful to them.

Webinar to the max. Webinars are not a particularly novel activity; however, we view them as a core tool of permeability. We host a webinar at the beginning of every application period for Grant and Fellowship programs to explain the process and what we are looking for. We also host them when we have a job opening to discuss the role and what it is like to work here. We host them annually for our Foundation initiatives to explain what we are up to and where we are headed. Most webinars feature a staff presentation followed by an open Q&A with videos archived on our website for anyone who missed it.

If you’ve been reading this series all week, you might be wondering when I’m going to get to the downsides of No Moat Philanthropy. All new approaches have their pain points.  So, come back tomorrow and I’ll share our pain and why we believe it is worth it.

--Jen Ford Reedy

No Moat Philanthropy Part 3: Building Your Network
October 4, 2017

Jen Ford Reedy is President of the Bush Foundation. On the occasion of her fifth anniversary leading the foundation, she reflects on efforts undertaken to make the Bush Foundation more permeable. Because the strategies and tactics she shares can be inspiring and helpful for any grantmaker exploring ways to open up their grantmaking, we are devoting our blog space all week to the series. This is the third post in the five-part series.

Reedyjenniferford-croppedIn yesterday’s post I shared how we have tried to bring different perspectives into the Foundation.  Today’s post is mostly about getting out of the Foundation, to meet new people.  This is the third principle of No Moat Philanthropy.

Principle #3: Continuously and intentionally connect with new people

Five years ago we had close working relationships with people in each of our initiative areas. While we valued those relationships, we kept a pretty tight circle. We knew people wanted money from us, and we also knew their chances of receiving it were slim. This can be awkward and who wants that? While avoiding awkwardness can make life more pleasant, we now believe embracing that awkwardness actually makes us smarter. While we can only fund a limited number of people and organizations, interacting with lots and lots of people and organizations helps us better understand our region and make better, more informed strategic choices and funding decisions.

We believe in the power of networks. We believe that a community’s strength and diversity of connections help define its capacity for resilience and innovation. We work to ensure we are continuously connecting with new and different people. Each year, we set outreach priorities for geographic areas, cultural communities and/or sectors based on our analysis of where our network is weakest. Then we strive to make new connections in a way that creates connections between others, too. Specifically we:

“We believe that a community’s strength and diversity of connections help define its capacity for resilience and innovation.”

Hold office hours to meet with people all around the region. We hold “office hours” in communities around the region for anyone interested in with our Foundation staff. These are sometimes coupled with a listening session, co-hosted with a local partner, that allow us to understand what issues are most important to the community.

Sponsor and attend other people’s events. We introduced an open process to request Bush Foundation sponsorship of events. We had been sponsoring some events, but we never considered it a program strategy. One of the primary criteria for event sponsorship is whether it will help us connect with people who might benefit from learning about our work. This might include having a Bush Foundation booth manned by staff members who are there to meet and field questions from attendees.

Host events designed for connection. We were already hosting a number of events to build relationships with and among our Fellows and grantees. In the past five years, however, we have taken our events strategy to a higher level by focusing on connecting people across our programs with people beyond our existing grantee and Fellowship networks. The best example of this is bushCONNECT, our flagship event which brings together 1,100 leaders from the region. To ensure we are attracting individuals beyond our community network, we engage “recruitment partners” from around the region who receive grant support to recruit a cohort from within their network to bring to the event, thereby ensuring bushCONNECT attendees more fully represent the geography — and diversity — of our region.

Take cohorts of people to national events. We also offer scholarships for cohorts of people from our region to attend national conferences together. During the event, we create opportunities to build connections with and among the attendees from the region. This allows us to meet and support more people in the region, build attendees’ individual networks, and ensure leaders in our region are both contributing to and benefitting from national conversations.

We are not throwing parties for fun.  We see relationship building as core to our strategy.  We see every interaction as an opportunity to influence and be influenced.  More on that tomorrow.

--Jen Ford Reedy

No Moat Philanthropy Part 2: Bringing the Outside In  
October 3, 2017

Jen Ford Reedy is President of the Bush Foundation. On the occasion of her fifth anniversary leading the foundation, she reflects on efforts undertaken to make the Bush Foundation more permeable. Because the strategies and tactics she shares can be inspiring and helpful for any grantmaker exploring ways to open up their grantmaking, we are devoting our blog space all week to the series. This is the second post in the five-part series.

Reedyjenniferford-croppedIn the past five years, we have been working hard at the Bush Foundation to go beyond transparency to permeability.  In yesterday’s post, I described our work to ensure we are sourcing and supporting the best ideas from the communities we serve.  Today I’ll share Principle #2 of No Moat Philanthropy and the ways we are working to ensure our work is informed by and improved by many different perspectives.

Principle #2: Bring lots of perspectives into your strategy design and decision-making

Five years ago, it said on our website to let us know if you had a good idea that aligned with our strategy, but there was no clear way for people to do so. We had formal advisors on our initiatives and advisors on the ground in North Dakota and South Dakota, whom we kept secret. While we talked a lot about co-creation, in practice it was hard to develop and nurture true partnerships with organizations given our very specific agenda and the power dynamics in our funding relationships.

Now, to ensure we are being shaped by more people and more perspectives, we:

Create learning cohorts to shape our strategies. In the early stages of learning about a new area, we invite others to learn with us and shape our thinking. For example, to inform how we should best integrate the arts into our work, we created the Community Creativity Cohort, providing organizations a $100,000 operating grant and asking them to attend a few workshops and provide counsel (see learning paper “Lessons from the Community Creativity Cohort”). As we develop our strategy to support individualizing education, we have an advisory group of regional experts in Native education to help us understand how our strategy can be most relevant in Native communities.

Bush-altlogo-colorUse external selectors. Historically, we have engaged community leaders to select Bush Fellows, and we have expanded that practice to other programs, including the use of state-level advisory committees to select Bush Prize winners and working through state-level intermediaries to promote and award our smaller-scale Community Innovation grants.

Hire staff and recruit board members differently. We have made numerous changes to our staff hiring and board recruitment practices to attract people from a variety of backgrounds with different perspectives. For staffing, we have always hired from a variety of professional backgrounds, and while we continue to do so, we are also now focusing on diversity in race and cultural background. In three years, we increased the percentage of Bush staff who identify as people of color from a 14 percent to 50 percent. From a Board perspective, we set (and met) specific five-year targets for Board composition, focusing primarily on geographic and racial diversity.

Host internal Fellows. In addition to staff diversification, we created an internal Fellows program designed to routinely bring new and different perspectives into our work. The Ron McKinley Philanthropy Fellows program, now operated by the Minnesota Council on Foundations, hires professionals from communities underrepresented in foundation leadership to work at foundations for three years. We have hosted 11 Fellows and hired four of them into permanent positions.

Build staff and board intercultural competence. If we really want to understand perspectives other than our own, we need the skills to do so. We invest heavily in staff development and board learning and have placed particular emphasis on intercultural competence. We hope that training, combined with out-of-the-office interactions and learning experiences, make us more aware of our biases, better able to truly understand the lived experiences of others, more effective in truly engaging and working with people across the region, and better able to make smart decisions and design strategies that actually work.

In combination, these tactics have given us more opportunity to hear different perspectives in our work and better skills to truly understand different perspectives in our work. Tomorrow I will share Principle #3, how to continuously and intentionally connect with new people.

--Jen Ford Reedy

What Will You #OpenForGood?
July 13, 2017

Janet Camarena is director of transparency initiatives at Foundation Center.  This post is part of the Glasspockets’ #OpenForGood series in partnership with the Fund for Shared Insight. The series explores new tools, promising practices, and inspiring examples showing how some foundations are opening up the knowledge that they are learning for the benefit of the larger philanthropic sector. Contribute your comments on each post and share the series using #OpenForGood.

Janet Camarena Photo

This week, Foundation Center is launching our new #OpenForGood campaign, designed to encourage better knowledge sharing practices among foundations.  Three Foundation Center services—Glasspockets, IssueLab, and GrantCraft are leveraging their platforms to advance the idea that philanthropy can best live up to its promise of serving the public good by openly and consistently sharing what it’s learning from its work.  Glasspockets is featuring advice and insights from “knowledge sharing champions” in philanthropy on an ongoing #OpenForGood blog series; IssueLab has launched a special Results platform allowing users to learn from a collective knowledge base of foundation evaluations; and a forthcoming GrantCraft Guide on open knowledge practices is in development.

Although this campaign is focused on helping and inspiring foundations to use new and emerging technologies to better collectively learn, it is also in some ways rooted in the history that is Foundation Center’s origin story.

OFG-twitter

A Short History

Sixty years ago, Foundation Center was established to provide transparency for a field in jeopardy of losing its philanthropic freedom due to McCarthy Era accusations that gained traction in the absence of any openness whatsoever about foundation priorities, activities, or processes.  Not one, but two congressional commissions were formed to investigate foundations committing alleged “un-American activities.”  As a result of these congressional inquiries, which spanned several years during the 1950s, Foundation Center was established to provide transparency in a field that had nearly lost everything due to its opacity. 

“The solution and call to action here is actually a simple one – if you learn something, share something.”

I know our Transparency Talk audience is most likely familiar with this story since the Glasspockets name stems from this history when Carnegie Corporation Chair Russell Leffingwell said, “The foundation should have glass pockets…” during his congressional testimony, describing a vision for a field that would be so open as to allow anyone to have a look inside the workings and activities of philanthropy.  But it seems important to repeat that story now in the context of new technologies that can facilitate greater openness.

Working Collectively Smarter

Now that we live in a time when most of us walk around with literal glass in our pockets, and use these devices to connect us to the outside world, it is surprising that only 10% of foundations have a website, which means 90% of the field is missing discovery from the outside world.  But having websites would really just bring foundations into the latter days of the 20th century--#OpenForGood aims to bring them into the present day by encouraging foundations to openly share their knowledge in the name of working collectively smarter.

What if you could know what others know, rather than constantly replicating experiments and pilots that have already been tried and tested elsewhere?  Sadly, the common practice of foundations keeping knowledge in large file cabinets or hard drives only a few can access means that there are no such shortcuts. The solution and call to action here is actually a simple one—if you learn something, share something

In foundations, learning typically takes the form of evaluation and monitoring, so we are specifically asking foundations to upload all of your published reports from 2015 and 2016 to the new IssueLab: Results platform, so that anyone can build on the lessons you’ve learned, whether inside or outside of your networks. Foundations that upload their published evaluations will receive an #OpenForGood badge to demonstrate their commitment to creating a community of shared learning.

Calls to Action

But #OpenForGood foundations don’t just share evaluations, they also:

  • Open themselves to ideas and lessons learned by others by searching shared repositories, like those at IssueLab as part of their own research process;
  • They use Glasspockets to compare their foundation's transparency practices to their peers, add their profile, and help encourage openness by sharing their experiences and experiments with transparency here on Transparency Talk;
  • They use GrantCraft to hear what their colleagues have to say, then add their voice to the conversation. If they have an insight, they share it!

Share Your Photos

“#OpenForGood foundations share their images with us so we can show the collective power of philanthropic openness, not just in words, but images. ”

And finally, #OpenForGood foundations share their images with us so we can show the collective power of philanthropic openness, not just in words, but images.  We would like to evolve the #OpenForGood campaign over time to become a powerful and meaningful way for foundations to open up your work and impact a broader audience than you could reach on your own. Any campaign about openness and transparency should, after all, use real images rather than staged or stock photography. 

So, we invite you to share any high resolution photographs that feature the various dimensions of your foundation's work.  Ideally, we would like to capture images of the good you are doing out in the world, outside of the four walls of your foundation, and of course, we would give appropriate credit to participating foundations and your photographers.  The kinds of images we are seeking include people collaborating in teams, open landscapes, and images that convey the story of your work and who benefits. Let us know if you have images to share that may now benefit from this extended reach and openness framing by contacting openforgood@foundationcenter.org.

What will you #OpenForGood?

--Janet Camarena

GrantAdvisor: A TripAdvisor for Funder Feedback
July 6, 2017

Michelle Greanias is executive director of PEAK Grantmaking. Follow her on Twitter @mgreanias. This post also appears in PEAK Grantmaking’s blog.

Michelle GreaniasFor funders, hearing honest input from grantseekers about what they think about a foundation’s practices and getting insights from their experiences working as a grantee partner is a critical component of effective grantmaking. Up until now, funders have needed to initiate the request for feedback via surveys, conversations, and third-party evaluators.  Now, a collaboration of funders, nonprofits, and others interested in improving philanthropy are exploring a new approach—GrantAdvisor, which recently launched in California and Minnesota with a goal of eventually reaching the entire country.

GrantAdvisor is like TripAdvisor—it’s a website that allows individuals (in this case, grant applicants, grantees, and others) to share their first-hand experiences with funding organizations, and for funders to have the opportunity to respond publicly.  The idea is that just as a traveler would check TripAdvisor when planning a trip, a nonprofit would check GrantAdvisor before applying to a funder. And, just as a hotel monitors TripAdvisor to see what your customers like best and least about them, funders can see how grantees and colleagues are experiencing working with them.

“Listening to unfettered feedback from grantees can help funders build more efficient processes and more effective partnerships, which ultimately increases impact.”

It works by collecting anonymous feedback from grantseekers and grantees. When five reviews have been submitted, the data will be shared publicly. A funder profile needs at least five reviews before it becomes public. The unpublished results are sent to the funder providing an opportunity for the funder to respond. After the first five reviews are published, subsequent reviews will be posted, and the funder can respond at any time. Funders are encouraged to register with GrantAdvisor to receive automatic notices when reviews are posted about their organizations and post responses when new reviews are submitted.

As a grants manager, this concept was a little scary to me at first—what if the feedback isn’t all positive?  How would it affect an organization’s reputation?  But the reality is that an organization’s reputation is already affected if grantseekers are having poor experiences with a funder. I want to know, and I believe most grants managers would agree, about any issues and be able to address them.  Especially since the alternative is allowing problems to build and multiply as bad practice impacts more and more grantees.

I also considered this transparent move through another critical lens—aligning values with practices.  In PEAK Grantmaking’s recent research, the top three common values held by grantmakers were collaboration, respect, and integrity.  Being open to feedback, even difficult feedback, is a concrete way to show that grantmakers are “walking the talk” by bringing those values to life through our practices.

Jessamyn Shams-Lau, executive director of Peery Foundation, and Maya Winkelstein, executive director of the Open Road Alliance, both support this work and see four reasons that GrantAdvisor.org is useful to funders:

  1. Feedback: Listening to unfettered feedback from grantees can help funders build more efficient processes and more effective partnerships, which ultimately increases impact.
  2. Benchmarking: With a common set of questions for every foundation, funders can benchmark the effectiveness of their grantmaking practices from the perspective of the grantee experience.
  3. Honest and Accurate Data. When foundations directly solicit feedback (even anonymously), respondents give different answers. Since GrantAdvisor.org collects reviews with or without funder prompting, this unsolicited feedback is the most honest feedback and honest reviews mean accurate data.
  4. Saving Time. Over time, the hope is that the sharing of information via GrantAdvisor.org will help potential grantees better self-select which foundations to approach and which are not well aligned. This will result in a higher-quality pipeline for foundations, which saves everyone time and gets funders closer to impact faster.

Given the promise and potential of this new feedback mechanism to strengthen grantmaking practice, I am honored to serve on the GrantAdvisor national advisory committee. I will share more information about this effort as it progresses and look forward to hearing from the profession about this tool, particularly those in California and Minnesota, where GrantAdvisor will be initially active.

--Michelle Greanias

Why Evaluations Are Worth Reading – or Not
June 14, 2017

Rebekah Levin is the Director of Evaluation and Learning for the Robert R. McCormick Foundation, guiding the Foundation in evaluating the impact of its philanthropic giving and its involvement in community issues. She is working both with the Foundation’s grantmaking programs, and also with the parks, gardens, and museums at Cantigny Park. This post is part of the Glasspockets’ #OpenForGood series done in partnership with the Fund for Shared Insight. The series explores new tools, promising practices, and inspiring examples showing how some foundations are opening up the knowledge that they are learning for the benefit of the larger philanthropic sector. Contribute your comments on each post and share the series using #OpenForGood. View more posts in the series.

Rebekah Levin photoTruth in lending statement:  I am an evaluator.  I believe strongly in the power of excellent evaluations to inform, guide, support and assess programs, strategies, initiatives, organizations and movements.  I have directed programs that were redesigned to increase their effectiveness, their cultural appropriateness and their impact based on evaluation data, helped to design and implement evaluation initiatives here at the foundation that changed the way that we understand and do our work, and have worked with many foundation colleagues and nonprofits to find ways to make evaluation serve their needs for understanding and improvement. 

“I believe strongly in the power of excellent evaluations."

One of the strongest examples that I’ve seen of excellent evaluation within philanthropy came with a child abuse prevention and treatment project.  Our foundation funded almost 30 organizations that were using 37 tools to measure treatment impact of treatment, many of which were culturally inappropriate, designed for initial screenings, or inappropriate for a host of other reasons, and staff from these organizations running similar programs had conflicting views about the tools.  Foundation program staff wanted to be able to compare program outcomes using uniform evaluation tools and to use that data to make funding, policy, and program recommendations, but they were at a loss as to how to do so in a way that honored the grantees’ knowledge and experience.   A new evaluation initiative was funded, combining the development of a "community of practice" for the nonprofits and foundation together to:

  • create a unified set of reporting tools;
  • learn together from the data about how to improve program design and implementation, and the systematic use of data to support staff/program effectiveness;
  • develop a new rubric which the foundation would use to assess programs and proposals; and
  • provide evaluation coaching for all organizations participating in the initiative.

The evaluation initiative was so successful that the nonprofits participating decided to continue their work together beyond the initial scope of the project to improve their own programs and better support the children and families that they are serving. This “Unified Project Outcomes” article describes the project and established processes in far greater detail.

But I have also seen and been a part of evaluations where:

  • the methodology was flawed or weak;
  • the input data were inaccurate and full of gaps;
  • there was limited understanding of the context of the organization;
  • there was no input from relevant participants; and
  • there was no thought to the use of the data/analysis;

so that little to no value came out of them, and the learning that took place as a result was equally inconsequential.

Mccormick-foundation-logo_2xSo now to those evaluation reports that often come at the end of a project or foundation initiative, and sometimes have interim and smaller versions throughout their life span.  Except to a program officer who has to report to their director about how a contract or foundation strategy was implemented, the changes from the plan that occurred, and the value or impact of an investment or initiative, should anyone bother reading them?  From my perch, the answer is a big “Maybe.”  What does it take for an evaluation report to be worth my time to read, given the stack of other things sitting here on my desk that I am trying to carve out time to read?  A lot.

  1. It has to be an evaluation and not a PR piece. Too often, "evaluation" reports provide a cleaned up version of what really occurred in a program, with none of the information about how and why an initiative or organization functioned as it did, and the data all point to its success.  This is not to say that initiatives/organizations can’t be successful.  But no project or organization works perfectly, and if I don’t see critical concerns/problems/caveats identified, my guess is that I’m not getting the whole story, and its value to me drops precipitously.
  2. It has to provide relevant context. To read an evaluation of a multi-organizational collaboration in Illinois without placing its fiscal challenges within the context of our state’s ongoing budget crisis, or to read about a university-sponsored community-based educational program without knowing the long history of mistrust between the school and the community, or any other of the relevant and critical contextual pieces that are effect a program, initiative or organization makes that evaluation of little value.  Placed within a nuanced set of circumstances significantly improves the possibility that the knowledge is transferable to other settings.
  3. It has to be clear and as detailed as possible about the populations that it is serving. Too often, I read evaluations that leave out critical information about who they were targeting and who participated or was served. 
  4. The evaluation’s methodology must be described with sufficient detail so that I have confidence that it used an appropriate and skillful approach to its design and implementation as well as the analysis of the data. I also pay great attention to what extent those who were the focus of the evaluation participated in the evaluation’s design, the questions being addressed, the methodology being used, and the analysis of the data.
  5. And finally, in order to get read, the evaluation has to be something I know exists, or something I can easily find. If it exists in a repository like IssueLab, my chances of finding it increase significantly.  After all, even if it’s good, it is even better if it is #OpenForGood for others, like me, to learn from it.

When these conditions are met, the answer to the question, “Are evaluations worth reading?” is an unequivocal “YES!,” if you value learning from others’ experiences and using that knowledge to inform and guide your own work.

--Rebekah Levin

From Good Idea to Problem Solved: Funding the Innovation Means Funding the Process
February 8, 2017

(Mandy Ellerton and Molly Matheson Gruen joined the [Archibald] Bush Foundation in 2011, where they created and now direct the Foundation's Community Innovation programs. The programs allow communities to develop and test new solutions to community challenges, using approaches that are collaborative and inclusive of people who are most directly affected by the problem.)

This post is part of the Funding Innovation series, produced by Foundation Center's Glasspockets and GrantCraft, and underwritten by the Vodafone Americas Foundation. The series explores funding practices and trends at the intersection of problem-solving, technology, and design. Please contribute your comments on each post and share the series using #fundinginnovation. View more posts in the series.

Mandy Ellerton

Molly Matheson Gruen

Good ideas for solving our toughest social problems come from a variety of places. But, we need more than just good ideas – we need transparent and thoughtful ways to get community buy-in and a wide variety of perspectives to make those ideas a reality.

For a cautionary case in point, take the origin story (later chronicled in the book The Prize) of the ill-fated attempt to transform the failing Newark public schools. A prominent governor, mayor and, later, an ultra-wealthy tech mogul, hatched the idea to radically transform the schools in the back of a chauffeured S.U.V. Commentary suggests that these leaders did not consult community stakeholders about the plan, only half-heartedly seeking community input much later in the process. As one community member put it to these leaders, "You have forced your plans on the Newark community, without the

measure of stakeholder input that anyone, lay or professional, would consider adequate or respectful." To some observers, it's no surprise that without initial community buy-in, nor a transparent process and over $100 million later, the plan ultimately crashed and burned.

But, let's not throw stones at glass houses. The Newark example is indicative of a larger pattern especially familiar to those of us in the field of philanthropy. We've learned that lesson the hard way, too. Many of us have been involved in (well-intentioned) backroom and ivory tower deals with prominent community leaders to magically fix community problems with some "good ideas." Sometimes, those ideas work. But a lot of times, they don't. And unfortunately, we often chalk these failures up to innovation simply being a risky endeavor, comparing our social innovation failure rates to the oft-discussed (maybe even enshrined?) business or entrepreneurship failure rates. What's more, we almost never actively, sincerely discuss and learn from these failed endeavors.

But social innovation failure often comes at a cost, leaving behind disillusioned community members, bad outcomes for some of our most vulnerable, and lots and lots of wasted dollars that could have gone to something better. Take the Newark example: the failed attempt to transform the schools created massive civic disruption, re-awakened historic hurts and injustice and will likely leave community members even more skeptical of any future efforts to improve the schools.

Through our work at the Bush Foundation, we've learned that truly good ideas–those that will really have a sustainable impact–are often created in deep partnership and trust between organizations, leaders, and–most critically–the people most affected by a problem.

But, that kind of deep community partnership and transparency takes a lot of work, time, and attention. And, most everything that takes a lot of work takes some funding.

Community-innovation

That's why we created our Community Innovation programs at the Bush Foundation in 2013: to fund and reward the process of innovation–the process of solving problems. While the emphasis in innovation funding is often on "early stage" organizations or projects, we joke that we are a "pre-early" funder or that we fund "civic R & D." We provide funding for organizations to figure out what problem to address in the first place, to get a better understanding of the problem, to generate ideas to solve the problem, and then, after all that work (and maybe having to revisit some of the earlier stages along the way), the organization might be ready to test or implement a good idea. See how we depict that "pre-early" problem solving process here.

Most importantly, throughout the innovation or problem-solving process, we also look for particular values to drive the organization's approach: Is the organization genuinely and deeply engaging the people most affected by the problem? Is the organization working in deep partnership with other organizations and leaders? Is the organization making the most of existing resources?

Let's bring it to life. Here are three examples of the 150+ organizations we've funded to engage in a process to solve problems in their communities:

  • World Wildlife Fund's Northern Great Plains initiative is bringing ranchers, conservationists, oil business developers, and government officials together to create a vision for the future of North Dakota's badlands and a shared energy development plan that protects this important landscape.
  • PACT for Families Collaborative engaged truant youth, their parents, education staff, and service providers to understand barriers to school attendance and redesign services and test strategies for positive, sustainable solutions to truancy in western Minnesota.
  • Pillsbury United Communities is using human-centered design processes to engage North Minneapolis residents to address their neighborhood's food desert and create North Market: a new grocery store managed in partnership with a local health clinic that will also be a clinic, pharmacy, and wellness education center.

"We've learned that truly good ideas–those that will really have a sustainable impact–are often created in deep partnership and trust between organizations, leaders, and...the people most affected by a problem."

Our grantees and partners are teaching us a lot about what it takes for communities to solve problems. One of the biggest things we've learned is that collaborative projects often take far more time than anyone initially expects, for a variety of reasons. Over the past few years nearly a third of our grantees have requested more time to complete their grants, which we have readily agreed to.

For example, the Northfield Promise Initiative is a highly-collaborative, cross-sector, community-wide effort to address education disparities in Northfield, Minnesota. The initiative utilizes action teams composed of diverse stakeholders to drive its work. Early on in the project they decided to stagger the rollout of the teams rather than launch them all at once. That allowed them to take more care in composing and launching each team and allowed interested stakeholders to engage in multiple teams. In addition, later teams could learn from the successes and challenges of the earlier ones. As the grantee put it, "Partners felt strongly that it is important to give the process this extra time to ensure that all the different community voices and insights have been included (thereby maintaining this as a community-owned initiative)." We gladly extended their grant term from two years to four years so that they could spend the time they believed necessary to lead the problem-solving effort thoughtfully and inclusively.

Bush-altlogo-colorFor more helpful examples, here are a couple of resources to explore:

  • One of our innovation programs is an award for organizations that have a track record of solving problems with their communities, called the Bush Prize for Community Innovation. Together with our evaluation partner Wilder Research, we created a report about some of our Bush Prize winners that digs into specific conditions, methods and techniques that appear to help organizations innovate.
  • We believe storytelling and transparency inspire innovation. Our grantees openly share what they're learning as they pursue solutions to community problems in grantee learning logs. The learning logs also include references to specific techniques and methods the organizations use to pursue innovation.

As funders, we also have a role in the innovation process that goes beyond writing the check. By virtue of our relationships and portfolios, we have a bird's eye view of the field. By opening up what we are learning, we hope to build trust with our stakeholders and help others build on our work, hopefully leading to more and better future innovations.

-- Mandy Ellerton and Molly Matheson Gruen

The Case for Opening Up Foundations Meetings to the Public
December 6, 2016

(Caroline Fiennes is Director of Giving Evidence, and author of It Ain't What You Give. She co-authored a recent report investigating the role open meetings play in increasing transparency. A version of this post was originally published on Giving Evidence, and has been reposted here with permission.)

Caroline FiennesAll charities and charitable foundations exist to serve the public good. Most of them are subsidized by the public through various tax breaks. Any publicly-listed company must have a meeting at least annually at which the directors are held accountable to the people whose capital they deploy. In over 15 years in this "industry," we’ve only encountered two charities/foundations in the UK which have meetings at which the public – or the intended beneficiaries – can know what goes on. The 800-year-old fund, City Bridge Trust in London, lets anybody observe its decision-making meetings, and Global Giving UK has an annual general meeting (AGM) at which anybody can ask anything. Why don’t more?

It’s hard to be accountable to people, or to hear from people, if they’re not in the room. So we wondered how many charities and foundations have public meetings.

Giving Evidence simply telephoned the 20 largest charities and foundations in each of the UK and the US and asked whether they ever have any meetings which are open to the public, and whether the public can ask questions. Of the 82 organizations we asked, only two have any meetings in public. None allows the public to ask questions.

Open-meetings-coverThis is about accountability and transparency to the people who provide subsidy and to the people the charities and foundations exist to serve.

Suppose that a nonprofit is treated poorly by a grantmaking organization. How can you tell the management of that funder of your experience? Or suppose that the foundation’s strategy could be strengthened by knowledge that you have about a particular population group or region? How can you offer your expertise? Or suppose that the grantees that a foundation is supporting are not providing the services they are supposed to be providing? How can you provide the foundation with your beneficiary feedback? For most foundations, you can’t. This seems to us not good enough.

Hence it’s not the norm elsewhere. For instance, all UK local authorities have their decision-making meetings in public, as does the National Institute of Health and Care Excellence which decides what treatments can be funded from public money.

What’s to hide? One foundation representative perhaps gave the game away by saying outright: “We are accountable to ourselves, not [to] the public. They do not fund us.” Given the tax subsidy, that just isn’t true.

Our purpose here is not to moan or cast blame, but to raise the issue and suggest some ways that charities and foundations can be more accountable and transparent to those who fund them. We are not suggesting that every single charitable entity be required to hold them; most of the 180,000 registered charities in the UK and a million in the US have zero staff. Rather, we suggest requiring organizations with budgets over a certain threshold to hold such events – that threshold might be £1m or $1m, and it might rise over time.

--Caroline Fiennes

Share This Blog

  • Share This

About Transparency Talk

  • Transparency Talk, the Glasspockets blog, is a platform for candid and constructive conversation about foundation transparency and accountability. In this space, Foundation Center highlights strategies, findings, and best practices on the web and in foundations–illuminating the importance of having "glass pockets."

    The views expressed in this blog do not necessarily reflect the views of the Foundation Center.

    Questions and comments may be
    directed to:

    Janet Camarena
    Director, Transparency Initiatives
    Foundation Center

    If you are interested in being a
    guest contributor, contact:
    glasspockets@foundationcenter.org

Subscribe to Transparency Talk

Categories