Transparency Talk

Category: "Logic Models" (4 posts)

The Journey from Practice to Theory: Developing a Foundation’s Theory of Change
February 7, 2013

Mary Gregory is the executive director of the Bella Vista Foundation, one of twenty-two foundations managed by Pacific Foundation Services (PFS). She has been with the company for fourteen years and enjoys the variety of philanthropic styles demonstrated by PFS’s clients.

Gregory-100I have the privilege of managing a number of grantmaking portfolios for PFS foundations, and each has taught me important lessons about the art and science of grantmaking. Most recently, as a result of many years of work with the Bella Vista Foundation (BVF), I had the opportunity to learn first-hand what it takes to develop a foundation’s theory of change. But first, let me give you some background. The Bella Vista Foundation (BVF) was started in 1999, and within a few years of making general grants to benefit children/youth, the board decided that one of its purposes should be to make a difference in the lives of children prenatal to three years old from low-income families in four Bay Area counties.

In 2007, after reviewing data, reading studies on infant development, and talking to experts in the field, BVF decided to fund programs that help parents and caregivers cope with stress and anxiety in order to prevent more serious mental health issues from arising which might negatively affect the health social and emotional development of their infants and toddlers. The foundation looks for high quality, culturally aware programs for parents and caregivers that may use any of a number of strategies to create well-being and community, including exercise, classes (such as parenting education), community activism, and peer counseling. These programs can be initiated by nonprofit organizations, county departments, or joint efforts between counties and independent organizations.

In 2012, with a grantmaking capability for this program area that currently amounts to about $1.2M per year, Bella Vista Foundation began to think about whether it could measure its impact. How could the foundation tell if parents and caregivers of very young children were actually better able to cope with anxiety and stress? BVF now encourages its grantees to set goals for their programs. Some programs already measure impact on their clients, using any of a variety of measurement tools that are easily available, to see if levels of stress and anxiety decrease in a meaningful way as a result of participation. Collection of this data also helps grantees to see if they need to revise their programs to get better results.

We realize impact measurement is tricky for foundations, as our investments are just part of a whole ecosystem of funding. BVF’s thinking is that if we aggregate the results of our grantees, we will at least know how many individuals were positively affected by these programs, and what percentage of the participants that represents. Through grantmaking, we are also getting a picture of how many agencies and/or nonprofits in each of our four counties are addressing parental stress and anxiety in families with young children. When Bella Vista Foundation is able to aggregate the programs’ results, we will have a sense of whether our grants are making a difference, and can also create a body of shared learning that will benefit our grantees beyond the grant investment.

Logic Model

View the logic model»

During the past year, in order to lead the way and to better understand the process, the foundation created and publicly shared its own Theory of Change (TOC). As board and staff crafted the TOC, we decided that this might also be a useful tool for our grantees, so we worked with a consultant to help us standardize our language, to review the foundation’s draft version, and to lead a workshop for grantees to get them started on creating their own TOCs. BVF then offered small technical assistance grants to six organizations that wanted to continue and refine their work, which is ongoing—the work will take place between now and early summer. We now know how difficult it is to create a Theory of Change! Foundation staff members are creating customized versions of our TOC for each of the four counties in which BVF makes grants because each county is different, so our activities and funding in each county will need to be customized. Bella Vista Foundation hopes that we can use this new set of tools to measure our progress towards our goals and our vision, and make our own course corrections when needed.

--Mary Gregory

50 Shades of Transparency
November 29, 2012

(Daniel Matz manages the Glasspockets web site.)

Daniel MatzGlasspockets is turning 50. Well, more like three — in January 2013 — but as of this week, the Glasspockets web site now hosts 50 transparency and accountability profiles. Collectively the foundations that have put themselves to the "Who has glass pockets?" challenge represent $138 billion in assets and more than $6.5 billion in annual giving – close to 15 cents of every foundation grant dollar distributed in the United States. Back in 1952, Russell Leffingwell, then chairman of the Carnegie Corporation, called on the philanthropic community to have glass pockets; that is, to make the work of foundations transparent and to make them accountable to the public and the communities they serve. We now have tangible proof this is happening. Congratulations to the Glasspockets 50 for showing their commitment to transparency and accountability, and meeting the Glasspockets challenge!

... this movement is not just about California or limited to grantmakers with the deepest pockets. Profiled grantmakers hail from 19 states and Washington, DC. And some of the most creative work and most forthright efforts have come from relatively modest quarters.

What exactly is a Glasspockets profile? For the uninitiated, since 2010 we have been cataloging foundations' online transparency and accountability practices — everything from the obvious like contact information and application procedures, to the more demanding like codes of conduct and diversity statements, through to the most challenging activities like making public their grantee feedback and assessing overall foundation performance. We've identified 23 such practices and keep track of them across six types of online communication vehicles (from web sites to blogs to RSS feeds). Together, these indicators help us all see into a foundation; they provide a snapshot of a foundation's "glass pockets." Trends across the sector emerge from aggregated findings displayed as a national Heat Map, the map itself becoming a unique reference tool for other foundations.

Who has glass pockets? Not surprisingly the list includes many of the very largest foundations, many of them at the forefront of the move toward transparency, championing better communication and challenging themselves to ever-greater openness. Some, like the James Irvine Foundation, have even gone so far as to support a statewide initiative to promote transparency among California grantmakers. Thanks, in part, to their support, California grantmakers now represent 40 percent of those foundations with Glasspockets profiles. With such a strong representation of California grantmakers, Glasspockets now features a Twitter feed that opens a window on what California foundations are saying, as well as a California-specific Heat Map showcasing transparency trends among the state's grantmakers. 

But I am also happy to report this movement is not just about California or limited to grantmakers with the deepest pockets. Profiled grantmakers hail from 19 states and Washington, DC. And some of the most creative work and most forthright efforts have come from relatively modest quarters. The Texas-based KDK-Harman Foundation (annual giving of less than $1 million) volunteered to have a Glasspockets profile, demonstrating that even smaller foundations can regularly assess their performance and have a strong voice in larger conversations around transparency. Foundations are increasingly fond of having grantees complete logic models as part of the proposal process. In the case of KDK-Harman, they've applied the logic model to their own work and use it as a way to report on progress and performance to all their stakeholders. In terms of creative energy, the Mitchell Kapor Foundation (annual giving of $4.3 million) produces video annual reports that are a master class on communicating more effectively using digital media technology.

The "Who has glass pockets?" profiles also provide an inventory of foundation communication vehicles, including social media efforts.  The Robert Wood Johnson Foundation was an early adopter of these tools and has been offering insights and advice throughout the year on the Glasspockets Transparency Talk blog about the ways in which social media tools are transforming the work of philanthropy as well as how to measure the impact and value of social media efforts.

These are just three of the dozens of narratives throwing light on the sector-wide push  toward transparency. The Glasspockets 50 also includes regional foundations, health foundations, and community foundations; foundations focused on international giving like Trust Africa, and small foundations like Nebraska's Woods Charitable Fund (whose web site is designed and hosted by the Foundation Center). The varied interests and size of these organizations (Woods has a staff of four) are the real measure of how deep the move to transparency has become.

Join us in being among the first 100 foundations to show the world their Glass Pockets. Submit your profile today or to learn more, contact Janet Camarena in our San Francisco office.

Explore the Glasspockets 50»

-- Daniel Matz

Evaluating the Impact of Social Media: Are We Wasting Our Time?
June 27, 2012

(Claire Gibbons, Ph.D., M.P.H., is a senior program officer in the Research & Evaluation Unit at the Robert Wood Johnson Foundation. She spends most of her time managing R&E projects for the Quality/Equality team.)

Gibbons_100Last month Steve Downs and I discussed some of the Robert Wood Johnson Foundation’s (RWJF) experiences using social media and our first steps towards evaluating the impact of RWJF’s social media use in a webinar for Council on Foundation members (you can view the slides here). In response to our evaluation discussion, a webinar participant asked whether it makes any sense to evaluate something as spontaneous and fun as social media. This was also a question raised by Allison Fine in her blog

Allison expresses concern that a logic model “…misses the essence of what makes social media so unique, the serendipity and fun that are essential parts of “being” social”. This is an interesting and valid question- by creating a stodgy old logic model do we defeat the purpose of social media?

Before I share some thoughts on this question, let me describe briefly what we’ve been doing at RWJF. The staff at the Foundation is using social media, and many are enthusiastic about its potential to increase our impact, but until recently, no one had sat down to elicit exactly what we expect social media to impact. In fact, we engaged in a fairly lengthy period of discussion and experimentation before we began to plan for evaluation. One of RWJF’s initial steps was to form a working group to consider how the Foundation could best take advantage of Web 2.0 tools, and what it would mean for the Foundation if it did use these tools. This working group released a report internally to all Foundation staff in December of 2009. This began a period in which all staff was strongly encouraged to experiment with social media. Just signing up for Twitter and listening to the conversation by following others was encouraged, for example. A second Web 2.0 group was formed after some time passed that was charged with getting some sense of whether RWJF was moving forward with its use of social media and sharing lessons across program areas. It was at this point that we began to focus our attention on evaluation.

We decided that the first step in evaluating our use of social media should be to develop a logic model. We did this, with the help of consultant Victoria Dougherty, based on interviews with staff that were knowledgeable and involved in our social media efforts and on review of documents about our social media philosophy.

Rwjf-logic-model-6-27-graphic.470

RWJF created a logic model to help evaluate the impact of social media.

View a PDF of the logic model »

The logic model has two pathways: the first describes how RWJF can approach its work over the next five years and the second describes some of the outcomes of the work. For RWJF to realize the potential of social media and eventually reach its long-term goals for being a more effective agent of change, and being a connector and facilitator that spurs broad participation in our work it must first position itself as a Web 2.0 organization and work to become more open and nimble. Social media use may also lead to creating new connections outside the Foundation and in turn lead to a greater ability to gather information from a broad network that can result in more effective programming. See my earlier blog post here for a more in-depth discussion of this logic model.

So, back to our earlier question: Does creating a logic model to drive an evaluation of the impact of social media defeat the whole purpose?

I don’t think so. But I’m pretty sure you guessed I was going to say that, given that I’m a Research & Evaluation Officer! So let me share my thinking.

  1. Yes, social media in many cases is driven by spontaneity. Videos that go viral on YouTube are completely driven by spontaneous interest. But not all social media use is purely spontaneous. We believe that social media can be used strategically to further our programmatic goals. That means we can plan ahead to use a social media tool or tools. For example, staff at RWJF used a virtual forum to create an open platform for discussion and idea-gathering about teen dating violence prevention. They received thoughtful input from people working in the field, teens who had experienced dating violence and parents who lost a child due to dating violence, as well as many others. Read more here.
  2. A logic model does not squash innovation! It describes it. Logic models are made to be broken and expanded and changed over time. The presence of a logic model is not meant to limit anyone’s activities to something that happens to be featured in a little box in the model. The logic model doesn’t dictate our programming- it’s simply a way to describe what we are doing and what we think the result will be.
  3. RWJF’s use of social media in the workplace is predicated on the idea that it will help us achieve our goals. We could be wrong. We won’t know if social media is getting us anywhere good, or anywhere good faster, unless we measure some outcomes that we think are related to our programming activities. And one very useful tool for eliciting expected programmatic and policy outcomes is a logic model.

This isn’t to say that we think all possible pathways between use of social media and some good outcome are contained in our logic model. Absolutely not! This is just the best picture we could come up with at this point in time. New tools will become available, and we will use them. Staff will continue to innovate in ways that we haven’t imagined yet, and we welcome that innovation.

We still have a long ways to go in our journey to use social media in a way that helps us reach our strategic objectives, and in measuring and evaluating our use of social media. We certainly don’t have many answers, but think we’re on the right track. What do you think?

-- Claire Gibbons

Transparency and Accountability in Today's Information Age
October 18, 2011

Jennifer Esterline(Jennifer Esterline joined the KDK-Harman Foundation in the fall of 2006 as its first program officer and was promoted to executive director in 2009. Prior to joining the Foundation, Jennifer served as a resource development director at the University of Texas at Austin and Texas RioGrande Legal Aid, and also worked in Mexico for a time on issues related to civil society and philanthropy.)

Before I worked in the world of grantmaking, I raised money for a variety of nonprofit organizations, primarily from foundations. That job was sometimes frustrating because of the lack of transparency of many foundations—particularly smaller, local family foundations. Many of you can probably relate to my experience:  digging for information that usually leaves you feeling more confused than before; making inquiry calls to the foundation that never get returned; and receiving standard rejection letters that tell you nothing about why your request was declined other than the fact that “the foundation had more inquiries than funds available.” Sound familiar?

Bringing that experience with me to the KDK-Harman Foundation, both the staff and leadership of the Foundation vowed to change those “more typical” foundation practices. From the Foundation’s inception, we developed processes that ensured transparency and a feeling of customer service in our grantmaking. We took a strategy straight from the playbook of a much larger and well respected foundation in Texas by disseminating a survey to all of our applicants and grantees, which allows them to evaluate ease of application and reporting processes, the quality of interaction, and the responsiveness and knowledge of staff —whether or not they received a grant.

We also collaborate with other education grantmakers in our region through the creation of the Central Texas Education Funders group—a volunteer-led membership group of foundations that have worked together to create a common application and evaluation report for area nonprofits, learning opportunities around public education,  and even collective grantmaking. This collaboration is meant to ease the application and reporting processes of nonprofits in our region and provide opportunities for learning and sharing among its members and between grantmakers and grantees.

We also adopted many other recommendations from Grantmakers for Effective Organizations (GEO) and the Center for Effective Philanthropy (CEP), among others, that encourage foundations to support capacity building, provide operating support, invest in evaluation and evaluative learning, and establish open and honest dialogue between a foundation and grantees in order to learn from one another’s experiences. Our staff and board realized early on that the effectiveness of the Foundation was completely dependent on the effectiveness of our grantee partners, and as such, we’ve tried very hard to create an environment and relationship with grantees that allows them to have the greatest impact in their communities.

In order to evaluate our effectiveness, we developed a theory of change and a logic model early on that would give both staff and board members a clear picture of what we were trying to accomplish and determine benchmarks to measure progress towards those objectives.  All of these documents are updated regularly and available online on our web site, www.kdk-harman.org.

Assessing performance sounds daunting, but it is well worth the effort.  For small foundations like ours wondering how they should get started with the development of a logic model or other evaluation metrics, here is some specific advice based on our successes and lessons learned along the way:

  • Start small—you don’t have to hire an outside consultant and invest lots of money. Start by bringing your staff and board together during a half-day retreat to talk through elements of a logic model to start a first draft.
  • Collaborate—work with like-minded funders in your area to learn about their evaluation processes.
  • Revisit your performance tools often—logic models are never static, they change over time as the foundation learns more about their areas of expertise and as community needs change.

The staff and trustees of the KDK-Harman Foundation dedicate a lot of time to ensuring that the Foundation is held to the same standards that we expect from our grantees. As a foundation working to break the cycle of poverty through education in Central Texas, we are only as effective as our grantee partners. You don’t have to give away millions of dollars a year to invest in processes that ensure transparency, accountability, and learning within your foundation. Just a simple willingness to learn and an open mind will get you further than you realize.

-- Jennifer Esterline

 

About Transparency Talk

  • Transparency Talk, the Glasspockets blog, is a platform for candid and constructive conversation about foundation transparency and accountability. In this space, Foundation Center highlights strategies, findings, and best practices on the web and in foundations–illuminating the importance of having "glass pockets."

    The views expressed in this blog do not necessarily reflect the views of the Foundation Center.

    Questions and comments may be
    directed to:

    Janet Camarena
    Director, Transparency Initiatives
    Foundation Center

    If you are interested in being a
    guest contributor, contact:
    glasspockets@foundationcenter.org

Subscribe to Transparency Talk

Categories