Transparency Talk

« TMI vs. ROI: Risks and Rewards of Philanthropy 2.0 | Main | Foundations Fail at Failing »

Building a Community for Excellence in Evaluation
January 10, 2011

(James Kemp has several years experience as a fundraiser and researcher with Oxford Research Group: an award-winning UK-based charity. In this role James worked closely with decision makers, independent experts, and the media to develop and promote policies relating to UK and international security. James has worked with Nominet since 2007, and is currently the Technical Department's Project Manager.)

Kemp_100 A Missed Opportunity

Project evaluation forms... what is a grantmaker to do? Generally, grantees would rather spend time on work that contributes directly to their objectives. Rightly so, given the use to which funders usually put data gathered through evaluation. But without high quality and well-designed project evaluation processes—plus good application of the data it creates—the consequences are great. Our communities of interest learn slower and are less efficient; funders are unable to reliably understand, predict, and communicate impact; and we are all less effective.

In short, project evaluation forms exemplify a missed opportunity; a difficult problem; a major under-utilization of data. How can we maximise the value of this aspect of grantmaking and social investment for the benefit of funders and grantees?

Addressing the Challenge and Adding Value

Being recently established, Nominet Trust can take a fresh approach to evaluation processes—and obstacles—from its earliest grant cycles. There is much to refine and test, but what may be of interest to various civil society organisations, is the tool we have built that lies at the centre of our evaluation process. The tool is called the Knowledge Centre (after Foundation Center naming conventions), and it is an open, searchable online library containing completed project evaluation forms as case studies. It is in the proof of concept stage, so we welcome feedback.

The Centre is not restricted to Nominet Trust grantees – which is critical. Anyone who has completed a project that matches one or more of our charitable objectives is invited to complete our online  case study form. (The content is covered by a Creative Commons license - Attribution-Noncommercial-ShareAlike 2.0 UK.)

Our  case study forms are short and ask questions designed for the benefit of people planning similar projects (e.g. questions to find out what people would do differently if they were to repeat the project). We also ask about partners, so in time we capture data for social network analysis. Feedback on this form would also be appreciated.

The Knowledge Centre is engaging and useful for both the people who submit forms and those who search for forms. Because each case study is published as a web page it can easily be shared via social media outlets. Forms also feature a contact button, project outputs like videos and reports, and reflect an organisation’s own branding. In effect, these pages double as an advertising platform and networking opportunity, which may be especially helpful for small to medium-sized organisations.

Benefits for Every Audience

So, how do we intend for this tool to be used? Well, in addition to searching the content, it will be possible to export the entire database as a spread sheet. We believe our Knowledge Center can serve a range of groups. 

Potential grantees would be expected to research the Knowledge Centre before applying, and adapt their project design accordingly. The tool is also designed for connecting people who share objectives: hopefully we’ll see more partnerships, resource sharing and maybe merges. One hope of this tool is that we improve efficiency across the community.

Nominet Trust intends to take each application we receive and cross reference Knowledge Centre data to evaluate the method, understand where the organisation fits into social networks, judge whether they have learned from others, and make well-informed decisions with a strong strategic overview. We expect to benefit from the knowledge-sharing and learning opportunities this tool presents, as much as other users. We also intend to analyse the content to understand and communicate more about models of change that are most effective, and so on.

Policy makers and researchers will benefit from the data set built by the Knowledge Centre over time. For example, you could take all case studies against a particular need within a certain region and time, and ask a researcher to analyse our data (along with other sets) to help assess the impact of certain initiatives, organisations or networks, markets or polices, etc. Knowledge Centre data will be available for mashing up with other data sets.

Beyond these primary users, anyone who is interested in civil society activities directed towards shared charitable objectives can use this tool to learn more about what’s going on, who’s doing what, what they would do differently if given the chance, and who to contact. We are simply making data available; we look forward to discovering what people do with it.

 

First Steps and High Ambitions

The first step is to get the design right, We have released our Knowledge Centre as a “proof of concept” so that we can benefit from feedback. The next and hardest challenge will be populating the database. We face an age old problem: how to incentivise people to divert resources to the creation of what could become a common good, if enough people participate? For our own grantees, inclusion in the Knowledge Centre is a grant condition. Knowledge Centre’s potential depends, however, upon a breadth and depth of data which Nominet Trust cannot generate alone.

Demonstrating the potential through advertising, blogs, and other communication channels will help, but these efforts will not be enough to reach a critical mass. We will appeal to people’s better judgment, but this could take a long time.  Maybe we can build a useful network of participants? Maybe other organisations working to our objectives can incorporate it into their own evaluation and case study management processes?

We have high ambitions for this tool. It is no panacea, but if we can generate enough content for its value to be proven, it could become immensely informative. It could serve as the basis for coordination (between grantees and funders). It could be put to all sorts of uses that improve project and organisational effectiveness, use resources more efficiently, inform policy making, and help to realize our common objectives.

Imagine if many funders implemented a similar tool, and used common vocabularies to create a database for social change. We do not underestimate the challenge we have set for ourselves. Building the Knowledge Centre is only the first step. Our hope is that many other people join us. Together we could build a tool with the power to make an immense contribution to civil society.

— James Kemp

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Share This Blog

  • Share This

Subscribe to Transparency Talk

  • Enter your email address:

About Transparency Talk

  • Transparency Talk, the Glasspockets blog, is a platform for candid and constructive conversation about foundation transparency and accountability. In this space, Foundation Center highlights strategies, findings, and best practices on the web and in foundations–illuminating the importance of having "glass pockets."

    The views expressed in this blog do not necessarily reflect the views of the Foundation Center.

    Questions and comments may be
    directed to:

    Janet Camarena
    Director, Transparency Initiatives
    Foundation Center

    If you are interested in being a
    guest contributor, contact:
    glasspockets@foundationcenter.org

Categories