Knowing How and When to Monitor, Assess, and Evaluate Grants

A family foundation's legal responsibilities for monitoring, assessing, or evaluating the grants it makes to organizations with 501(c)(3) tax exempt status are minimal. The IRS requires little in the way of detailed reporting on the outcome of specific grants—except for grants to organizations that lack 501(c)(3) status. Even without required reporting requirements, however, most foundations ask their grantees for a final report that gives a financial accounting of how funds were used, a narrative description of any accomplishments of the project or program, and copies of any media coverage of the project. For small grants, this type of self-evaluation is generally considered sufficient.

Alfred L. Castle of the Samuel N. and Mary Castle Foundation in Honolulu, Hawaii (assets of $40 million) offers the following advice on how small foundations can get grantees to prepare self-evaluations:

"Trustees of small foundations who would like to have an inexpensive self-evaluation completed by a grantee should…let grant applicants know at the time they apply that this is a condition of receiving the grant. It also helps to send them a copy of the questions at the onset of the grant. Where possible, the grantee should play a role in developing the questions. This process works best when approached as a collaborative effort, since all foundations are partners of nonprofits in achieving mutual goals."

Beyond self-evaluation, many choices and styles of grants evaluation are available. Some family foundations opt to use their in-house personnel, while others rely on hired consultants to conduct the analysis. Some choose to conduct their analysis on the basis of interviews and written responses to questions, others conduct site visits while the grant is in progress or at its conclusion, and still others employ a combination of these methods. In the case of larger grants, foundations sometimes decide to pay for a more objective analysis, which may mean an investigation using all the rigorous tools of modern social science and statistics. Others take a more relaxed approach, using a consultant to interview program participants and staff and to draw conclusions with the detachment of an outside observer.

William J. O'Neill, Jr. and the William J. and Dorothy K. O'Neill Foundation of Cleveland, Ohio (assets of $5.2 million), require different levels of evaluation, depending on the size of the grant and its importance to the foundation's strategy. For "small" grants of less than $5,000, no evaluation is required. For mid-size grants (up to $15,000), self-evaluation by the grantee is required. For larger grants, a site visit by a foundation representative is conducted to verify the self-evaluation. Finally, its current "focus area project"—a project to which the foundation has awarded more than $200,000 for two years—the foundation collaborated with another area family foundation to include the services of an independent evaluation specialist in the project. Of course, the detailed evaluation of the focus area project is intended not only as verification but also as part of the overall strategy to learn from a model project and to disseminate important findings.

Often, outside assistance for evaluations can be obtained through collaboration with other grantmakers. Such a collaboration might establish one evaluation process for a project, even though the project is receiving grants from many different foundations. In addition, it may be possible to borrow someone with evaluation skills from a community foundation or larger private foundation at less cost than tapping the open consultant market.

Approaches to Monitoring and Evaluating Grants

Most family foundations with assets of less than $10 million are managed by trustees and family members without the support of a large administrative budget or professional staff. Trustees of these foundations often assume that monitoring and evaluating a grant is difficult and expensive. As a result, they choose to fund only those organizations with which they are familiar. This assumption is incorrect: monitoring and evaluating grants can be accomplished relatively easily and cost-effectively.

Defining Monitoring and Evaluation. Grants monitoring and evaluation attempt to determine the worth or effectiveness of grants based on the grantmaker's criteria. Family foundations of every size can benefit from these activities; they are essential to an ongoing process of self-development and "course correction."

Monitoring is the process of ensuring that a grantee fulfills certain commitments during the course of a grant—files reports on schedule, allocates grant money to the purposes designated in the grant agreement, and carries out promised activities. Orderly monitoring can be very important in detecting management or fiscal accountability problems before they get out of hand. Monitoring is no substitute, however, for the actual assessment of results, which is evaluation.

IRS regulations do not require grantmakers to monitor grants or other activities. Grantmakers can ask for appropriate IRS documentation from the grantee and then monitor the grantee to determine whether the funds were spent according to the grant agreement and possibly whether a project or program has been completed. If time permits, a brief site visit by a board member can also provide useful information.

Evaluation has been defined as "the systematic collection of information about the activities, characteristics, and outcomes of programs for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programs are doing and affecting."

Evaluation goes beyond monitoring in that it tells trustees how well their grants have been implemented and to what extent the foundation's mission is being fulfilled. Some smaller family foundations find it useful to use evaluation to determine community needs or the impact of grants. The key point is that evaluation can be a useful tool for those who wish to use it, and it is a tool that can be employed in greater or lesser degrees as the foundation governors and managers see fit.

There are, technically, six types of evaluation:

Information from evaluations can help a family foundation board to:

Techniques for Conducting Evaluations. Small foundations can rely on grantee self-evaluations or make use of an experienced consultant or a graduate student at a local university to conduct evaluations. They can also perform site visits. Each of these approaches is discussed briefly below.

Grantee Self-Evaluation.The most common form of grant evaluation employed by smaller foundations is the grantee self-evaluation. One family-managed foundation with assets of $2.5 million, for example, requires that each grantee sign a contract stating that it will provide a narrative evaluation of its project at the end of twelve months.

Grantees are sometimes reluctant to admit to their project's weaknesses, or may be unfamiliar with the self-evaluation process and thus unsure of what to report. Nevertheless, when a relationship of trust between the foundation and grantee has developed, grantees almost always appreciate participation in the evaluation process, and are more likely to accept and implement findings.

To help grantees with the self-evaluation process, foundations might ask that they answer the following five questions:

Other helpful information to request includes copies of press releases, media stories, or other materials published regarding the funded program. Such information is very useful in preparing an annual report or other communications.

Outside Assistance. Consultants can bring objectivity, autonomy, and credibility to grant evaluations. Family foundations sometimes locate consultants by checking with staff of a larger private foundation or a community foundation or through peer networks.

Foundations with limited funds for part-time consultants might consider these means for getting an external assessment at the lowest cost:

Trustee Site Visits. Site visits, conducted after a grant has begun, are relatively inexpensive and allow funders to see grantees and their grant dollars at work. During these site visits, foundation trustees might ask grantee personnel such questions as:

In addition, trustees might want to consider asking the intended beneficiaries of the foundation grant:

As William S. White, president and CEO of the Charles Stewart Mott Foundation in Flint, Michigan, notes, the bottom line for any evaluation is to help a foundation ensure that its objectives are on target and that its programs are meeting those objectives:

"Evaluation is a continuous process, one that should provide useful information to decision makers both inside and outside the organization. The single most important feature of any evaluative system is its ability to prompt learning. And it is in the process of reflection and dialogue about what is learned that a foundation can apply lessons in order to improve its service."

Tips for Making Evaluations

Drawing on several sources in the field, here are some tips from experienced grantmakers:

What Foundations Are Really Doing: Two Surveys

In the past few years, the Council on Foundations on its own, and the University of Pittsburgh, in collaboration with the Lilly Endowment, have sought to determine what foundations are doing to evaluate the performance of their grantees. Results of their two surveys provide insights into the realities of grantmaker evaluation practices.

Foundation Management Survey. The 1994 Foundation Management Survey was the first survey in which the Council on Foundations explored grantmakers' evaluation practices in some depth. The questions were intended to determine the variety of program and self-evaluation practices among the different types and sizes of grantmaking organizations: what kinds of grants are evaluated, how evaluations are conducted, and how the findings are used. Selected excerpts of the analysis:

National Survey on Role of Evaluation. The second study is called "A National Survey of Small and Mid-Size Foundations: The Role of Evaluation," and was published in July 1996. The Lilly Endowment supported the survey as part of the Endowment's partnership effort with the University of Pittsburgh to enhance its capacity to evaluate its grantmaking in education. An additional goal has been to contribute to what is known about evaluation use in foundations.

The survey queried 238 members of the Council on Foundations, all of which had assets below $250 million and 25 percent of which had assets below $10 million. Family foundations were not identified as a distinct category. The authors developed twelve findings from their data:

The authors found that evaluation "clearly centers on grantees," which, after foundation personnel themselves, are the most likely evaluators. The authors concluded that:

This emphasis on self-evaluation has mixed results from the perspectives of foundation personnel. There are concerns about the grantees technical adequacy and the natural tendency to be less than candid. However, the grantees involvement in evaluation reflects the notion of the 'reflective practitioner" that is considered a best practice in the literature on effective organizations.