Introduction

This publication focuses on helping those engaged in program evaluation efforts draft well-written, impactful executive summaries (ESs). It aims to equip readers with an understanding of the role of ESs, and practical strategies for their writing. It is a resource for Extension professionals and evaluators within the Cooperative Extension System (CES), program evaluators in other fields, and individuals involved in evaluating, writing ESs for their evaluation reports, and disseminating the impact of their programs.

The original examples presented in this document are part of a hypothetical case, ‘a recycling program’, that aims to illustrate best practices for well-written ESs. For this publication, the term ‘program’ is used to mean ‘intervention’, referring to a program, project, or strategy.

Compiling Executive Summaries of Program Evaluation Reports

An executive summary is a standalone section embedded in the evaluation report (ER). It provides a snapshot of the completed evaluation, without requiring a full read of the ER (Vassallo, 2003).

The summary can also serve as input to the evaluation's dissemination efforts. When ESs are used in program evaluation for dissemination efforts and follow-ups, they are produced as separate documents and given different names. For instance, ESs are also known as evaluation briefs, key insights reports, findings at-a-glance, or snapshots of findings.

The intended primary audiences of ESs are the same as those of the evaluation. These are the ES intended readers, including those interested in the program being evaluated and the evaluation findings, to make informed decisions. These audiences can be internal and external to the organization commissioning the evaluation. For these audiences, the ES might be the only part read in an entire ER (U.S. Government Accountability Office [GAO], 1986). So, when writing an ES, it is important to understand who the prospective audience is and what their information needs are.

Before drafting an ES, reflect on and answer the following questions about its audience:

• What does the prospective audience already know about the program and its goals and outcomes?
• What is the prospective audience's level of interest in the program?
• What kind of decisions can this audience make based on the evaluation findings?
  • Then, reflect on: What do we want the ES audience to know, how much information they need, and what decisions do we intend to influence? (Baxter & Braverman, 2004).

"The What" Content

A typical ES in program evaluation consists of three core parts: introduction, body, and conclusion (Table 1). Each part contains all the necessary elements to provide an overview of the program and the evaluation, both extracted from the full ER.

Table 1. Parts of an Executive Summary

Introduction Section

An ES begins with an opening statement that presents the ER: “This report…” It is followed by information about the program, and the evaluation stories.

Examples can be found in Tables 3 and 5.

About the Program

The introduction sets the stage for the program's context, by identifying the program and what it’s about (Table 2).

Reflect on the following:

• What are/were the top-level goal(s) of the program?
• What are/were the key expected outcomes?

Table 2. Necessary Elements to Introduce the Program: Program-Specific Information

Table 3. Example: Introduction-About the Program

About the Evaluation

The introduction provides an overview of the evaluation purpose, objectives, and intended use (see Table 4). When relevant, it also identifies the core evaluation criteria. For example, the Organization for Economic Co-operation and Development (OECD) criteria: relevance, effectiveness, coherence, impact, sustainability (2021).

Table 4. Necessary Elements to Introduce the Evaluation

Table 5. Example: Introduction-About the Evaluation

In the introduction section, avoid using words or phrases that carry advisory or necessity, such as phrases that carry ‘should’ or ‘must’, as well as phrases that communicate any type of recommendations, such as “The organization X commissioning the evaluation needs to” (U.S. GAO, 1986).

Body Section

About the Program

The first part of the body of an ES communicates a high-level overview of efforts to implement the program, what was done, and the expected outcomes (see Table 6). See an example in Table 7.

Table 6. Body: Necessary Elements to Understand the Program Story.

Reflect on the following questions:

• What should be the activities that give readers a representation of how the program is/was implemented and that they should remember?
• What are/were the main program activities that led to achieving the most critical outcomes?

Table 7. Example: Body-About the Program

About the Evaluation

The second part of the body of an ES provides a high-level overview of the evaluation journey, including a brief introduction to the methodology and evaluation approach, the main findings, conclusions, and all recommendations (see Table 8).

Table 8. Body: Necessary Elements to Understand the Evaluation Journey.

Table 9. Example: Body Section-About the Methodology and Data Analysis, Findings, Conclusions, and Recommendations

Conclusion Section

About the Program

The conclusion part communicates two key messages:

The thesis of the program’s value conveys the overall judgment of its value and significance in context. This judgment is based on the program’s performance (i.e., whether positive or negative), and why it matters (i.e., whether it contributes to solving or ameliorating the issue or advancing the opportunity it addressed). It also presents lessons learned intended to prompt reflection among decision-makers about whether it was worth investing time, effort, and money in implementing the program. See an example in Table 10.

Table 10. Example of Thesis Statement.

The call-to-action statement advises the utilization and influence of the evaluation findings. It is an invitation to make informed decisions about the suggested program's next steps; whether to continue, discontinue, or scale up. The message also aims to inspire, motivate, influence, provoke reflection, and potentially activate commitments, actions, advocacy, policy dialogue, or other related activities among decision-makers and program contributors. When writing the call-to-action statement, consider:

• Using commanding verbs that prompt action (‘what to do’) and convey benefit-focused cues that are expressed as a priority to advance the program or problem it addresses;
• Including words that matter by assessing the power of each word and verb;
• Persuasive language: Use change-inspiring language that invites and motivates prospective ES users to prompt action or activate thoughts based on evaluation knowledge from evidence, and which focuses on the general, not including the details of the solution (Syed et al., 2020; Vasallo, 2003);
• Use polite persuasion and tentative language through suggestions expressed as means of possibilities ‘could’ (suggestion), and ‘would’ for hypothetical situations of the recommendations (UNEG, 2021);
• Avoid using direct commands. “Do this approach….” (U.S. GAO, 1986);
• Communicate without exaggerations or overreaching into persuasion ((U.S. GAO, 1986), so as not to deviate from the evaluative evidence reflected in the findings and conclusions.

Reflect on the following questions:

• What is the central message and why is this message critical?
• Does this message potentially lead to decisions, constructive actions, inspire thoughts, or influence advocacy, or policy dialogue?
• Based on the evidence, what actions are needed to advance the strategic goals of the organization commissioning the evaluation?

Table 11. Example of the Conclusion Section.

"The How"

When drafting ES, ensure you read the full report and address the following:

Writing Clarity

Substitute highly technical vocabulary with accessible terms (Vassallo, 2003). Use simple terms that replace some common statistical terms used to describe program results (see Table 12)

Reflect on the following question:

• Does the ES speak to a general audience?

Table 12. Recommendations to Replace Technical Terms.

Acronyms

Spell out acronyms for the first time they are used.

Information Extraction

Summarize critical information and reflect on the following:

• What information is important?
• To whom is the information important?

Paraphrase

Reword the meaning of the text based on interpretation, ensuring that the original meaning in the ER is maintained, except for the recommendations, which should be maintained in their original form.

• Avoid cut-and-paste information from the ER or adding new content (Vassallo, 2003).

Conciseness

Reduce the amount of information through the following strategies:

• Express ideas using fewer words by substituting unnecessary adjectives, filler words, and inflated language and minimizing the use of transitional words and phrases. Examples: “Due to the fact that…”, “It is important to note that…”;
• Employs only words that convey meaning (content language);
• Avoid detailed and unnecessary background information (Vasallo, 2003);
• Do not include raw quantitative data, statistics, or qualitative data from respondents' quotes.

Reflect on the following:

• What information can be omitted without compromising a full understanding of the evaluation story?

Length

ESs should be short. Ideally, an ES in program evaluation should not exceed three pages; others recommend a maximum of five pages (United Nations Children’s Fund [UNICEF], 2021). However, the specific length of the ES varies based on the clients’ needs (Vassallo, 2003). It is the responsibility of the ES writer to identify these needs and expectations and ensure they are met.

Format

To effectively present information for easy access, it is advised to address the following:

• Use headings and subheadings organized by topic, themes, or criteria;
• Use bullet points and lists or brief statements when appropriate (Vasallo, 2003);
• Use third person (e.g., ‘Cooperative Extension personnel recommend’ instead of ‘we recommend’) (U.S. GAO, 1986);
• Highlight in bold letters the main evaluation findings, conclusions, and recommendations to communicate the importance of the content (Kho, 2022);
• Use relevant evaluation criteria, topic areas, or themes to organize the main evaluation findings and conclusions.

See examples in Tables 13 and 14.

Table 13. Example: Headings and Subheadings, Bullet points, and Lists

Table 14. Example: Bold letters for Findings and Conclusions.

Organization of the Results, Conclusions, and Recommendations

• Rank the main findings based on importance, urgency, or potential use or influence, so that those that matter most to decision-makers are placed first;
• Start by organizing the results with titles and sub-titles based on evaluation questions, topics, themes, or evaluation criteria;
• Present each result with its corresponding conclusion and recommendation (see Table 15).
  • Title (or Sub-titles)  -> Key Result  -> Conclusion  -> Recommendation

Table 15. Example: Organization of Findings, Conclusions, and Recommendations

Recommendations

Consider these suggestions:

• Write recommendations as presented in the ER, preserving them all (United Nations Evaluation Group [UNEG], 2018).
• Avoid using charts, dense tables, and figures that add cognitive load to readers.
• Adhere to the design, format, and length requirements of the funding, donor, or organization commissioning the evaluation or the evaluation's end users. For instance, in the UNICEF Global Evaluation Reports Oversight System (GEROS), the support evaluation function used to meet standards and commitments employs the GEROS evaluation quality assurance tool to rate the quality of final ER. The version 2021 template includes quality criterion dimensions, with one quality criterion assessing ES quality, comprising three indicators. One indicator states that the ES must not exceed five pages, except where different instructions are provided in the evaluation terms of reference (UNICEF-GEROS, 2021).
• Rank in order of priority for prospective users (United Nations Evaluation Group [UNEG], 2017).
• Numbered and highlighted in bold (Hendricks & Papagiannis, 1990).
  • As appropriate and relevant to the purpose and scope of the evaluation and the intended ES readers, consider the strategies suggested below:

• Present the recommendations in line with three major categorizations: (a) focus (strategic,
• operational, tactical) (Baxter & Braverman, 2004), (b) timing (short-, intermediate-, long-term) (Hendricks & Papagiannis, 1990), and (c) the team or person responsible for implementing them (UNEG, 2017). The strategic recommendations go first, followed by tactical recommendations (see example in Table 16), and lastly operational recommendations (see example in Table 17).
• Present in a table format. See examples in Figures 1 and 2.
• Use infographics to enhance understanding of the content.

Table #16. Example: Recommendations Organized by Focus

Table 17. Example: Recommendations Organized by Program Component and Priority Level

Figure 1. Example of Executive Summary Format Organized by OECD Evaluation Criteria

Figure 2. Example of Executive Summary Format

Quality Assurance of the Executive Summary

When proceeding to the final draft of the ES, Vassallo (2003) recommended the following strategies:

Self-reflection

• Does the information in the ES matter to the primary intended audience?
• Are the findings reduced to the most critical?
• Is the ES talking to experts in the subject matter rather than everyday readers?
• Does the overall meaning of the ER remain in the ES?
• Can the ES be independently read and still make sense of the evaluation story?

Feedback from Reviewers

It is advisable to gather constructive feedback, from reviewers on the evaluation team, or from those familiar with the program and the evaluation. For instance, in the Cooperative Extension System (CES), this person could be a staff member implementing the program, or a peer working in Extension, or a county Extension advisory committee member. Ask reviewers to provide objective feedback, including reasons for suggestions, and ways to improve the ES. Integrate those suggestions that you consider relevant for the purpose and scope of the evaluation and the intended readers (Keller, 1994).

Quick Test

The final step is to assess the clarity of the ES content with at least one volunteer reader. Recruit individuals with no specialized knowledge or technicalities of the program’s subject matter, and who may share similar characteristics with the intended readers. Ask them to read the ES and evaluate whether they can understand the information in a single reading (Vassallo, 2003), without needing to refer to the full ER (U.S. GAO, 1986). If they report any challenges in understanding, the ES will require revisions to improve clarity. If no issues are found, you can conclude that the intended readers will comprehend the ES, and no improvements are needed (Vassallo, 2003).

Conclusion

Writing a well-written executive summary is essential to meet the needs of its intended readers, as it may be the only part of an evaluation report that some audiences read. Producing a well-written ES is a critical step for effective evaluation reporting. The strategies outlined in this document are designed to assist ES writers in drafting well-written ESs.

Resources

Baxter, L. W., & Braverman, M. T. (2004). Communicating results to different audiences. Foundations and evaluation: Contexts and practices for effective philanthropy, 281-304. John Wiley & Sons, Inc.

Centers for Disease Control and Prevention.  (2018). Public health emergency preparedness and response capabilities: National standards for state, local, tribal, and territorial public health.

Federal Emergency Management Agency. (2011). National disaster recovery framework: Strengthening disaster recovery for the nation.

Hendricks, M., & Papagiannis, M. (1990). Do's and don'ts for offering effective  recommendations. Evaluation Practice, 11(2), 121-125.

Keller, C. (1994). Stay healthy with a winning executive summary. Technical Communication, 41(3), 511-517.

Kho, V. (2022). McKinsey Executive Summaries: Short but Powerful. Writing for University and Beyond: A Journal of First-Year Student Writing at University of Toronto Mississauga, 2(1), 87–95.

Syed, S., El Baff, R., Kiesel, J., Al Khatib, K., Stein, B., & Potthast, M. (2020, December). News editorials: Towards summarizing long argumentative texts. In Proceedings of the 28th International Conference on Computational Linguistics (pp. 5384-5396).

Organization for Economic Co-operation and Development. (2021). Applying evaluation criteria thoughtfully. OECD Publishing, Paris.https://doi.org/10.1787/543e84ed-en.

Patton, M. Q. (2011). Essentials of utilization-focused evaluation. Sage Publications.

United Nations Evaluation Group. (2018). Improved quality of evaluation recommendations checklist. Retrieved from https://www.unevaluation.org/uneg_publications/improved-quality-evaluation-recommendations-checklist

United Nations Children’s Fund. (2021). Global Evaluation Reports Oversight System (GEROS). Quality assurance tool to rate the quality of final evaluation reports.

United States General Accounting Office. (1986). Guide for writing executive summaries.

Vassallo, P. (2003). Executive summaries: Where less really is more. ETC: A Review of General  Semantics, 60(1), 83–90.