In short: Impact analysis means naming the specific change an organization wants, weighing the quality of its evidence, and asking what would have happened without the intervention. Organizations that build these steps into program decisions overstate their impact less and produce analysis funders trust more.

Impact organizations make consequential decisions constantly (expand a program, pause one, redirect funding toward a different intervention) and the quality of those decisions depends entirely on whether they’re grounded in a real impact analysis or in intuition about what seems to be working.

Define the outcome before you decide anything

Impact analysis starts with naming, specifically, what change the organization is trying to produce and for whom, not a general goal like “help the community,” but a defined outcome that can actually be measured against a baseline.

Weigh evidence quality, not just evidence volume

A decision built on a small amount of rigorously collected, representative data should carry more weight than one built on a large volume of self-reported, unverified figures. Impact analysis has to grade its own inputs before using them to justify a decision.

Build in the counterfactual question

The hardest and most important question in impact analysis is what would have happened anyway, without the intervention. Organizations that skip this question tend to overstate their own impact; the ones that build it in, even informally through comparison groups or trend data, produce analysis funders trust more.

Let the framework do the structuring

A consistent measurement framework (IMMCF, Impactedia’s Impact Measurement, Management, and Communication Framework, is one built to be interoperable with IRIS+, GRI, and the UN SDGs rather than compete with them) keeps impact analysis comparable across programs and over time, instead of each decision being assessed on its own improvised criteria.

Using Impact Analysis Before Changing a Program

Before scaling, changing, or ending a program, an organization needs a clear view of what will happen as a result: to the people it serves, to funding relationships, and to its own capacity.

Map dependencies first

A program rarely exists in isolation. It depends on staff capacity, partner relationships, and funding timelines. Mapping these dependencies before a major change, such as scaling to a new region or shifting a delivery model, surfaces risks that are not visible when looking at the program alone.

Plan scenarios for funding uncertainty

Impact organizations operate with more funding volatility than most businesses. Modeling program outcomes under a funding shortfall, a delayed grant, or an unexpected surge in demand helps leadership make faster, more grounded decisions when conditions shift.

Separate operational risk from program impact

Two related questions deserve separate answers: what happens operationally if we make this change (staffing, budget, timeline), and what happens to program impact itself (outcomes for the people served). Conflating the two can lead an organization to favor operational convenience at the expense of mission outcomes.

Turn the analysis into a plan

Impact analysis is only useful if it produces a concrete plan before a program change is implemented: who needs to be informed, what capacity needs to be built, and which risks need mitigation.

This is the analytical discipline behind Impactedia’s Insights Lab.

Frequently asked questions

What is impact analysis?

Impact analysis starts by naming the specific change an organization is trying to produce and for whom, as a defined outcome that can be measured against a baseline. It then weighs the quality of the evidence and asks what would have happened without the intervention.

What is the counterfactual question in impact analysis?

It asks what would have happened anyway, without the intervention. Organizations that skip it tend to overstate their impact. Those that build it in, even informally through comparison groups or trend data, produce analysis funders trust more.

What should an organization analyze before changing a program?

Map the program’s dependencies on staff capacity, partner relationships and funding timelines. Model outcomes under different funding scenarios. Answer operational risk and program impact as separate questions. Then turn the analysis into a plan: who needs to be informed, what capacity needs to be built, and which risks need mitigation.