Knowledge Foundations
Evaluating knowledge mobilization is not the same as evaluating a training programme or a single intervention. It asks whether evidence travelled, was taken up, and changed something downstream — across long time horizons, multiple actors, and conditions that rarely permit a controlled experiment. The strongest evaluation design holds the foundations below in view simultaneously.
The six knowledge foundations
1. The Impact Pathway — distinguishing the levels of effect
Knowledge mobilization moves through interconnected stages — problem identification, research, output generation, knowledge transfer, and application — each a candidate for measurement. Distinguishing reach from uptake, uptake from practice change, and practice change from population-level outcome is the conceptual spine of any credible KMb evaluation.
2. Knowledge-to-Action and Implementation Frameworks — locating where uptake stalls
Process frameworks define what is worth measuring and where uptake tends to stall. The Knowledge-to-Action cycle distinguishes knowledge creation from a surrounding action cycle; the Consolidated Framework for Implementation Research (CFIR) and PARIHS locate the contextual factors — barriers and facilitators — that determine whether evidence is adopted.
3. Theory-Based, Contribution-Oriented Evaluation — reasoning about cause
Evidence-to-practice causation is rarely amenable to randomised trials. Theory-based approaches — contribution analysis, realist evaluation, theories of change — build a credible account of whether and how an initiative plausibly contributed to an outcome, given the other factors at work. This reframes the question from attribution to contribution.
4. Indicators and Measurement — making effect observable
A defensible evaluation rests on indicators that are relevant, usable, measurable, and logically defensible — spanning reach, usefulness, use, partnership, and practice change. Mature practice draws on validated instruments with known psychometric properties, while acknowledging that many existing tools are context-specific and inconsistently reported.
5. Research–Practice Partnership Health — the relational carrier of evidence
Much of knowledge mobilization is relational, carried through partnerships between researchers and knowledge users. Partnership synergy, trust, equitable participation, and shared agenda-setting are themselves evaluable — and their erosion is a leading explanation for why otherwise sound initiatives fail to move evidence into use.
6. Equity and Context Responsiveness — whose outcomes are counted
Whose outcomes are counted, and on whose terms, is a design decision. Community-level indicators track change across populations rather than individuals; equity-centred and community-led measurement asks not only whether evidence was used, but whether its benefits were shared. Realist questioning — what works, for whom, in what circumstances — keeps context in the frame.
Use the impact pathway to decide what to measure and at what level. Use implementation frameworks to locate where uptake is likely to stall. Use theory-based methods to build a credible causal claim where experiments are not feasible. Use validated indicators and instruments to measure rigorously, and partnership and equity lenses to ensure the evaluation captures what actually matters. Rigorous practice holds all of these in view at once.
Why KMb evaluation is a distinct discipline
Most accountability pressure in knowledge mobilization now centres on a single question: did the investment change practice or policy, or did an activity merely take place? Answering it well requires designing measurement into the initiative from the outset, not appending it at the end. Evaluation that is bolted on after launch can rarely establish a baseline, observe the conditions that shaped uptake, or assemble the evidence needed to reason credibly about contribution.
The attribution problem sits at the centre of the field. The pathway from evidence to outcome is long, indirect, and shaped by many actors and forces beyond any single initiative. Long time lags between dissemination and application — often years — mean that the effects an evaluation cares about most may not be visible within its reporting window. Treating this as a reason to measure only what is immediately countable produces evaluations that are precise about activity and silent about impact.
The discipline's response is to substitute credible contribution claims for spurious certainty. Rather than asserting that an initiative caused an outcome, sound practice assembles a contribution story: an explicit theory of how change was expected to happen, evidence gathered against that theory, honest attention to alternative explanations, and a reasoned judgement about plausibility. Realist evaluation complements this by asking which mechanisms produced results in which contexts, so that findings travel to new settings rather than staying trapped in the original one.
Beyond bibliometrics
Traditional bibliometric indicators — citation counts, journal impact factor, the h-index — measure influence within academia, not use beyond it. Alternative metrics that track mentions in policy documents, practice guidance, news, and social channels extend the view toward societal engagement, but they are time-dependent, vary with collection method, and do not by themselves demonstrate that anything changed. The field's direction of travel is toward mixed measures of genuine use, interpreted through theory rather than read off a dashboard, and toward the responsible-metrics principles that caution against allowing a convenient number to stand in for the outcome it only approximates.
Design Process
Effective KMb evaluation is structured, theory-driven, and designed in from the start. The stages below represent a comprehensive approach; in practice, scope and depth are calibrated to each initiative's purpose, time horizon, and resources. Understanding the full sequence clarifies what is gained or lost when particular stages are abbreviated.
Clarify Purpose, Audience, and Scope
Deciding what the evaluation is for before deciding how to do it
An evaluation serves a purpose and an audience. Formative evaluation improves an initiative while it is still taking shape; summative evaluation judges its results; developmental evaluation supports adaptation in complex, fast-moving conditions. Funders, programme teams, partners, and the communities affected each need different things from the evidence.
Clarifying purpose early prevents the common failure of designing a measurement system that answers no one's actual question. A core principle of KMb evaluation is to define goals and expected outcomes before strategies are built, so that what is measured aligns with what success was meant to look like.
- A stated evaluation purpose — formative, summative, developmental, or a deliberate blend
- A map of audiences and the decisions the evidence is meant to inform
- Agreed scope, boundaries, and time horizon
Initiatives that skip this step frequently produce data that satisfies a reporting template but cannot answer whether the work mattered.
Map the Impact Pathway
Making explicit how evidence is supposed to become change
A theory of change, expressed as a logic model, sets out the chain from inputs and activities through outputs to outcomes and impact — and, crucially, the assumptions linking each step. For knowledge mobilization this means naming how reach is expected to lead to uptake, uptake to practice change, and practice change to a downstream effect.
The pathway is what makes any later measurement interpretable: an indicator only means something in relation to the causal story it is meant to evidence. Building the pathway collaboratively with partners surfaces divergent assumptions about how change happens, which is itself a finding.
- An explicit theory of change or logic model for the initiative
- Named assumptions linking each stage to the next
- Identification of the stages at which measurement will focus
Define Evaluation Questions and Indicators
Choosing what counts as evidence at each level of the pathway
Evaluation questions translate the theory of change into things that can be investigated. Indicators then operationalise them across levels — reach, usefulness, use, partnership and collaboration, and practice change — distinguishing outputs such as downloads or citations from outcomes such as changed practice from impact such as population benefit.
Sound indicators are relevant to the question, usable in practice, statistically measurable, and logically defensible. Where validated instruments exist, using them improves comparability and credibility; where they do not, indicators are constructed transparently and their limitations stated rather than hidden.
- A small set of answerable evaluation questions tied to the pathway
- Indicators mapped to output, outcome, and impact levels
- A rationale for each indicator and an honest note of its limits
Select Methods and Causal Strategy
Matching the approach to the question and the conditions
Method follows question. A mixed-method design is the field's default, because qualitative methods capture the social dimensions of how knowledge moves while quantitative methods measure uptake and its relationship to intended outcomes. The causal strategy is chosen deliberately: where a controlled comparison is neither feasible nor ethical, contribution analysis and realist evaluation provide rigorous, theory-driven alternatives.
For outcomes that cannot be specified in advance — as in advocacy, policy influence, or systems change — outcome harvesting and outcome mapping work backwards from observed change to assess an initiative's contribution. The aim throughout is a design proportionate to the stakes and honest about what it can and cannot establish.
- A method mix matched to each evaluation question
- An explicit causal strategy suited to the conditions
- Selected instruments and data-collection approaches
Establish Infrastructure and Baselines
Putting measurement in place before the initiative runs
Outcome- and impact-level evaluation depends on data structures and baselines established before the initiative launches. Without a baseline, change cannot be demonstrated; without agreed data sources and collection routines, later analysis is reconstructed from whatever happens to have been recorded.
This stage also sets the cadence for relational measurement — how partnership health and stakeholder experience will be tracked over time — and resolves the practical questions of who collects what, when, and under what data-governance and consent arrangements.
- Baseline measures captured before activity begins
- Data sources, collection routines, and governance arrangements
- A schedule for repeated and longer-horizon measurement
Because key outcomes may appear only after long time lags, infrastructure should be designed to keep observing after the funded period, or to hand off measurement to those who remain.
Collect and Synthesise Evidence
Gathering and integrating data across methods
Data collection in KMb evaluation typically combines interviews and focus groups, observation and journaling, surveys and validated scales, and administrative or platform data. Qualitative analysis codes and categorises experience to ensure trustworthiness; quantitative analysis measures uptake and its relationship to intended outcomes.
Synthesis integrates these strands against the theory of change, so that numbers and narratives illuminate rather than contradict one another. Attention to partnership functioning and to alternative explanations for what is observed is part of the analysis, not an afterthought.
- Integrated qualitative and quantitative evidence
- Analysis of partnership functioning and contextual factors
- An assessment of rival explanations for observed change
Build the Contribution Claim, Report, and Learn
Reasoning to a credible conclusion and feeding it back
The final stage assembles the evidence into a reasoned account of whether, how, and how much the initiative contributed to the outcomes observed — acknowledging the other influences at work rather than claiming sole credit. Reporting then translates that account for each audience, communicating use and contribution in terms decision-makers and funders can act on, without overstating certainty.
In a developmental posture, evaluation does not end at the report. Findings feed back into the design of the next cycle, closing the loop between what is learned and what is done — which is what turns evaluation from an accountability exercise into a source of genuine improvement.
- A credible contribution story, with limits stated plainly
- Audience-appropriate reporting of use and impact
- Lessons fed back into the next design cycle
Most KMb activity is reported only at the level of outputs and reach. Reasoning credibly to practice change and outcome requires the theory, the baselines, and the analytical discipline established in the earlier stages.
Key Decisions That Shape an Evaluation
No two evaluations are identical. Decisions about purpose, causal strategy, time horizon, indicator depth, partnership, and equity determine what kind of evaluation is appropriate — and what it can credibly conclude. These choices are made deliberately and collaboratively with those who commission and use the work.
Purpose: is this evaluation to improve, to judge, or to adapt?
Formative, summative, and developmental purposes call for different designs. Conflating them — for example, applying a summative frame to an initiative still being shaped — produces evidence that misleads. Naming the purpose first prevents the most common design error in the field.
Causal strategy: can a controlled comparison be made — and should it?
Where experimental designs are infeasible or unethical, the question shifts from attribution to contribution. Contribution analysis and realist evaluation build credible causal claims without a counterfactual, by reasoning explicitly through theory and rival explanations.
Time horizon: how long until the outcomes that matter become visible?
Long time lags between dissemination and application mean the most important effects may fall outside the reporting window. Choosing the horizon, and being explicit about what cannot yet be observed, is more honest than measuring only what is immediately countable.
Indicator depth: outputs, outcomes, or impact — and at what cost?
Reach and usage are inexpensive to capture but weak evidence of change. Practice-change and population-level indicators demand infrastructure and baselines established in advance. The depth chosen should match the claim the evaluation will be asked to support.
Partnership measurement: will the health of the partnership itself be evaluated?
Because knowledge mobilization is carried through relationships, partnership synergy, trust, and equitable participation are often the leading indicators of whether evidence will move. Assessing them early surfaces problems while they can still be addressed.
Equity lens: whose outcomes count, and on whose terms?
Community-level indicators and community-led measurement determine whether an evaluation captures distributional effects or only aggregate ones. Designing the equity lens in from the start is a methodological choice, not a reporting afterthought.
The absence of a randomised design is not an absence of rigour. A well-specified theory of change, evidence gathered against it, serious attention to alternative explanations, and a reasoned judgement of plausibility constitute a defensible method in their own right. The risk is not the lack of an experiment; it is the unexamined causal claim.
Core Competencies
The following competencies form the professional foundation of knowledge mobilization evaluation work. They are not sequential — practitioners draw on several at once depending on role and context. One pattern is worth naming: the relational and causal-reasoning competencies are the ones most often underweighted, yet they most determine whether an evaluation's conclusions are believed.
Theory of change and logic modelling
Making explicit how an initiative is expected to move evidence into use — the chain from activities to outputs, outcomes, and impact, and the assumptions linking them. This is the analytical backbone that makes every subsequent measurement interpretable.
Indicator design and the impact pathway
Constructing indicators that distinguish reach, uptake, practice change, and population-level outcome — relevant, usable, measurable, and defensible. This includes knowing which validated instruments exist and where their psychometric properties are robust enough to rely on.
Mixed-methods measurement
Combining qualitative and quantitative methods so that the social dynamics of knowledge movement and the measurable signals of uptake illuminate one another. This requires fluency in survey and scale design as well as interview, observation, and qualitative analysis.
Contribution and realist analysis
Building credible causal claims where experiments are not possible — assembling a contribution story, testing it against rival explanations, and identifying the mechanisms that produced results in particular contexts. This is the competency that distinguishes KMb evaluation from generic programme evaluation.
Partnership and relational evaluation
Assessing the synergy, trust, and equitable participation of research–practice partnerships — using validated partnership tools where available — and recognising relational health as a leading indicator of whether evidence will ultimately move.
Developmental and utilisation-focused evaluation
Using evaluation as a real-time learning tool in complex, adaptive initiatives, and ensuring findings are designed to be used — by the people positioned to act on them — rather than filed. This closes the loop between what is learned and what is done next.
Equity-centred and community-led measurement
Designing community-level indicators and measurement approaches that ask whose outcomes are counted and whether benefits are shared — including principled, non-tokenistic involvement of the communities whose knowledge and interests are at stake.
Communicating impact to decision-makers
Translating evidence of use and contribution into terms funders, leaders, and partners can act on — conveying meaningful measures of use rather than bibliometric proxies, and stating the limits of a claim plainly rather than overstating certainty.
Implementation-framework fluency
Reading an initiative through frameworks such as the Knowledge-to-Action cycle, CFIR, and PARIHS to locate where uptake is likely to stall — and designing evaluation that measures those barriers and facilitators rather than only the final outcome.
Outcomes
Rigorous evaluation design creates value at two horizons. The shorter-term outcomes are tangible and visible within a project cycle. The longer-term shift affects how an organisation understands, demonstrates, and improves its mobilization of evidence over time.
Shorter-term outcomes: what becomes visible and actionable
| Outcome | What this looks like in practice |
|---|---|
| Evaluation is designed in, not bolted onBaselines, indicators, and a theory of change are in place before launch. | The initiative can demonstrate change rather than reconstruct a story afterwards. |
| Levels of effect are distinguishedReach, uptake, practice change, and outcome are measured as distinct things. | The conflation of activity with impact that weakens most KMb reporting is ended. |
| Credible claims without an experimentContribution analysis and realist reasoning produce defensible conclusions where randomised designs are infeasible. | Findings carry weight with funders and peers, and state their limits honestly. |
| Partnership health becomes visibleRelational strain is surfaced while it can still be addressed. | Problems are diagnosed in time to act, not only in hindsight as the reason an initiative stalled. |
Longer-term change: what becomes embedded in practice
| Outcome | What this looks like in practice |
|---|---|
| Measurement infrastructure persistsThe data structures and routines built for one initiative become a standing capability. | Evaluation no longer starts from zero each time. |
| Evidence use becomes accountable on evidenceValue is demonstrated to funders and leaders through credible measures of use. | The case for mobilization work rests on evidence rather than advocacy or activity counts. |
| Metrics move beyond bibliometricsMeaningful measures of use, interpreted through theory, replace citation counts as the default account. | Whether research mattered beyond academia can actually be addressed. |
| Equity is embedded in what countsDistributional questions become a routine part of how impact is defined. | Whose outcomes improved, and on whose terms, is asked as a matter of course rather than as an addendum. |
The effects that matter most in knowledge mobilization often appear only after long delays and through pathways shaped by many hands. A well-designed evaluation does not resolve this by manufacturing certainty; it resolves it by reasoning transparently about contribution, observing for long enough, and being candid about what cannot yet be known. Honesty about the limits of a claim is part of its credibility.
Organisational Reflection
The questions below are intended to help surface useful considerations about how your organisation currently evaluates knowledge mobilization. They are not a formal assessment. Take your time with them — the most useful answers are honest ones, not aspirational ones. Working through these with colleagues who hold different roles tends to be more productive than working through them alone.
On when evaluation is considered
- When does evaluation currently enter your knowledge mobilization work — at the design stage, or when a report falls due?
- For your most recent initiative, was a baseline captured before activity began — and if not, what could later be demonstrated without one?
- Is it clear, before you begin, who the evaluation is for and what decision it is meant to inform?
On distinguishing levels of effect
- Does your measurement separate reach from uptake, and uptake from practice change and outcome — or does it report activity as though it were impact?
- Where do your current indicators sit on the pathway, and what would it take to add even one credible outcome-level measure?
- Which of your routinely reported numbers actually evidence use, and which mainly evidence effort?
On establishing contribution
- Where a controlled comparison is not possible, how do you currently establish that your initiative contributed to an outcome?
- When you describe impact, do you systematically consider the other factors that could explain what you observed?
- How plainly are the limits of your causal claims stated in what you report — and to whom?
On partnership health and equity
- Is the health of your research–practice partnerships treated as something to be evaluated, or only noticed when it falters?
- Whose outcomes does your evaluation count — and are there communities whose benefit, or lack of it, is currently invisible in your measures?
- Where outcomes may appear only after long delays, what arrangement, if any, keeps measurement going beyond the funded period?
The organisations that make meaningful progress on evidence use tend to be those that create space for honest conversations about what their measurement can and cannot yet show — rather than what they would like it to show.
Citations
The frameworks, methods, and evidence in this resource draw on established scholarship and professional practice. Sources are grouped by the area of the resource they primarily support, and each is given in full so that it can be independently verified.
Graham, I. D., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J., Caswell, W., & Robinson, N. "Lost in Knowledge Translation: Time for a Map?" Journal of Continuing Education in the Health Professions, 26(1), 13–24, 2006. Source for the knowledge creation funnel and the seven-step action cycle.
doi.org/10.1002/chp.47Rogers, E. M. Diffusion of Innovations. New York: Free Press of Glencoe, 1962 (5th ed., Free Press, 2003). Source for adopter categories, perceived attributes of innovations, and the role of change agents.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. "Fostering Implementation of Health Services Research Findings into Practice: A Consolidated Framework for Advancing Implementation Science." Implementation Science, 4, 50, 2009.
doi.org/10.1186/1748-5908-4-50Mayne, J. "Addressing Attribution through Contribution Analysis: Using Performance Measures Sensibly." Canadian Journal of Program Evaluation, 16(1), 1–24, 2001; elaborated in Contribution Analysis: An Approach to Exploring Cause and Effect (ILAC Brief 16), 2008.
doi.org/10.3138/cjpe.016.001Pawson, R., & Tilley, N. Realistic Evaluation. London: Sage, 1997. Source for Context–Mechanism–Outcome configurations and the question of what works, for whom, in what circumstances.
Funnell, S. C., & Rogers, P. J. Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco: Jossey-Bass, 2011. With the foundational statement by Weiss (1995) and the W. K. Kellogg Foundation Logic Model Development Guide (2004).
Earl, S., Carden, F., & Smutylo, T. Outcome Mapping: Building Learning and Reflection into Development Programs. Ottawa: International Development Research Centre (IDRC), 2001.
Wilson-Grau, R., & Britt, H. Outcome Harvesting. Cairo: Ford Foundation MENA Office / Outcome Mapping Learning Community, 2012. For identifying outcomes after the fact and working backward to assess contribution.
Davies, R., & Dart, J. The "Most Significant Change" (MSC) Technique: A Guide to Its Use. 2005. A participatory method for capturing and interpreting unanticipated change.
Kirkpatrick, D. L., & Kirkpatrick, J. D. Evaluating Training Programs: The Four Levels. 3rd ed. San Francisco: Berrett-Koehler, 2006 (model first published by Kirkpatrick, 1959). Phillips (1997) adds a fifth, return-on-investment level.
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., et al. "Process Evaluation of Complex Interventions: Medical Research Council Guidance." BMJ, 350, h1258, 2015.
doi.org/10.1136/bmj.h1258Patton, M. Q. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford Press, 2011.
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. "Bibliometrics: The Leiden Manifesto for Research Metrics." Nature, 520, 429–431, 2015. Ten principles for the responsible use of quantitative indicators.
doi.org/10.1038/520429aGrimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. "Knowledge Translation of Research Findings." Implementation Science, 7, 50, 2012. On moving synthesised evidence into use.
doi.org/10.1186/1748-5908-7-50