S16a · From Systems Thinking to Evaluative Judgement

10:15–11:45 · 3.7 The Studio

Jonny Morell (4.669 Evaluation and Planning) and Liliana Olivia Lucaciu (Lideea Development Actions; EES TWG8)

Evaluators often have to reach conclusions while evidence is uneven, explanations are contested and change is still unfolding. This judgement lab explores how systems-informed thinking can help make conclusions more defensible and useful under such conditions. Through a mix of short inputs, practical examples and participant discussion, the session examines how criteria, alternative interpretations, system relationships and assumptions about the future can shape evaluative judgement. Participants will test and refine claims, consider how uncertainty affects recommendations, and identify practical questions they can use to strengthen their own evaluation reasoning and communication.

S16b · Accountability as Learning or as Control?

10:15–11:45 · 3.7 The Studio

Liliana Olivia Lucaciu (EES TWG8), Kari Korhonen (Finnish Evaluation Society) and Tiina Huvio (Finnish Agri-Agency for Food and Forest Development, FFD)

When does accountability enable learning, and when does it become control? This practice lab treats accountability as something that can help people learn: communities, implementers, commissioners and decision-makers interpret evidence together and act on what they learn. The session draws on a real organisational case from FFD, uses Outcome Harvesting as a practical lens and works with participants’ own dilemmas. Small groups design and stress-test an accountability–learning loop and leave with a first step they can take.

S16c · Evaluation as Sense-Making in Complex Systems Under Uncertainty

13:00–14:30 · 3.7 The Studio

Facilitated by Kirsten Bording Collins (AdaptivePurpose; EES TWG8), with Kecia Bertermann, Pablo Vidueira (Universidad Politécnica de Madrid) and Angelica Fleischer

Complex systems rarely unfold as planned, and different people perceive, understand and value their effects differently. This workshop explores evaluation as shared sense-making, from project evaluation to systems change. Three practitioners share experiences from contexts ranging from humanitarian settings to systems change. Participants then compare what worked and what didn’t, and together build a collection of practical sense-making tips and resources.

S16d · Shifting evaluation design: which systems method?

15:00–16:30 · 3.7 The Studio

Bob Williams (lead facilitator), with Barbara Schmidt-Abbey, Liliana Olivia Lucaciu and Emanuela Carta (EES TWG8)

Many evaluators say they take a systems approach, yet the systems field’s own methods are rarely used in evaluation. This whistle-stop tour introduces three established methods: Soft Systems Methodology, Critical Systems Heuristics, the Viable System Model and Cultural-Historical Activity Theory. It covers what each does and which evaluation questions it suits best. Using the conference itself as the case, participants experience how each method builds a different evaluation of the same intervention. No prior knowledge of systems methods is needed.

S16i · Sensemaking as a deliberate pause between evaluative insight and action

08:30–10:00 · 3.5

Irene Guijt

The session recognises an active form of sensemaking as an essential yet often invisible practice in evaluation. It sits at the threshold between knowing and doing - between analysis and recommendation, between insight and decision. Crucially, it can also be the moment where evaluators and learning facilitators help stakeholders decide not to act: to hold, to wait, to remain in the question rather than rush to closure.

Through an immersion, the session invites participants to experience sensemaking in action: working with a real-world evaluation scenario, moving through rapid perspective-gathering, and practising the deliberate pause before decision. Participants will leave with a sensemaking framework they can apply immediately in their own practice.

S16e · Evaluators as agents of change: navigating power, independence and influence in complex systems – Speaking truth to power!

10:30–12:00 · 3.1 C

Emanuela Carta (Verian; EES TWG8), with Martin Reynolds, Carlos Hinojosa, Daniela Ulicna and Giorgi Davidovi

What happens when evidence challenges power and independence comes under pressure? How can evaluators become agents of change while remaining independent? This session challenges the idea of evaluators as neutral observers, exploring instead how they operate within complex systems of power, relationships and competing interests. This session will explore evaluators as actors “within the system”, rather than observers outside of complex systems. Drawing on systems thinking and a real evaluation case, the session will explore new frontiers by integrating Critical Systems Heuristics (CSH) with Contribution Analysis. It will unpack three provocative themes—agency, independence and non-use—through expert perspectives, practical insights and lively, interactive group work.

S16f · Who Does Evaluation Feed — and Who Does It Consume? A Living Inquiry into Evaluative Practice and Democratic Accountability

13:15–14:45 · 3.7 The Studio

Sebastian Wolf Siebzehnruebl, Shivalika Gupta, Catherine Kelly and Pablo Vidueira

Many evaluators know the moment when the work takes more than it gives, such as when trust built with a community yields stories for a report, but no one ever goes back. This living inquiry starts from that moment. Participants work in small groups of three, where everyone speaks without interruption, before a voluntary sharing in plenary. It is not a panel but a space where every voice in the room is heard. The session is designed for 36–54 participants, so please arrive early.

S16j · Collaboration dynamics and models in evaluation

13:15–14:45 · 3.8

Carlos Hinojosa (OECD) and Adrien Flichy (Quadrant Conseil)

This session goes beyond the usual choice between internal and external evaluation. It introduces a framework of evaluation actors who collaborate along a spectrum from isolation to full embeddedness. Working with a simplified case, groups will design their own collaboration model and weigh its trade-offs: context knowledge, methodological expertise, resources, independence and credibility. A fishbowl discussion then compares the models and the reasoning behind them.

S16g · Learning First! Shifting from Upward Accountability to Systemic Learning in Fragile Contexts

15:15–16:45 · 3.6

Maartje Van Der Meulen, Mariska Van Beijnum, Catherine Flew, Ruby Quantson Davis, Clara Pergola, Reem Diyab and Syed Ali Abbas

In 2023, the Netherlands Ministry of Foreign Affairs created a peace and security grant framework built on 8-year partnerships, locally led adaptive programming and learning at its centre. The donor, an international NGO intermediary, an in-country partner and the learning partner share how practice has changed, including the mistakes. They open up the real dilemmas: centring locally led monitoring, evaluation and learning while meeting accountability requirements, deciding what counts as quality evidence, and building a learning culture that allows experimentation in high-risk contexts.

S16h · Changing with the world: new pathways for evaluation in complex systems

15:15–16:45 · 3.7 The Studio

Fabienne Corvers, Kristina Bayingana (Enabel), and Eva Kunseler and Pia Nabielek (PBL Netherlands Environmental Assessment Agency)

Three institutions share how evaluation is changing, followed by a discussion with participants:

• European Commission: Fabienne Corvers presents the Commission’s expanding toolkit for designing policy and what it means for EU evaluation.
• Enabel: Kristina Bayingana shares the Belgian development agency’s ongoing “Reinvent Evaluation” process, asking what to preserve, what to change and what risks becoming irrelevant.
• PBL: Eva Kunseler and Pia Nabielek share early insights from a navigation guide for evaluating systems change, developed with the Dutch decentralised energy transition monitor.

S16k · How Do Complex System Behaviors Affect Program Theory, Logic Models, Methodology and Data Interpretation?

15:15–16:45 · 0.2

Jonny Morell (4.669 Evaluation and Planning)

The programmes we evaluate are complex systems nested in complex contexts. Yet behaviours such as equilibria, self-organisation and emergence are seldom recognised when programmes are designed or evaluated, which weakens both logic models and methods. In repeated cycles of short input and facilitated discussion, each building on the previous round, participants explore how knowledge of complex system behaviour can change their own programme theory, methodology and interpretation of data.

S17d · Evaluation Ecosystems for Tomorrow

Thursday 29 October · 15:00–16:30 · 2.2

Young and emerging evaluators of EES TWG8: Irina Stavila, Roberta Vancea, Beste Naz Gülen and Alessandra Merlo, with Emanuela Carta and Liliana Olivia Lucaciu

Evaluation systems were built for bounded programmes, linear logic and accountability driven by funders. In the polycrisis, they need to become living evaluation ecosystems: adaptive, diverse and shaped by the interactions between their actors. Drawing on the Three Horizons Evaluation Initiative, this session puts young and emerging evaluators at the centre of that transition. In a mirror debate, experienced and emerging evaluators argue from each other’s perspective, then face live crisis scenarios together. Participants take part through their phones: no panel and no PowerPoint.

The session brings a systems lens to one of Strand 17’s key questions: how young and emerging evaluators can become contributors in democratic societies. It shows how systems approaches connect with the wider conference.