Last updated 9 October, 2020 by Judy Oakden
Collaborating with others at conferences offers the opportunity to share new ideas. This page captures some recent presentations which helped surface new thoughts on systems thinking and futuring. It also includes past ideas on the use of rich pictures and approaches for tacking wicked problems which Pragmatica has incorporated into our ways of working in evaluation, research and facilitation.
Evaluating in uncertainty: The curse of the wicked problem
Predictability and certainty are hallmarks of traditional evaluation, especially at the program level. We used to believe that we should be able to predict, produce, and evaluate outcomes for all funded programs. Today, exceptions to this rule of predict-and-control are all too familiar. Nonprofit and philanthropic sectors tackle increasingly complex, systemic challenges, where predictability is limited.
Many available options aim to support credible evaluation in complex contexts, including developmental evaluation, outcomes harvesting, and contribution analysis. These approaches are effective, but they may require clients to rethink their evaluation approach. Some clients are not always ready.
In this session, we offered an alternative. We suggested that when evaluators understand the sources of uncertainty in a complex system, they can adapt their evaluation approaches to be more sensitive to complex dynamics, without introducing an entirely new or radical evaluation strategy. In this session, we:
- Introduced five sources of uncertainty in complex environments
- Explored, with participants, situations where each source might affect the evaluation
- Identified evaluation adaptations to accommodate uncertainty as it arose
Glenda Eoyang and Judy Oakden presented this session at the American Evaluation Association Conference in Minneapolis.
Rich pictures are a Soft Systems Methodology tool that offers a quick and efficient way to work with key stakeholders to better understand their ‘problematical situation’ (Checkland & Poulter, 2006). In a demonstration session called If a picture paints a thousand words: the use of rich pictures in evaluation Judy Oakden offered session participants a chance to try using rich pictures for themselves. You can download the e-book companion from this demonstration workshop by clicking on the title above.
This session was run at the American Evaluation Association International Conference in Denver.
Two webinars on the use of rich pictures in evaluation.
Judy produced two follow up webinars on how to use rich pictures in evaluation.
- University of Colorado 2015 Webinar series: Practical Application of Systems to Conduct Evaluation: Cases and Examples:
- Soft Systems Methodology: The Use of Rich Pictures from Evaluation
- Visit the webinar here
American Evaluation Association: Rich pictures: using an effective Soft Systems Methodology tool in evaluation (available to AES members).
I wasn’t expecting THAT! Imagining future scenarios – and the implications for evaluation
Change is on us – rapid, multi-dimensional change. In evaluation, we often hear people say “what a surprise – I wasn’t expecting that”. Buried in the tyranny of the day-to-day, we have few opportunities to consider the bigger, longer-term picture.
At the AEA conference in Minneapolis we ran a demonstration session where participants explored four possible futures, based in 2051, that were highly relevant to evaluation practice. Drawing from a six-month futuring exercise, this demonstration session provided insights and a new way of framing the future of evaluation. We thought about a wide range of issues – from climate change to population pressures and changing migration patterns, growing food and water insecurity, and the impact of accelerating technology that is all on the way.
During this exercise, we considered the future of evaluation. We shared possible scenarios for evaluation in 2051, considered evaluation practice in these possible futures and encouraged participants to explore options in more depth for themselves.
Judy Oakden along with Human Systems Dynamics founder, Glenda Eoyang, and fellow Associates, Royce Holladay, Wendy Morris, Stewart Mennin and Claudette Webster, ran the session.
Tackling Wicked Problems: evaluative processes that go to the heart of the interrelationships, perspectives, and boundaries to influence the outcomes of complex innovations
Achieving sustainability in agricultural systems is a complex problem. To address this, there is a need to navigate multiple stakeholders’ perspectives and motivations. There are also multiple objectives (economic, social, cultural and environmental) to consider.
To address complex problems, stakeholder engagement is vital. Solutions need to develop iteratively. Facilitators can create the conditions which lead to innovation. Traditional decision-making approaches used in agricultural policy are not so useful.
The presenters shared their experience of undertaking a staff engagement and stakeholder socialisation process. This occurred within a Regional Council. It focused on better understanding the inter-relationships, perspectives and boundaries related to sustainable agriculture. The engagement set out to create:
- the right conditions to support innovative solutions
- the monitoring and evaluation to track progress of the multiple work streams.
Kate McKegg, Judy Oakden and Wendy Boyce presented this paper at the Aotearoa New Zealand Evaluation Association Conference in Hamilton.