Our work with complexity framing
Pragmatica draws on a range of complexity theory approaches in our work. Collaborating with others at conferences offers the opportunity to share new ideas.
Check out some of our recent work which helped surface new thoughts on complexity theory and systems thinking.
Commissioning and contracting to enhance service delivery: learnings from practice and theory for funders, service providers and evaluators
Many in the public sector are being asked to commission and contract in ways that are more flexible and find themselves on the margins. Pragmatica recently facilitated training for evaluators looking into the current context and how we got here to understand why some of the systems and processes that we have work in the way they do.
By putting on a different pair of glasses “complexity lenses” we can see things differently.
By taking some of those complexity ideas we can generate alternative possibilities if our contracting is framed differently
Although we can't change the whole system, we can have influence to get some improvements within the current system.
The training spanned two half days and can be delivered online or in person. For more information please go here.
Contracting public health and social services: Insights from complexity theory for Aotearoa New Zealand
Public health and social services are often hard to specify, complex to deliver and challenging to measure. This research uses a complexity theory-informed lens to explore the challenges and opportunities of contracting out for public health and social services in Aotearoa New Zealand. This qualitative study considers the implications of complexity concepts with ten public sector managers experienced in contracting out for public health and social services.
Recently published findings show that public sector managers are experimenting with different ways of contracting out, yet the underlying New Public Management ethos, which is being applied in many administrative arms of government, can hamper initiatives. There is a growing impetus to find alternative approaches to contract out more effectively. An alternative, complexity theory-informed, framing highlights where changes to contracting out organisation and practices may support more effective service provision. This research also provides insights into why achieving change is hard.
Working with collaborative stakeholder groups
Waikato Regional Council invested in a Developmental Evaluation as part of a collaborative stakeholder process to successfully agree on ways of managing water quality and use for diverse populations. This high-stakes evaluation was a trailblazing, three-year journey working collaboratively with multiple stakeholders in a highly contested setting. Significant for the evaluation field this project demonstrates high quality and effective Developmental Evaluation using an evaluation-specific methodology.
The evaluation team working on the project included Kate McKegg, Judy Oakden, Debbie Goodwin and Jacqui Henry.
This evaluation won the 2019 American Evaluation Association (AEA) Outstanding Evaluation Award.
“For Waikato Regional Council, winning the award affirms that our investment in evaluation for the collaborative stakeholder process was absolutely the right thing to do. …The evaluation award demonstrates to the organization, alongside River Iwi (tribes) and all stakeholders involved in the policy process, the value to be gained from evaluation as a key component of any project. We are thrilled to represent the academic and research expertise that is available in Aotearoa/New Zealand and stand proudly on the world stage and be acknowledged in this way.”
Evaluating in uncertainty: The curse of the wicked problem
Predictability and certainty are hallmarks of traditional evaluation, especially at the program level. We used to believe that we should be able to predict, produce, and evaluate outcomes for all funded programs. Today, exceptions to this rule of predict-and-control are all too familiar. Nonprofit and philanthropic sectors tackle increasingly complex, systemic challenges, where predictability is limited.
Many available options aim to support credible evaluation in complex contexts, including developmental evaluation, outcomes harvesting, and contribution analysis. These approaches are effective, but they may require clients to rethink their evaluation approach. Some clients are not always ready.
In this session, we offered an alternative. We suggested that when evaluators understand the sources of uncertainty in a complex system, they can adapt their evaluation approaches to be more sensitive to complex dynamics, without introducing an entirely new or radical evaluation strategy. In this session, we:
- introduced five sources of uncertainty in complex environments
- explored, with participants, situations where each source might affect the evaluation
- identified evaluation adaptations to accommodate uncertainty as it arose
Glenda Eoyang and Judy Oakden presented this session at the American Evaluation Association Conference in Minneapolis.
Evaluation in complex situations
Drawing on examples from the Sustainable Farming Fund Evaluation this presentation offered practical tips to build usability into an evaluation from the start. The presenters showed how they built use into the evaluation at all stages in the process. This included:
- the scoping, relationship building and contracting stages
- the way external evaluators engaged with commissioners to design a multi-purpose evaluation approach
- the stance taken to project management and client engagement
- planning communication from the outset, which ensured findings were communicated with stakeholders.
Judy Oakden (external evaluator) and Clare Bear (commissioner) presented this paper at the ANZEA Conference, in Wellington.
