By Nora F. Murphy Johnson, PhD
In program and evaluation design, the significance of context is often underestimated, if not outright ignored. This oversight, whether intentional or not, can have profound implications. In fields where "evidence-based best practices" are revered, the nuanced realities of varied environments and populations are frequently overlooked in favor of a standardized approach. However, this one-size-fits-all mentality can lead to ineffective, and sometimes harmful, outcomes.
In this blog post, I talk about the critical role context plays in shaping successful interventions and evaluations, and I offer a strategy to make this task more manageable, and I offer a more nuanced, adaptable framework that respects the unique characteristics of each setting and population.
The Consequence of Overlooking Context in Research, Evaluation, and Programming
In the realm of complexity theories, a core tenet is that causality is an elusive concept, not discernible in advance and, sometimes, only somewhat knowable in hindsight. This theory challenges the conventional understanding of cause-and-effect relationships, emphasizing that causal connections and patterns are invariably unique, shaped by intricate interactions within specific timeframes and contexts. This perspective underscores the idea that attempting to predict outcomes or make sense of past events requires an in-depth appreciation of the context involved.
But, in many models for designing programs and their corresponding evaluations, context is often relegated to a mere footnote. This (intentional?) oversight is particularly evident in the field of evaluation, where the emphasis frequently lies on "evidence-based best practices." Such practices, rooted in the belief that there is a singular optimal approach to an issue, advocate for the replication of proven methods. However, this notion is fundamentally flawed.
Consider, for example, a program in Washington DC that successfully prevents middle school students at risk of incarceration from entering the justice system. Its effectiveness leads to it being hailed as a "best practice." Later, an organization just hours away in rural West Virginia, compelled by grant requirements to adopt an evidence-based approach, implements this same program. The starkly different circumstances of the rural West Virginian youth compared to their urban New York counterparts might render the program ineffective or, worse, harmful.
Yes, harm.
When a program designed for urban youth, who have their unique set of challenges and resources, is transplanted into a rural setting with vastly different socio-economic dynamics, cultural norms, and available resources, it might fail to address the specific needs of the rural youth. Worse, it could inadvertently cause harm. This harm isn't just about wasted resources or unmet goals; it can manifest in more profound ways, impacting the self-esteem, cultural identity, and emotional well-being of the individuals involved. In effect, we are setting people up to fail and holding them accountable for that failure.
I would encourage you to remove the term 'best practice' from your list of phrases and instead think of concepts such as 'evidence-informed', 'evidence-based guiding principles', or 'context-sensitive approaches'. These terms emphasize the importance of integrating empirical evidence with a keen awareness of the unique circumstances and variables in each situation, promoting more adaptive and responsive strategies that are tailored to specific needs and environments.
The Two Approaches to Surfacing Contextual: Finite and Infinite
So before we take one thing, and drop into a new setting, identifying meaningful context is a crucial step. And while it may require additional time, effort, and additional resources initially, its importance cannot be overstated. Taking the time to do this well can help you avoid potential harm and ensure effective outcomes.
It can be daunting, I know. Consider these two approaches to contextual analysis to break it down and make the task more manageable.
Finite Context: This method concentrates on gathering detailed knowledge about the specific elements relevant to your program or evaluation. It zeroes in on the immediate, tangible aspects of the context, including the particular people, places, strengths, and concerns pertinent to your area of focus. This level of context is somewhat discernible and can be uncovered through various means such as document analysis, community forums, and journey mapping. It's about piecing together a concrete picture of the current situation, based on specific, identifiable factors.
Infinite Context: Infinite Context: This approach takes a more expansive view, delving into the broader array of people, politics, history, and culture over a longer timeframe. It goes beyond just the immediate scope of your evaluation to explore deeper, more enduring influences that shape the context. This might involve examining historical events and long-term narratives that, while not always explicitly mentioned, continue to impact the present work. Examples include understanding the long-term effects of the Dakota War on land ownership in Minnesota (link) or the legacy of highway constructions in enforcing segregation in Chicago, Illinois (link). This approach asks you to consider the historical and future contexts that invisibly yet significantly influence the current landscape.
Evidence-Based Guiding Principles: An Alternative to Context-Free Best Practices
Continuing with our previous example, while the successes of the New York City program offer valuable insights, translating these practices to a distinctly different setting like rural West Virginia demands careful consideration of local circumstances, cultures, and needs. It's essential to recognize that elements effective in one environment might require significant adjustments to be equally effective in another. This approach underscores the importance of customizing and refining strategies to fit the specific context, rather than relying on a one-size-fits-all model. Developing evidence-based guiding principles, grounded in context, is one way to do this and serves as a robust alternative to the traditional notion of 'best practices.'
Guiding principles, by definition, are not static rules but dynamic frameworks that integrate empirical evidence with deep contextual understanding. They offer a more flexible approach, allowing for adjustments and refinements based on specific community needs, cultural nuances, and situational variables. This method aligns with the philosophy that each intervention must be thoughtfully tailored, recognizing that what works in one setting may not work in another. By focusing on evidence-based guiding principles, we can create more effective, sustainable, and culturally sensitive programs.
But how?
Developing guiding principles for effective program implementation and evaluation should indeed be a multi-faceted process, incorporating a variety of perspectives and sources of knowledge. This often seems daunting to people and organizations I work with, but we can break it down into four sources that can help you gather the evidence needed to develop evidence-based guiding principles:
National and International Evidence Base on the Specific Challenge: Start by examining the broader evidence base, both nationally and internationally, about addressing the specific challenge your program aims to tackle. This involves reviewing studies, best practices, and outcomes from various contexts to understand what has been effective and why. This research offers a foundational understanding of the issue at hand and helps identify potential strategies that could be adapted to your specific context.
Evidence Base on Working with Similar Groups: Next, focus on the evidence base that specifically relates to working with groups sharing characteristics with your target community. This might include demographic factors like age, socio-economic status, cultural background, or geographic location. Understanding the nuances of working with these groups can provide insights into the types of approaches that have been successful or unsuccessful, and why.
Community Knowledge and Insights: One of the most critical aspects of developing guiding principles is to engage directly with the community you aim to serve. Gather knowledge from people in the community about their struggles, hopes, dreams, strengths, and aspirations. This first-hand information is invaluable, as it provides a deep, nuanced understanding of the community’s unique context, needs, and preferences. This step ensures that the guiding principles are grounded in the realities of the people who will be most affected by the intervention.
Experiences and Recommendations from Trusted Community Workers: Lastly, incorporate knowledge from individuals who have worked successfully in these communities and are considered trustworthy and credible by the community members. These practitioners, having direct experience and established trust within the community, can offer practical insights, highlight potential challenges, and suggest strategies that are more likely to be accepted and effective. Their experiences can bridge the gap between theoretical knowledge and practical application.
By integrating these four components, you can develop guiding principles that are evidence-informed, context-sensitive, and community-centered. This approach not only increases the likelihood of program success but also fosters community engagement, ownership, and sustainability.
Conclusion
Every program and evaluation scenario is distinct, shaped by a specific mix of factors: people, politics, history, resources, constraints, values, needs, interests, and even chance. The key to designing a successful program and partnered evaluation lies in meticulously considering these elements. This approach focuses on creating designs that are not only useful and practical but also ethical and accurate.
The uniqueness of each situation calls for a design that emerges organically from these distinct characteristics and conditions. By doing so, it becomes possible to achieve outcomes that are more accurate, ethical, and effective. Moving away from a generalized, one-size-fits-all mindset towards a context-centered approach not only makes evaluations more relevant but also ensures they positively contribute to the overarching goal of fostering a more integrated, beautiful, and just world. This shift recognizes the importance of nuanced understanding and tailored approaches in addressing complex challenges.