How can we measure the impact of community-based activities?
We all want to promote meaningful change—social, environmental, economic, or all of the above. The discourse around impact is expansive and inclusive, encompassing many trends, definitions, and target audience. One can utilize examples from a variety of different sectors: the public sector (government), the business sector, unions, civil society, entrepreneurs, non-profit organizations, and more.
Measuring Social Environmental Impact in Communities
The question everyone is focused on is: where do we start?
In this post, I will provide an overview of the impact measuring ecosystem and its structure, and I will present three tools for social, economic, and environmental impact.
Before we begin, why is it at all important to measure environmental, social, or economic impact?
There are many different reasons, but at the end of the day, they all contribute to the organization’s success:
- Responsibility: allows for providing reporting and feedback based on data
- Fundraising: by measuring impact and demonstrating results, it is possible to show what has been with X resource, and therefore what would be possible if Y resource were allocated for that purpose
- Improving and learning: impact measurement can help increase the quality of activities, and improve outcomes for those interested
- Collaboration: involving other organizations in measurement toward establishing partnerships (financial, social, and inter-sectorial)
- Marketing and communication: data and stories of impact from activities help to promote the organization
- Decision making: impact measurement provides insight that can drive beneficial decision making
Free tools that can help measure your impact—today!
Online surveys: Google forms, Typeform, Survey Monkey
Collecting Data and mapping partnerships: Sheets, Mindmap, Coogle, Airtable, Kobo Toolbox
There are additional data collection tools that are available —which I will discuss in the next article, stay tuned!
To ensure that we are efficiently measuring impact, I will provide an overview of two existing methodologies, each with their benefits and challenges, as with everything in life.
Tool 1: INFOCUS
Infocus, a global consulting agency that deals with impact measurement, proposes a seven-stage approach.
Before we jump into them, a key clarification:
Internal measurement assesses the organization’s focus, relative to time and process depth. On the other hand, external measurement assesses the extent to which the organization is affecting its target audience.
Infocus further sub-categorizes external measurement as it relates to beneficiaries: (1) social coherence (2) peace and conflict mitigation (3) finding employment (4) gender equality (5) people with disabilities (6) sport (7) heath and quality of life and (8) education. Internal organization measurements include (1) organizational development (2) partnership development and (3) system and behavioral change. See the full document here.
Stage 1: Selecting a measurement approach
Includes creating a vision to understand what we’re trying to measure? What is the problem we want to address?
Next, we continue to monitor, evaluate, research, selecting indicators, and identify those responsible for the measurement process. Will it be the organization’s leadership? Volunteers? Consultants? More than one of these groups?
For example, I run an organization that has an application supporting people with visual impairments. The goal is to enable this target audience’s sense of independence, as well as monitor and increase technological developments available to support this audience.
As such, I would choose to measure:
- The number of users who did not have seeing-eye dogs for six months after they started using the application, but beforehand, did have a support dog.
- The amount of time expected relative to the actual amount of time it took to accomplish arrive somewhere on their own.
- The number of Israeli businesses that installed these services on their premises for over six months.
- The type of Israeli businesses (retail, food, transport, academia) that installed the services on their premises for over six months.
- The number of CEOs/senior managers who had previous experience with people with disabilities, who installed the services on their premises.
- The number of businesses that recommended the system to colleagues.
Stage 2: Selecting what to focus on
Using a Theory of Change:
- Inputs: What are we investing in the project?
- Activities: What are we doing to bring about the desired change?
- Outputs: What will change, in the short-term?
- Outcomes: What are the benefits that we want to achieve on a broader scale?
- Impact: What is the goal and what will change in the long-term?
4 questions to keep in mind:
- To what extent were the outcomes a result of my activities?
- What were the outputs?
- What are the interim outcomes of my activities?
- What are the short-term social change (outcome) and the long-term social change (impact) of my activities?
- At what level do we want to measure? On the individual, communal, or societal levels?
Create an organizational thumbprint, prioritize, and formulate research questions, toward constructive conclusions to spur learning and improvement.
An example:
Theory of Change: Elderly Independence through Social Connections
Inputs: Who? Retirement communities
What? Improving their independence through connectivity
Activities:
1. Group meetings and events
2. One-on-one meetings
3. Monitoring
4. Mapping activities (with families and others)
Outputs:
1. Increased number of residents able to participant
2. Increased number of institutions that receive funding
3. Increased number of institutions that see an improvement in 6 months
Outcomes:
1. Pensions and budgets for residents
2. Improved attendance at activities
3. Improved sense of security in the living area
4. Participants utilizing the community network
Impact: Increased sense of independence through connections between the elderly
*Stakeholders: target audience, and affected communities
Stage 3: Developing indicators that measure the problem. To what extent did you achieve the goals in the short and long terms?
Make sure to distinguish between outcome indicators associated with the goals, as opposed to indicators associated with the desired impact of the organization.
Stage 4: Planning of data collection
Questions to keep in mind:
- What are the tools that you need to use? Which tools are appropriate for me? For example, anonymous samples, surveys, and questionnaires.
- Is there a way to be creative in collecting data?
- Is my data quantitative, qualitative, or some of both?
Stage 5: Developing tools to collect data
Make sure to:
- Be aware that we all have biases; try your best to overcome these
- Use existent questionnaires
- Create appropriate surveys (according to what you chose in Stage 4)
Stage 6: Data collection and management
- Prepare and guide staff
- Find useful software
Stage 7: Use, study, and make conclusions based on data
- Create the storyline for the organization: What happened as a result of my activities?
- Prepare and present the data
- Analyze and come to conclusions
*Pro-tip:
You can organize your data ahead of time so that it’s organized according to themes, which I’ll discuss below
- For investors
- For staff evaluation
- For partners and stakeholders
- For the annual report
- For marketing purposes
Advantages: This model targets results, and based on the desired outcome, shapes the activities. Quantitative and qualitative data, as well as questionnaires and outcomes, include many circles of influence.
Disadvantages: Interacts only with theories of change, and is goal-oriented. There are no other methodologies in the ecosystem.
Tool 2: SOPACT
This company leverages capital toward a goal and is a thought-leader in impact investment. They developed The Impact Cloud platform, that allows investors to access sustainable investments across the world.
They suggest 8 stages to measure impact:
Stage 1: Do not start from zero. Find already existing resources.
Stage 2: Create a theory of change,
It ties together evidence with the desired impact.
♥ Start with understanding, what is the problem you’re trying to solve? (like the previous tool).
See further details below.
Stage 3: Develop a matrix – with its own designed indicators
Stage 4: Check the quality of the indicators:
- Are they measurable?
- Are they holistic?
- Are the indicators selected balanced between quantitative and qualitative measures?
Stage 5: Holistic activities:
The content from indicators can be assessed in a variety of different ways. For example, with low income, what does this mean? Does the person have less than $1,000 or more than $10,000?
Even though the wording of the indicators will be rooted in your perspective, it could be that it means something different to others. For example, a young person from a developing country. Therefore, it’s important to be as clear as possible, including creating helpful reading materials—how did you choose it, how we found this data.
Stage 6: Connect to the Sustainable Development Goals
In 2015, 17 global issues were mapped that were to be solved by 2030 to help us and future generations. These challenges address a variety of topics, both environmental and social. For example zero hunger, safe water, biological diversity, access to quality education for everyone, collaboration across sectors, and more. For more information, please read the blog post: Community and Impact; Good deeds, and Good Money, and join the SDG Israel Community.
Note: In the coming series on impact in communities, I will address the SDGs often.
Stage 7: Connect to the IMP
Impact Measurement Project, which created 5 measurable questions for investors, communities, organizations, stakeholders to understand the type of impact, its depth, its contribution, and the potential risks that need to be considered. More about this in the next article.
Stage 8: Synchronize data
Synchronization between sources, stakeholders, surveys, outcomes, and activities, with the indicators that you selected to measure impact. Synchronizing data allows improved tracking of progress, building a resilient model based on consistent evidence and mutual influence between stakeholders. Present data in a way that supports the organization’s mission.
Advantages: systematic, allows for process-oriented expansion of organizational progress. Interacts with existing methodologies.
Disadvantages: Challenging to quantify each stage, except for at the end. Comparisons between organizations need 6 different sources.
Data can become experience
I reviewed two holistic tools for measuring social impact among stakeholders and communities. It’s important to note that questionnaires are a key part of collecting data, but they can also be one-dimensional—feedback before and after a process. As such, make sure to distinguish between questionnaires and gathering results:
- Results in terms of inputs versus outputs. What are the resources that we’ve invested, and what are the outcomes that we received?
- Results are based on a process, whether or not we classify progress, with the understanding of program efficiency and risks.
- Questionnaires do not collect results. As such, it’s important to use defined indicators before beginning. Questionnaires may also be influenced by the past: by past relationships with measurement, by the desire to validate an investment, the composer of questions may be biased, for example by past studies, that are irrelevant to the current process.
- Preference and relevance: collect diverse data as much as is possible. Select data that shows a causational relationship with the problem we want to answer. Choose specific audiences: is it a wider audience or more selective? Recognize your resources before starting.
- Relationships before everything! Impact needs to be relevant to the needs of your organization. Data design and presentation need to portray the complexities of the relationships will all involved actors: with the activity plan, with partners, with the community, with the product/service, with the projects, and with the activities’ participants. In addition to measurement tools, it’s crucial to be cognizant of the pace and ability of the process, your staff, their motivations, the relationships within the community, and how each of these affect the external impact. Select motivated professionals with a personal connection to the matter at hand, who can bring their curiosity alongside their professionalism.
Data can sound daunting, but it also holds immense opportunity. Turn it into an experience that involves those interested, and gives them a say to leave a mark on your organization. Many people we work with, who take part in our activities, have expansive knowledge and potential if we give them the space to take part.
Do you have a tool that didn’t appear here? Do you have interesting insight? Please get in touch!
I am supporting and consulting organizations and businesses in creating profitable communities, leveraging impact measurement strategy. This is the first of a series of blogs on impact measurement and communities. To continue receiving updates on future posts, sign up, or reach out via Ray of Impact.
About the Author
Leave a Reply