What is the best way of measuring the impact your Transition initiative is having, whether social, economic or environmental?
In the excited swirl of starting a Transition initiative, measuring progress isn’t generally top of anyone’s list of priorities. However, finding straightforward ways of gathering data is very useful from early on. Many books on sustainability auditing and measuring impact can prove bewildering to all but the qualified accountant.
As the effect of your projects grows, it will become increasingly important that you document it. Getting into the discipline from an early stage will stand you in good stead for later, as well as providing insights that will help increase your impact.
In the excited swirl of starting a Transition initiative, measuring progress isn’t generally top of anyone’s list of priorities. However, finding straightforward ways of gathering data is very useful from early on. Many books on sustainability auditing and measuring impact can prove bewildering to all but the qualified accountant. Certainly, measuring the exact amount of carbon saved through the endeavours of the initiative is a big job, but there are various ways your group can start to gather data. Let’s start from the easiest and go through to the most complicated . . .
Harvesting simple data from events and projects
Keep a spreadsheet of your events and the number of people who came. Record other easily measurable outputs from projects, such as the number of trees planted, students worked with in the local school, new allotments created and so on. Simple feedback forms after any trainings or key events can check for satisfaction and degree of agreement with particular suggestions. Ask the different working groups to record data that emerges from their work. Also, although strictly speaking it isn’t measurement, keep a photographic record of your Transition process.
Using tools that gather data
Initiatives like Transition Together have great potential for gathering useful data for reductions in carbon emissions. Also, in Transition Horncastle, the group has worked with British Gas and the Energy Saving Trust’s ‘Green Streets’ programme, which gives participating households Smart Meters and other support for energy reduction. One resident who took part stated that it has “revolutionised our lives”. The project has also provided Transition Horncastle with data about what degree of energy use reduction is possible.
More detailed surveys
You may, if resources permit, decide to do an annual survey of the community to establish how opinions have changed, and what effect, if any, you are having. This level of research may be helped considerably with input from a local college or university. Sometimes you may be approached by students wanting to research your work, or you could approach teachers of sustainability courses, or see if any students researching dissertations might want to do such a survey for you.
Some tips for designing survey
- Make it as short as possible: We all have busy lives and better things to do than trawl through endless questions.
- Identify in advance what is essential to know: Why are you doing the survey? What do you want to learn from it?
- Use simple language: Write in plain and accessible English. Avoid complex, lengthy or very abstractly worded questions.
- Don’t use leading questions: Keep the tone as neutral as you can, so your impartiality and scientific method are respected.
- Try it out first: Pilot your survey on a few people first to iron out any glitches, any questions that don’t make sense, etc.
Designing a good questionnaire is an art. We have all been asked to fill in a 15-page questionnaire and not done so. There are a number of good guides to designing good surveys.[i] Far more reliable data is gained by door-to-door surveys than just stopping people in the street. Choose your sample carefully, trying to get as representative a spread as possible across the area. Someone with knowledge of statistics will need to process your data.
[Insert pic: Ing 2.3 - Caption: Completed survey forms in Totnes, part of research conducted by Transition Town Totnes and University of Plymouth, which, among other things, fed useful information into the town’s Energy Descent Action Plan].
Other, less formal, tools
There are also tools for gathering useful data that are more about people’s attitudes and behaviour. This approach is just as useful as the more quantitative ones outlined above. Semi-structured interviews, where an interviewer has a list of topics that are explored through a series of open-ended questions, can be great for, for example, determining what those involved in a Transition initiative, or a particular project, would perceive as its being successful. You might also run some focus groups,[ii] great for brainstorming what people would see as indicators that a project had been successful or failed. You can then return to these points at the end of the project.
Surveys and questionnaires can also be part of listening to your community. For example, Transition Helena, in Montana, USA, has done an attitudes survey as a precursor to any Transition awareness-raising activities. This involves a sit-down, face-to-face interview with key community leaders to find out what they’re already doing and to identify the concerns of the community. The results of this will underpin how Transition Helena formulate their activities.