Experiment Design

“The map appears to us more real than the land.”
— D.H. Lawrence

Our goal is to generate new data, insights and learnings, which retain their context and help us re-orientate towards a preferable future.

This practice is derived from a mix of action-learning, entrepreneurship and innovation management, and seeks to make explicit what action will be taken and what has been learnt. It is a form of documentation which builds up a written record of learning, which can be shared with team members, stakeholders and other interested parties.

The experiment sheet is a scaffold for the process of:

Designing research activities
Carrying out research and collecting data
Sensemaking and gaining insight
Identifying what needs to happen next

Experiment Sheet Example

This template is based on various sources listed in my original article.

Process

Decide on which assumption from the backcasting exercise we will be testing. We then need to flip it around to become a learning goal.
Once you have a clear goal, it is time to design a research activity. This is the core function of the left hand side of the experiment sheet, which includes:
1. Background is about the context - why we're looking at this learning goal, and perhaps how it intersects with other elements, such as a theory of change, whether it's one experiment run in parallel to others or other types of notes along those lines.
2. Learning/Hypothesis is about recognising the logic/leap of faith we're testing, and introducing some boundaries to the test, such as a date. Depending on the granularity of the experiment, we can use metrics here if quantification is useful.
3. Actions is the specifics of the experiment - Who, What, Where, When, How? Be sure to include not just what you're going to do, but how you plan to evaluate the experiment's outcomes.
In our workshop we filled in this sheet individually, in pairs and as a group. There is no ‘right way’, however sometimes it’s easier for people to write solo, and then combine in other opinions. Each experiment sheet should be a targeted activity - not so large that it tries to answer multiple questions, as the aim is to retain the granular detail of the learning. However a research activity might have several experiments running at the same time, looking for different types of learning - for example, an ethnographic study which looks at user experience in one experiment, and surrounding system factors in another.
Research activities will need to be passed through the existing process of the Research Lab - such as ethics approvals, and funding sign off.
Group reflection: do these research activities serve our project well? What other methods could we have used for these learning goals? Is the activity likely to create good value for our time? How will we generate value for the people involved in the research?

Good Practice

Keep experiments small initially - for example, something that can be done in 2 hours, whilst you’re learning the process.
Try not to integrate. Several experiments may make up a single activity.
Review and critique with critical friends.

PreviousCausality & Assumptions NextExperiment Review

Last updated 5 years ago

Process

Decide on which assumption from the backcasting exercise we will be testing. We then need to flip it around to become a learning goal.

Once you have a clear goal, it is time to design a research activity. This is the core function of the left hand side of the experiment sheet, which includes:

Background is about the context - why we're looking at this learning goal, and perhaps how it intersects with other elements, such as a theory of change, whether it's one experiment run in parallel to others or other types of notes along those lines.
Learning/Hypothesis is about recognising the logic/leap of faith we're testing, and introducing some boundaries to the test, such as a date. Depending on the granularity of the experiment, we can use metrics here if quantification is useful.
Actions is the specifics of the experiment - Who, What, Where, When, How? Be sure to include not just what you're going to do, but how you plan to evaluate the experiment's outcomes.

In our workshop we filled in this sheet individually, in pairs and as a group. There is no ‘right way’, however sometimes it’s easier for people to write solo, and then combine in other opinions. Each experiment sheet should be a targeted activity - not so large that it tries to answer multiple questions, as the aim is to retain the granular detail of the learning. However a research activity might have several experiments running at the same time, looking for different types of learning - for example, an ethnographic study which looks at user experience in one experiment, and surrounding system factors in another.

Research activities will need to be passed through the existing process of the Research Lab - such as ethics approvals, and funding sign off.

Group reflection: do these research activities serve our project well? What other methods could we have used for these learning goals? Is the activity likely to create good value for our time? How will we generate value for the people involved in the research?