When to make the product available to the first users?
Launching a new product is a complex process with many phases. Research and user testing are required at each phase, especially when dealing with services that are complex for customers, such as using a credit card or biometric identification.
At the very beginning of the project process, we go through a discovery phase where we use qualitative methods such as in-depth IDI interviews or ethnographic research to define the customer’s needs and problems as accurately as possible.
The next step is to design the solution and build the initial prototypes, which we then usability test to iron out any kinks before implementation.
We refer to the Early User Programme as the point at which a project has implemented key functionality, passed internal testing and is ready to be opened up to the first external users to validate previous assumptions.
How to prepare for the first step?
Workshop
Planning for the early user phase of the programme starts with collecting the hypotheses we want to test. Again, the user paths and tasks we want to test need to be well mapped out. An important point here is the workshop with the product team, where the objectives are refreshed, structured and accepted by the whole team.
It is also a good idea to appoint a coordinator from the outset to help manage the test plan and communication within the team.
Feedback
The next step is to design the feedback collection moments. How can you make it easier for testers to give feedback? When should you ask them for their first impressions so that their answers are as reliable as possible?
For products with complex processes, it is a good idea to break the survey into smaller modules and ask users about their impressions in a contextual way (e.g. with short surveys) after they have completed the registration path, so that they remember as much as possible.
The tool used to collect feedback must also allow early adopters to easily and quickly attach photos, screenshots and descriptions – this is very important for the team to accurately diagnose the cause of the problem in the future.
Communication channels
An important stage in the planning of the Early User Programme is the proper preparation of the communication channels associated with the product.
These may include:
- a dedicated helpline,
- a dedicated email,
- a dedicated chat room,
- a others, depending on the tool.
Sometimes first-time testers reflexively contact the company via the official hotline when they have a problem.
During the Early User Programme we should instruct both the testers how to contact us and the company’s employees well in advance to prepare them for collecting spontaneous feedback.
It is particularly valuable to us as it often contains the most pressing issues. Well-captured and processed feedback is critical to the success of this stage, so it is worth taking the time to design this step.
Am I in the “beta” version?
When planning the work, it is also important to consider the issue of visual labelling of the application or its elements, so that users can clearly distinguish between finished functionality and that which is still in the ‘work in progress’ phase. Elements such as “beta” or “test version” tags or additional messages in the application layer can work well here.
User group recruitment
We need to be particularly careful here – sometimes we think of first-time external testers in the context of Friends and Family, which immediately suggests the selection of a particular group of people.
This is not always a mistake, but this approach, while tempting in terms of accessibility, has a downside – can we rely on an objective assessment from our enthusiasts or employees’ families? Focusing on recruiting people who match the target group will give the best result.
A good idea when recruiting is to create a dedicated landing page to help collect and process sign ups from interested testers, as well as promoting the product.
We also use our extensive respondent database to organise the Early User Programme phase.
If you have an existing application and are implementing a new service into it, you can display the invitation to participate in testing as a message in the application to a selected group of customers.
It is important that the rules of the test are clear and transparent, and that first-time users are aware that their feedback will be used to improve the test version.
Execution of the tests
After recruitment, the first users are given access to the product. It is crucial to check the communication channels that have been mapped out in advance and to monitor the responses that come back from the first surveys.
The data will be tagged and categorised accordingly. During this time, the project team should schedule regular meetings to discuss the initial feedback, which usually comes in after a few days. This is the point at which we can satisfy our initial curiosity, but it is also necessary to manage communication within the team appropriately so that we do not jump to conclusions.
Finish testing
The collected material is compiled into recommendations and presented to the product team. At this stage, a further workshop should be organised to prioritise the changes to be made and identify ‘no go’ factors before the official product launch.
This is the most important moment at this stage. Key findings from testing need to be translated into specific tasks for the design and implementation team to ensure that the issues mapped during testing are correctly interpreted for the wider customer base.
During the workshop, Edisonda’s specialists will help to assess the seriousness of the problem and draw up an action plan so as not to hinder the work and to remove as many doubts as possible.
What happens to the testers after Phase 1?
Once the tests are complete, the first users should be given the choice of continuing to use the product and keeping in touch with the company, or abandoning it for the time being.
It’s very important to realise that your product now has a base of real, external users, so any changes need to be made carefully.
What to consider when planning an Early User Programme?
- A common mistake at this stage is to focus on the technical performance of the product, when the fundamental value of this stage is to test whether customers understand the service and whether it responds well to their needs.
- The team should be confident that the functionalities are ready for testing – checking reactions to those we know don’t work as they should is not a way to reliably test the impact of first impressions.
- Testing without working communication channels wastes the potential to test all interactions of the service. For example, if we don’t ensure that a text message reminder of an upcoming payment deadline is implemented in time for testing, we can’t be sure it will work in the future.
- The testing phase should not end with recommendations, but with a workshop to plan the specific actions needed for implementation before the official launch.
What are the benefits of the Early User Programme?
A well-run Early User Programme phase will allow you to:
- Identify user issues early and plan for improvements,
- Minimise the risk of negative comments when presenting the product to a wider audience,
- prepare for feedback,
- evaluate usability against previous prototypes,
- test configuration of analytics tools,
- test the effectiveness of different customer communication channels.
What’s more, a product that is well prepared at this stage can gain a base of ambassadors for the solution, as well as a group of testers who can be approached in the future when new features are introduced.
You may also discover problems that were not apparent in earlier prototype testing. For example, in one of our projects it turned out that a key reason for the negative feedback from early users was that the application was too slow.
This helped to reprioritise the work and speed up a number of improvements.
Support from EDISONDA
Running the Early User Programme phase is a logistical challenge, often involving several company departments, so consider specialist support in planning the key steps and selecting the right tools.
If you decide to work with EDISONDA, you will benefit from the Early User Programme through
- Preparatory workshops, mapping of objectives and key user paths,
- support in designing feedback collection moments,
- assistance in selecting appropriate feedback collection tools,
- a report with recommendations,
- workshops to plan priorities for change and improvement.
In addition to checking that everything works as it should from a technical point of view, you also get the chance to test your hypotheses in a real environment, observing real interactions with the product.
