How to evaluate XMReality through a pilot project

If your company is considering implementing a remote guidance or remote assistance solution, it’s important to have a clear idea of how you will use it. Since the technology is often completely new to your organization, knowing what it takes to implement successfully can be tricky. Therefore, a great first step is to do a pilot project to thoroughly evaluate that the remote guidance solution you are leaning towards will fit your company and foresee potential internal obstacles.

All you need to know for a successful pilot project

A few key points can help you get the most out of your pilot project. These tips will help you avoid common pitfalls when implementing a remote guidance or remote assistance solution and also ensure that you set your company up for success!

It's all about good preparation

1. Put together your project team

At the start of the pilot project, you must choose a few other project members to help you with the evaluation. A tip is to select people you know who are interested in business development and new technology; they will be your internal champions to help test this in the organization. It’s also good if your project members are some of the intended future users (which ties into the second point to consider before starting).

Most likely, management has an expected outcome for the project; make sure everyone is aware of that outcome and knows why this project is important for your company. Also, set clear roles and responsibilities in the group so everyone knows who’s doing what.

2. Identify your use case(s)

For the pilot project to be successful and bring value, you must ensure that you have a clearly defined use case(s) - not just for the pilot period but what you intend to use it for after implementation. You probably already knew about use cases when you started looking into different remote support solutions, so this shouldn’t be too hard! But make sure that the use case is aligned with your team members so you agree on what you will test.

Pro-tip for you to identify your potential use cases:
If you need more inspiration on use cases, you can read this article, “5 typical use cases for remote guidance”. It’s always good to start by considering what existing pain points you have at your company - it can be both for internal processes or external facing processes, such as interactions with customers or suppliers. A typical pain point is a step of a process that is not working well (low efficiency, high cost, time-consuming, etc.), and the next step is to understand why it’s not working well. Depending on the root cause, you then determine if remote support software is the solution or part of a solution to the problem.

Identify your use cases-1

3. Decide how to measure success

Once you have the use cases defined for the pilot, you have one challenging thing left to do - but this is also one of the most important things: agree on how you measure success and define those KPIs:s! To ensure that remote guidance brings value to your company, you'll need to know the added value you want to create in each use case. You also need to know how to follow up on those KPIs:s.

A good place to start is to look at what KPIs:s you are working with today and what data you have access to. What you would like to ensure that you, by the end of the pilot period, have clear results for each use case: did using remote guidance contribute to the value you aspired for?

Help with setting goals for the project.
When considering setting the goals for your pilot project, it’s always good to fall back on the ‘SMART’ goal methodology. Your plan needs to be Specific, Measurable, Achievable, Relevant, and Time-bound. You can read more about how to set SMART goals here.

Start the evaluation period - and gather input

Once you start the actual evaluation phase, you should have a clear start and finish date for the project. It’s also good to have a few milestones throughout the project so you can monitor progress. Here are some tips for making the most out of your evaluation phase:

Training - make sure that all the users in the project are familiar and comfortable with the software. Otherwise, the project will always stay on. Make it clear whom they should turn to if they need help.
Regular interviews - deep-dive interviews with the pilot group's most and least active members, designed to provide deeper insights into the user experience.
Admin portal statistics - gathering user data through the admin portal, providing information such as duration and volume of calls.
Ongoing documentation - hopefully, you have decided how to measure success, but the next step is to ensure that these measurements are carried out throughout the project. Think of a process to gather this data during the evaluation period.
Final survey - a detailed survey sent to all pilot group participants, gauging experiences and potential new use cases to provide clear conclusions as to the value of the product or service

Leadership, preparation and conclusion

Best practice - the three rules

To summarize, these three key factors are critical to getting the most from your pilot project:

Leadership - You need one person to ensure that the team follows the protocols, uses the product correctly, and reports back their experiences (both good and bad) in a format that can easily be collated for later review. Have a single point of contact who takes ownership of the project.
Preparation - We have already covered this, but we can’t stress the importance enough! Before the project even starts, identify the key stakeholders and, together with them, agree on what problems you are trying to solve with this project. Secondly, establish what success would look like - What would be an ‘acceptable’ improvement? What would be an ‘excellent’ one?
Conclusion - Once you’ve gathered all the data and feedback, have a clear plan for the next step. Have a roadmap for applying the knowledge you’ve gathered, either in refining your product/service or looking at other areas of your business that could provide value.

XMReality has helped several of our customers with pilot projects to evaluate our software, and every project has been unique in content and period. You can read about one of these pilot projects we did with a manufacturer of overhead cranes and get some insights on what they thought were the key success factors - and some surprising discoveries as well!