User feedback and user experience studies are essential tools to turn innovative ideas into solutions that improve people’s everyday lives. However, in the field of transportation, creating even a simplified laboratory setup for evaluating new concepts can be prohibitively expensive, tying up significant portions of a project’s budget. Virtual reality offers a way to overcome this challenge by enabling realistic and highly immersive evaluations without relying on physical prototypes.
From the gaming PC to the lab
Fortunately, technologies, originally designed for PC gaming, are now leading a pathway to circumvent this roadblock. Over the last decade, huge strides have been made to make game engines such as Unity or Unreal more accessible to non-programmers through visual scripting interfaces. In addition, companies such as Meta pushed powerful VR development-kits to the market, enabling seamless integration of tools such as hand and gaze tracking. While VR has yet to achieve mainstream commercial success, the technologies developed in its pursuit have proven surprisingly valuable for research applications. Together with the increasingly popular open-source 3D design software Blender and the ever-growing catalogue of available assets, this creates an ecosystem where researchers can rapidly develop and deploy photorealistic worlds to present their ideas in a highly immersive manner.
From first prototype to final interior
Not only does using a 3D modelled VR world significantly lower development costs, but it also speeds up development cycles considerably. This enables real-time adjustments during testing and reduces even extensive redesigns to a matter of days.
Using virtual environments also holds further advantages beyond the cost and speed factors. In many cases it is possible to deploy the testing environment directly on the VR device (or in cases where this is not possible it can be deployed on a modestly powerful laptop) which makes the whole setup highly portable. The entire testing setup can be carried around in a backpack and test sites can be set up with almost no preparation anywhere.
Unity Editor showing the VR interface from a wheelchair user’s perspective
In the case of the Karlsruhe demonstration site, using real automated shuttles was never an option since all the costs associated with such a vehicle would have been unbearable in the scope of this project. Instead, we opted to implement a VR environment in the Unity engine. Relying on available assets, we rapidly deployed an initial test environment and implemented the UI interface for our user tests.
Initial interior of the autonomous bus asset
Upon evaluation we then found that the chosen design had several weaknesses. It was too bulky and during height adjustments it would have collided with the railing, and it was also partly blocking the door. For those reasons it did not fulfil its intended purpose of making the interface more accessible. One display was removed and a second lower one introduced on the opposite side of the door. In a final design step, upon feedback from our focus group study, a pair of displays was added to the ceiling of the vehicle to provide information about the journey, as is commonly known from buses or trams.
Final interior design
Challenges and Difficulties of using VR for the evaluation of CCAM services
Accessibility is a key topic in the context of CCAM, and, especially in the scope of the CulturalRoad project, VR allows participants with physical impairments to participate in experiments that would otherwise be hardly accessible for them. It also allows an ordinary user to experience an experiment from the perspective of, for example, a wheelchair user.
Furthermore, by implementing hand tracking, users can interact with the UI in a more natural manner without relying on traditional controllers, further lowering the barrier to entry. This was surprisingly straightforward during development, since the hardware manufacturer and the engine developers are now providing out-of-the-box toolkits to implement such features with minimal development effort.
However, VR is also associated with significant drawbacks. For one there is the issue of motion sickness, a condition caused by a mismatch between perceived and felt motion, leading to dizziness and nausea. For the Karlsruhe pilot we therefore took several measures to minimise motion sickness, such as restricting movement to the inside of the shuttle and choosing a high-quality VR device with excellent stabilisation and a very high refresh rate.
There are also obvious issues related to participants with visual or neurological impairments. Unfortunately for those groups VR is often not a viable option in its current state. It is therefore important that researchers take these limitations into consideration when they are implementing VR technology, especially in the context of accessibility or social acceptance.
Democratising research with VR
VR technology has matured to a point where it is no longer the exclusive domain of large industrial players with large budgets. By combining low development costs, rapid iteration cycles and a highly portable setup, it opens the door for research that would previously have been out of reach for smaller projects with limited funding. While VR is not a silver bullet, accessibility constraints remain a real challenge for certain participant groups. Nevertheless, its advantages make it a compelling tool for user experience research in the context of CCAM and beyond. As VR hardware continues to improve and its ecosystem grows, these barriers are likely to diminish further, making it an increasingly attractive option for researchers across disciplines.
Illustration of the initial bus design compared to the result as modelled in Blender
