ARBay

Vision: Customers receive in-home consultation when choosing a sofa
Vision: A consultant supports a client from their own workstation

Imagine you need a new bookshelf for your living room. So you go to a furniture store you trust and browse their selection. After a while, you ask for a quick consultation on the spot to get an overview of your options. However, since you can’t quite picture how it would fit with your other furniture, you borrow a pair of augmented reality glasses from the store. You put them on at home and, guided by a chatbot—essentially a digital salesperson—you can select and view different configurations. If you have any difficult questions, you can also contact a store advisor through the glasses. This advisor can then give you tips and advice on which configurations make sense and where you could place the shelf. If you’re already sure of your choice, you can order the desired shelf right away. If you still have some doubts, you can upload the room you scanned with the glasses to a showroom at the furniture store. There, you can place your favorite display items in different spots to help you make the best choice for you.

Based on this concept, the ARBay project aims to develop a system that brings this vision—as well as other future scenarios—to life. To date, the consultation process involved in purchasing large, highly customizable, and high-quality goods such as furniture has been complex and, as a result, has not yet been integrated into the online market. While work is already underway to enable the display of and interaction with comparable goods or objects, HCIS is particularly interested in the technical support for interaction between customers and sellers. In this context, the focus of the proposed solutions is based on augmented reality technologies. HCIS is attempting to explore how interaction and cooperation among multiple people take place and can be supported. In the ARBay project, more specific questions can also be examined in relation to the general research direction:
  

  • What technical and non-technical features are important or helpful for such a support system?
  • How do customers and salespeople communicate with each other? How can this type of communication be supported in an intuitive and user-friendly way?
  • How does the interaction differ when the salesperson is only available remotely?
  • In what ways can objects, positions, or configurations in the real and virtual worlds be referenced?
  • What information or configuration options are needed during the process? Which of these can reasonably be expected of a salesperson? Which ones overwhelm the customer?
  • What as-yet-unknown problems or synergies might arise?

HCIS will investigate these and other questions over the course of the project and attempt to clarify them. To gain an initial overview of consultation and sales conversations, HCIS will, as a first step, accompany salespeople in the furniture stores of project partner tejo and document the consultation and sales processes taking place there. The experiences of customers and salespeople during these interactions are also relevant for gaining a comprehensive understanding and later mapping out a realistic process. This knowledge can then be drawn upon when considering the design of the supporting system’s user interface.
Beyond the scope of the project, HCIS will publish the findings and experiences gained to optimize communication processes in the future.

Current prototype: Customer with a virtual sofa and a consultant in the living room
Current prototype: Configurable piece of furniture designed to fit a space specified by the customer

The ARBAY project itself launched in September 2018 and aims to bring the consultation and configuration of large, highly customizable items—such as furniture—right into the customer’s living room. Supported by virtual and augmented reality technologies and automated chatbots, the goal is to create new digital sales channels. Customers can use chatbots to make an initial selection and then discuss details with a salesperson or seek their advice. The principles of existing sales platforms must be expanded to accommodate this. Specifically, the pilot application developed in the project is to be deployed and tested in the furniture industry. Furthermore, the solution concept will be developed in such a way that, with technical adaptations, it can also be used for other types of goods (e.g., vehicles, etc.).

With support from the Federal Ministry of Education and Research, we are collaborating with various project partners to ensure the best possible coverage of all necessary areas of expertise.

Over the course of three years, through multiple iterations, the current situation will be analyzed, prototypes will be tested, and a final pilot application will be developed.
Within the project, HCIS is primarily responsible for the following work package:
Cooperative 2.5D/3D AR Environment for Consultation, Configuration, and Sales Discussions / HMI

Based on the current state of visualization and interaction concepts, processes must be developed and tested that map interactions between customers and salespeople. By observing and analyzing current sales and consultation processes, a design is being developed and refined that enables interaction that is as seamless and user-friendly as possible with the help of AR technologies. When it comes to user interface concepts, particular attention must be paid to the potential differences between customers, salespeople, and subgroups. While salespeople can receive extensive training on the system and its operation, the solution must be intuitive for customers to use within a very short time after a brief introduction.

Additionally, HCIS is leading the ARBAY project.


Publications:

Blunk, O., Brown, G., Osmers, N., & Prilla, M. (2020). Potentials of AR technology for the digitalization of consultancy-intensive sales processes: The case of furniture sales. Developments, Opportunities, and Challenges of Digitalization, 57–72. Potsdam: GITO Verlag. https://doi.org/10.30844/wi_2020_t1-blunk 

Blunk O, Brown G, Prilla M (2020) Opportunities for digitalization in the consultation of highly configurable and highly customizable goods. HMD Prax Wirtsch. https://doi.org/10.1365/s40702-020-00630-x 

Kahrl, N., Prilla, M., & Blunk, O. (2020). Show me your Living Room: Investigating the Role of Representing User Environments in AR Remote Consultations. Proceedings of Mensch Und Computer 2020, 11. https://doi.org/10.1145/3404983.3405520 

Brown, G. & Prilla, M., “The effects of consultant avatar size and dynamics on customer trust in online consultations,” in Proceedings of the Conference on Mensch und Computer, New York, NY, USA: Association for Computing Machinery, 2020, pp. 239–249. doi.org/10.1145/3404983.3405591

Brown, G. & Prilla, M. (2019). Evaluating Pointing Modes and Frames of Reference for Remotely Supporting an Augmented Reality User in a Collaborative (Virtual) Environment. Mensch & Computer 2019.