LUIDA

    Large-scale Unified Infrastructure for Digital Assessments

    A Large-Scale VR/HCI Experiment Framework for Metaverse Platforms

    Overview

    LUIDA is an integrated framework for efficiently conducting VR experiments on metaverse platforms.

    With LUIDA, anyone can easily participate in experiments in the metaverse, and researchers can conduct experiments efficiently. For example, psychological experiments and VR experience surveys that previously required visiting a laboratory can now be joined from home using a VR device connected to the internet. For researchers, the entire process from participant recruitment to data recording is automated, enabling the collection of large-scale data in a shorter time. This system is expected to facilitate large-scale experiments and surveys targeting diverse populations that were previously difficult to conduct.

    LUIDA was jointly developed by Cluster Metaverse Lab, Cyber Interface Lab at the Graduate School of Information Science and Technology, The University of Tokyo, and Hiraki Laboratory at the Faculty of Library, Information and Media Science, University of Tsukuba.

    Key Features

    Web Console for Experimenters

    A web interface for managing experiments, including registering experiment information, configuring questionnaire content, and viewing and downloading collected data. Through this console, researchers can manage publication settings, participation requirements, and monitor data collection in real time.

    Metaverse Space for Participants

    The space for experiment participants consists of two parts: a "Recruitment World" for finding experiments, and an "Experiment World" where experiments are conducted.

    • Recruitment World: Experiment participants gather in this world and browse the bulletin board to find experiments of interest. The design resembles a tavern to lower the psychological barrier to participation.
      Visit Recruitment World
    • Experiment World: When a participant joins an experiment, an independent experiment world instance is automatically generated for that session. This ensures participants can complete experiment tasks without interference from others.

    Low-Code Experiment World Development

    Experimenters build experiment worlds using Unity-based "implementation templates." These templates provide a GUI-based interface for variable management, state machine-based experiment flow control, and data collection format definition, allowing researchers with limited programming experience to design complex experiments.

    Automated Data Collection and Management

    During experiments, behavioral data such as user avatar position and posture can be recorded using Cluster Script features. Custom data such as task scores defined by researchers can also be collected and stored within the experiment world. Collected data is stored in a database with anonymity ensured, and can be viewed and downloaded from the web console at any time.

    Advantages of LUIDA

    • Large-Scale Parallel Experiments: Independent experiment world instances are automatically generated for each participant, eliminating the need for researchers to individually manage participants and enabling simultaneous execution of numerous sessions.
    • Efficient Participant Recruitment: Available experiments are listed in a dedicated "Recruitment World" within the metaverse, where users can freely participate, greatly reducing recruitment effort.

    Researcher Usage Guide

    1. Step 1: Register Experiment on Web Console
      First, access the web console and register the basic information for your experiment (title, description, participation requirements, compensation, etc.) and the consent form. You can also configure the questionnaire questions and options to be used during the experiment.
    2. Step 2: Develop Experiment World with Templates
      Download the Unity-based "implementation template" and create the experiment world. Using the template's GUI, link questionnaires registered in the web console by ID, define data recording formats, and automate experiment flow using state machines.
    3. Step 3: Upload and Publish Experiment World
      Upload the completed experiment world to cluster. Then return to the web console and switch the experiment's publication status to "Published." This will display your experiment on the Recruitment World bulletin board, starting participant recruitment.
    4. Step 4: Data Collection and Analysis
      Once the experiment begins, data is automatically saved to the server. Researchers can monitor collection progress in real time on the web console. When the target number of participants is reached, stop recruitment (switch the experiment's publication status to "Testing" on the web console) and download the collected data for analysis.

    Limitations

    LUIDA currently has several known limitations. We continue to work on research and development to address these issues. Below are the main current limitations.

    • Platform Dependency: Currently, the LUIDA framework is built exclusively on the metaverse platform "cluster," so participants are limited to cluster's user base. There are also constraints due to cluster's specifications (e.g., lack of finger tracking support).
    • Network Latency: As a network-based system, variations in data recording timing may occur, which is a limitation for experiments requiring millisecond-level precision. LUIDA records timestamps, making it possible to identify the impact of latency.
    • Device Variance: Since participants use different VR devices, performance differences between devices may affect experiment results.
    • Session Interruption: Currently, session recovery when a participant exits the experiment world is not supported.

    Notices

    • LUIDA is released as an experimental framework and does not guarantee stable service operation.
    • The developers and their affiliated institutions bear no responsibility for any losses or damages arising from the use or inability to use LUIDA.
    • LUIDA may change specifications, update, or discontinue service without prior notice. The developers and their affiliated institutions bear no responsibility for any losses or damages resulting from such changes.
    • When conducting research using LUIDA, please comply with applicable laws and regulations of your affiliated institution. Additionally, please obtain ethics review approval and inform participants of the approval number.
    • If your experiment involves the use of participants' personal information, please obtain consent from participants and use the service at your own responsibility in accordance with personal information protection laws and other applicable regulations.
    • For other issues arising from the use of LUIDA, the cluster Terms of Service shall apply.

    Acknowledgments

    LUIDA was developed as part of the Cabinet Office/JST Moonshot R&D Program Goal 1 "Development of Cybernetic Avatar Technology and Social Infrastructure for Creating Physical Co-creation" (Grant number JPMJMS2013).