Mulit-input interaction; handling CSV within Unreal

Exploring Multi-Input Interaction Between Arduino and Unreal Engine

After getting the basic serial communication between Arduino and Unreal Engine working, I wanted to push the setup further — not just sending a single stream of data, but handling multiple inputs at the same time. The idea was to move from simple one-sensor control to a system capable of reading different kinds of data and using them to drive changes in real time within a virtual environment.

Phase One: Connecting and Testing with a Single Input

I started by connecting a potentiometer to the Arduino and sending its values to Unreal through the serial port. Once I confirmed the data was arriving correctly, I used it to control a visual parameter inside Unreal — for example, adjusting the brightness of a light in real time.

This first step was crucial for understanding how Unreal’s serial communication works and how data from Arduino could be mapped to environmental changes. It also helped me get familiar with how to handle Blueprints, Unreal’s visual scripting system.

Phase Two: Managing Communication Between Blueprints

Once I had data coming in, I needed a way for different Blueprints to share and use that information.

At first, I tried direct Blueprint referencing, manually linking one Blueprint to another. This works but isn’t scalable — especially when more Blueprints need to access the same data.

So I moved to a better solution: using a Game Instance or Game Mode Blueprint to store data globally. This means that values coming from the serial communication Blueprint can be saved in a shared space and then retrieved anywhere — even across different levels.

In my setup:

• The Serial Communication Blueprint reads data from the Arduino.
• The Game Instance Blueprint stores the values globally.
• The Animation Blueprint retrieves those values to control animation states.

For example, if the sensor value goes above a certain threshold, the animation Blueprint triggers a specific motion.

This made the system modular and ready for more complex interaction.

Phase Three: Moving to Multi-Input (Potentiometer + Capacitive Sensing)

Next, I wanted to add another input — a capacitive sensor.
This meant Arduino would now send two separate values:

1. The potentiometer reading
2. The capacitive sensing value

Both needed to be read by Unreal, processed, and used independently.

On the Arduino side, I updated my sketch to send the two readings in one line, separated by a comma (,)

  // --- Send both readings to Unreal Engine ---
  // Format: potentiometer,capacitiveSensor
  Serial.print(mappedValue);
  Serial.print(",");
  Serial.println(capValue);  // Sends e.g. "128,2450\r\n"

Each line contains both sensor values.

Inside Unreal, this required some changes in the Blueprint logic. Previously, Unreal expected just one value per line. Now, Unreal had to:

1. Read the full line as a string.
2. Split that string using the “Parse Into Array” node.
3. Use the comma as the delimiter (because that’s how the Arduino sends data).
   
   

This function breaks the string into separate elements in an array. For my setup:

• Index 0 contains the potentiometer value.
• Index 1 contains the capacitive sensor value.

Phase Four: Parsing and Converting Data in Unreal

After parsing the string into an array, I used two local variables inside the Blueprint:

• PotValue (for the potentiometer)
• CapValue (for the capacitive sensor)

From the array, I Get the value at index 0 and index 1.
Since Unreal receives them as strings, I then use String to Float to convert them to numbers.

Finally, I assign these values to the local variables. This gives me two live, numerical data streams from Arduino — both updating in real time inside Unreal.

Reflections: Building Toward a Digital Twin

This multi-input setup opened a door to more complex, multimodal interactions — what could be considered the foundations of a digital twin system. Instead of a single control parameter, multiple sensors can now inform and influence the virtual environment simultaneously.

Whether it’s lighting, animation, or physical simulation, each input can be mapped to different components in Unreal. And because the data is handled globally, it’s easy to expand: adding new sensors or mapping them to other behaviours is just a matter of extending the parsing and assignment logic.

In essence, this phase marks the shift from a simple Arduino-to-Unreal bridge to a flexible, scalable communication system that supports multiple real-world inputs and real-time virtual responses.

Arduino sketch, potentiometer and capacity sensing as inputs.