To connect the microcontroller with the game engine, a serial communication object is created with a baud rate that matches the value set in the Blueprint of the plugin. Data is retrieved from the Arduino using the analogRead() function and sent to Unreal Engine via Serial.println(). In Unreal, the data is received and read using the Serial Read Line function after a successful connection to the serial communication port.
For the prototype of the technical implementation testing, I connected an LED controlled by a potentiometer to an Arduino, which sends the mapped potentiometer data to Unreal Engine via serial communication.
Arduino LED code.
LED connected to the PIN 3 Potentiometer input on A0, performs an Analogue Read from A0pin and stores it in the sensorValue variable The mapped value is sent from Arduino
Serial.println :
“Prints data to the serial port as human-readable ASCII text followed by a carriage return character (ASCII 13, or ‘\r’) and a newline character (ASCII 10, or ‘\n’). This command takes the same forms as Serial.print().”
Referencing allows for communication between blueprints, the sender and the receiver.
To enable communication between two Blueprints in Unreal Engine — for example, a sender (like a serial communication Blueprint) and a receiver (such as a Spotlight Blueprint) — I followed the steps below:
I saved both Blueprints in the same directory to ensure that one could easily reference the other.
I placed both Blueprints in the World View by dragging the sender and receiver Blueprints into the level so they existed as instances in the world.
Next, I created a reference variable in the Sender Blueprint (the one responsible for sending data). In the Variables panel, I clicked the + icon to create a new variable and changed its type to the class of my Receiver Blueprint (for example, BP_Spotlight). I named this variable ReceiverRef to keep it meaningful.
After that, I made the variable editable. With the variable selected, I went to the Details panel and checked Instance Editable (the small eye icon appeared next to the variable name). This allowed me to assign a reference to the variable directly from the Level Editor.
In the level, I selected the Sender Blueprint instance and, in the Details panel, found my editable variable under the Default section. I then used the dropdown menu to assign the Receiver Blueprint instance (for example, the Spotlight Blueprint) that I had placed in the world.
Finally, inside the Sender Blueprint, I dragged the ReceiverRef variable into the Event Graph and used it to call functions, set variables, and trigger events in the Receiver Blueprint.
So basically, I have creates 2 blueprints. The first one handles serial port communication. 1st the serial communication plugin that I outsourced from GitHub, but then I’ve modified it. I created a custom event that reads data coming from the Arduino as strings, converts those strings into floats, and then maps the range of values (from 0–255 on the Arduino side) to a new range of 0–100,000 in Unreal Engine. This mapped value represents the light intensity inside Unreal.
Using that data, I update the light’s intensity in real time, making the digital twin of a red LED in Unreal Engine. I managed to get the two blueprints communicating with each other by referencing the red light blueprint inside the serial communication blueprint.
Next steps:
Now, I’m going to be working on a new blueprint for the character animation. I will be working with Unreal state machine and transition state that will be initialised with Arduino data. I want to use the Arduino data again to control this transition — for example, switching from idle to walking when the input value crosses a threshold.
Let’s say if the Arduino’s light intensity value is greater than 100, I want the character to transition from idle to walking. In other words, the character’s animation changes dynamically based on the real-time Arduino input, reflecting user interaction and sensor data in the scene. This will be a real-time interaction of predefined animation states.
To bridge Arduino and Unreal Engine, I used a plugin originally made for Unreal Engine 4 and later adapted for Unreal Engine 5. The plugin is called Unreal Engine SerialCOM Plugin, developed by Ramiro Montes De Oca (Ramiro’s Lab). I sourced it from GitHub: https://github.com/videofeedback/Unreal_Engine_SerialCOM_Plugin
I’m running Unreal Engine version 5.4.4, which meant I had to rebuild the plugin version for 5.0.3 available in the resource folder, to make it compatible with this version. It worked smoothly after a few configuration steps.
Setting It Up in Unreal
Once I downloaded the plugin ZIP from GitHub, I followed these steps:
Create a Plugins Folder Inside my Unreal project directory, I created a folder named Plugins.
MyProject/Plugins
Place the Plugin Inside I extracted the downloaded plugin folder directly into this Plugins directory — not inside the /Content folder.
Restart Unreal After placing the plugin, I closed the project and reopened it. Unreal automatically detected the new plugin and asked to rebuild it for version 5.4.4. I let it rebuild, and everything compiled correctly.
Enable the Plugin In Unreal, I went to:
Edit → Plugins
Then I searched for “SerialCOM,” ticked the checkbox, and made sure it was active. Once enabled, I restarted Unreal one more time to apply the change.
Setting Up the Connection
The plugin comes with a Blueprint example inside the ZIP package, which made the setup process much easier.
Here’s what I did:
Open the Example Blueprint I added the provided Blueprint to my level and opened it.
Find the Right COM Port I checked which port my Arduino was connected to by opening the Device Manager on Windows and looking under “Ports (COM & LPT).” For me, it was something like COM3.
Update the Blueprint Settings In the Blueprint, I changed the serial port to match my Arduino’s COM port and adjusted the Baud Rate to the same value I had used in my Arduino sketch.
Test the Connection Once everything was set, I ran the project in Unreal. The connection status appeared in the window, confirming that Unreal was successfully talking to the Arduino.
What Worked and What Didn’t
It wasn’t all smooth sailing — there were a few hiccups along the way:
Plugin detection issues: Unreal wouldn’t recognize the plugin at first because I’d placed it inside the /Content folder by mistake. Moving it to /Plugins fixed that instantly.
Rebuild prompts: Unreal had to rebuild the plugin every time I changed versions or made small edits, but it eventually stabilised.
COM port mismatch: If the wrong port or baud rate was set, the communication failed silently — double-checking these two settings was key.
Port Initialised in debug view, confirming Arduino successful connection with Unreal.
Final Thoughts
Getting Arduino and Unreal Engine to communicate felt like unlocking a new level of interactivity. The SerialCOM Plugin by Ramiro Montes De Oca works really well once it’s set up properly, and it opens up so many possibilities for combining physical hardware with virtual experiences — from motion sensors and haptic devices to robotics and interactive art installations.
This process taught me a lot about how Unreal handles plugins and serial communication, and now that the setup is working, I can start experimenting with sending real sensor data directly into my Unreal projects.
My work bridges the physical and digital worlds, exploring the transition between real-life (IRL) and virtual reality (VR) events. These two domains are often seen as opposites — the virtual world as an alternative to the real one — a juxtaposition of the real and the non-real, the real and the fake, the real and the surreal, the physical and the digital, the real and the imaginary, colliding in complex ways.
The motivation for this project was to create an experience through which the audience can discover that actions and experiences in VR can have real consequences, as David Chalmers argues in Reality+ (2022). The physical and the virtual are thus connected, rather than distant.
Human cognition is shaped by collective experience; we approach new situations with expectations formed by past encounters.
From Virtual to Physical
Chalmers argues that experiences in VR can affect our emotions, relationships, and choices. I make this more tangible by materializing it through a physical computing sculpture that reacts to actions taken in the VR experience — creating immediate, real-world results from virtual actions.
From Physical to Virtual
We enter VR through head-mounted displays (HMDs) and interact with the world as avatars. We experience, explore, and undergo cognitive changes. In essence, we operate within the capacity of technology, entering a hybrid relationship between the human and the non-human. Our agency emerges from this bidirectional relationship — we shape technology, and technology shapes us in return (as described by Actor-Network Theory, ANT).
To capture this bidirectionality, I designed the interaction to be open-ended: the audience’s engagement with certain physical flowers directly influences the VR experience.
