Bridging Physical and Virtual Realms with Arduino and Unity, respectively
For this part of the project, I intended to explore the interaction between physical and virtual environments. I aimed to establish a two-way communication channel where actions in the virtual world could produce real-world effects. To achieve this, I used an Arduino microcontroller to receive output signals from a Unity-based VR environment. This allowed me to control a physical LED light in response to a virtual event – shooting bullets from the gun – via the controller’s index finger trigger.
Setup Overview
The physical setup consisted of a single LED connected to an Arduino board. The Arduino was linked to my laptop via a USB-C cable. On the software side, I developed a Unity project using the OVRInput system to trigger virtual shooting events. These events would send a signal through a serial port to the Arduino, prompting the LED to turn on briefly.
Initial Challenges and Troubleshooting
The setup proved to be more challenging than expected, particularly in terms of serial communication and platform compatibility. Below is a breakdown of key issues I encountered and how I addressed them:
1. Arduino Upload Issues
At first, I was unable to upload sketches from the Arduino IDE to the board, despite:
- The Arduino is being detected by the device manager
- The correct drivers are being installed
- Successful code compilation
Even though the COM port was visible and correctly selected, the IDE failed to upload the code. After troubleshooting extensively and rechecking the USB connections, I found that a simple system reboot resolved the issue. This was unexpected, but it allowed uploads to proceed normally afterwards.
2. Unity and Arduino Serial Communication; Arduino Sketch and C# description
Unity does not natively support serial communication with external devices like Arduino. To bridge this gap, I relied on the .NET System.IO.Ports namespace, which provides serial communication capabilities.
I wrote a basic Arduino sketch that turns an LED on or off based on a received character ('1' for on, '0' for off). In Unity, I implemented a custom C# script that uses the SerialPort class to send these signals. This script was attached to an empty GameObject and referenced within the RayGun script to trigger LED activation when the player fires the gun.
Based on the tutorial for setup communication: Unity & Arduino Communication
This is a simple Arduino sketch designed to control an LED. In the setup() function, serial communication is initialized at 9600 baud (bits per second), and pin 2 is configured as an output to control the LED. Although a global buffer and a char array (buf[]) are defined with a size of 50, they are not actively used in the final version of the code. I originally experimented with reading multiple characters at once, but I noticed this caused the LED to remain continuously on — which didn’t work well for my intended shooting feedback effect. As a result, I opted to read only one character at a time, which allowed for more responsive and accurate LED control.
In the loop() function, the sketch checks whether any data is available on the serial port. If data is detected, a single character is read and stored in the cmd variable. If this character is '0', the LED is turned off (digitalWrite(2, LOW)); if it’s '1', the LED is turned on (digitalWrite(2, HIGH)). This allows Unity (or any external controller) to send simple serial commands ('0' or '1') to toggle the LED in real time.
I also included a short delay of 200 milliseconds after each loop cycle. This was partly based on recommendations from online tutorials, but also confirmed through testing: it helps synchronize the communication and prevents the Arduino from reading too frequently or reacting too rapidly, which could cause inconsistent behavior. This delay ensures that the LED only responds once per input, making it more suitable for the quick, discrete signals used in a VR shooting mechanic.
In terms of the C# implementation, the script makes use of the System.IO.Ports namespace, which provides access to serial communication via the SerialPort class. This is essential for enabling Unity to communicate with external hardware such as an Arduino.
Within the Start() method, a serial connection is established using COM6, which corresponds to the port associated with my Arduino controller connected to the PC. The communication is initialized at 9600 baud, matching the settings specified in the Arduino sketch (Serial.begin(9600)).
The SendSignal(bool on) method is designed to send simple control signals — either '1' or '0' — to the Arduino. If the on parameter is true, it sends '1', which lights up the LED. If it’s false, it sends '0', turning the LED off. This binary approach allows Unity to provide immediate physical feedback in response to in-game events, such as shooting.
Lastly, the OnApplicationQuit() method ensures that the LED is turned off when the Unity application is closed. It sends a final '0' to the Arduino before closing the serial port. This prevents the LED from remaining on unintentionally after the game ends.
In summary, this script acts as a bridge between Unity and the Arduino, using serial communication to synchronize digital actions (e.g., pressing a button on the controller) with physical outputs (e.g., turning on an LED). This implementation enables a simple but effective feedback loop between virtual and physical environments.
Key Technical Bottlenecks
Namespace and Class Recognition Errors
A major obstacle was Unity’s failure to recognise the System.IO.Ports namespace and the SerialPort class. The error message stated that the class existed in two assemblies: System.dll and System.IO.Ports.dll, causing a conflict.
To resolve this:
- I changed the API Compatibility Level in Unity (via Project Settings > Player > Other Settings) to .NET 4.x.
- I manually downloaded the
System.IO.Portspackage from Microsoft’s official NuGet repository. - I experimented with placing different versions of the DLLs in Unity’s
Assets/Pluginsfolder, but most led to version mismatches or runtime errors.
Ultimately, changing Unity’s compatibility settings resolved the issue, and no additional DLLs were required. As per the image below, I manually located files and copied them into the assets folder of my Unity projects, as Unity failed to retrieve the class of the namespace. This is something I’ve been advised to follow from the online group discussion. These, however, end up throwing an issue later on, with Unity suggesting a class definition left in 2 separate file versions. 1 came via plugin downloaded on VSC (Visual Studio Code), the other one manually via a zip file folder of the website. In the end, both were required to be kept on the local machine, but not under the same project. Very confusing haha.
Windows-only Support Confusion
Another issue arose from Unity reporting that System.IO.Ports it was “only supported on Windows”—despite my work on a Windows machine. This turned out to be a quirk in Unity’s error handling and was resolved by ensuring:
- The Unity platform target was correctly set to Windows Standalone.
Final Implementation Outcome
After several hours of testing and debugging:
- Unity successfully sent serial data to the Arduino each time the player pressed the fire button.
- The Arduino correctly interpreted
'1'the signal to light the LED, and'0'to turn it off after a short delay. - The interaction was smooth, and the LED reliably responded to gameplay events.
This implementation serves as a foundational example of hardware-software integration, particularly in VR environments.