In an effort to understand the limitations and features of new input methods, we used Unity to create a minimalist port of the popular block game Jenga with a Leap Motion and Oculus DK2.
Code on Github.com.
We were put in teams with a developer, artist, designer, and project manager. Roles were fluid as there were only three UX designers in the class (the rest of the students were either developers or artists) and the majority of time was spent on programming. Each week the teams would present their work in front of the class for critique. We showed off our current state of development, shared problems and insights, talked about our weekly playtesting, and gave demos. In addition to our classmates, our professor (Tom Corbett of Electronic Arts) and occasionally outside game developers/designers would critique our work. The whole project was about seven weeks: two weeks for mouse and keyboard, two weeks for Leap Motion integration, and three weeks to integrate the Oculus DK2.
In the days leading up to working with a new technology like the Leap Motion or Oculus DK2, I spent a weekend downloading and playing games designed for those devices. As a UX designer, it is important that I understand how others have solved the problems I will encounter — while keeping an open mind for new and better solutions.
This was my first experience building a project in VR and my second experience with Unity (my first being a “Let’s Learn Unity” demo project). The class was largely video game developers and artists with three students from HCII (including myself). Having no previous experience working in the video game world, I struggled to find my place. It was also the first time students entrenched in video game design and development worked with user experience designers.
We worked around our field jargon to create a workflow: I created initial designs, we all critiqued them, and then the programmers started implementing what we all agreed on. Throughout development, I explored games built for the Leap Motion and Oculus DK2 and made relevant UX suggestions (e.g. how to best position your hand for the Leap Motion to read it).
Early on I suggested to the team that we use a minimalist aesthetic. Since none of us were performance graphic specialists nor were our laptops powerful (which were to later power a DK2), it made sense to find a new aesthetic that would be fun to build and play and would still run at a high frame rate. Additionally, the minimalist aesthetic would differentiate us from the other teams who were building (1) a cell-shaded, kid-esque Jenga and (2) a Godzilla, city sized Jenga.
For week one, I designed an interactive wireframe for the menu and some of the interactions. Since we were aiming for minimalism, the designs were not too far from the final desired aesthetic.
In the first week, we had our first version of Jenga running. It wasn’t beautiful and it wasn’t the easiest to use. An example: when you clicked a block just a bit, it would respond by turning red. This confused users because of red’s usual correlation with doing something wrong. I refined the color palette and we had more positive feedback.
On the interface side, I had initially designed a start screen that included both a side menu and a semi-playable version of Jenga on the right. This also confused users. “Is this Jenga the game I’m playing?” “Do I need to press start if it already began?”
For the second version, we added what began as an end game error. Before the developer finished writing the game’s end-state, there was an error where blocks kept falling from the sky like a waterfall. Finding the waterfall playful, we added it to the start screen in lieu of the playable tower. This worked well until we started working with the Oculus. In virtual reality, waterfalling blocks are too disorienting.
I also explored the use of small elements that would give more information about what was happening in the game. In the top right of the above image, you can see a tilt icon: how much and which way the tower is swaying. In the bottom left, there’s an element that shows the user’s score. The bottom right has a clickable icon for keyboard shortcuts, looks like the standard walking keys in video games, and happens to use the first letter of each name of the original team.
Interacting with the tower proved more difficult than we expected. With mouse and keyboard controls, there does not exist a great way to move in three spatial dimensions. Leap Motion helped, but once we were working with the Oculus, the Leap’s position on the headset reduced hand-tracking confidence. I suggested we work around this by reaching into the tower with an open faced hand then grasping a block with a whole fist. These exaggerated motions improved hand-tracking, but distanced our game from normal Jenga.
Poor hand tracking also caused players to accidentally drop a block when they pulled it from the tower. Then, when trying to grab the block on the ground and put it back on the top of the tower, there was a good chance hand-tracking would fail and the user would have to try again. To get around this, we put blocks back on the top of the tower for the users.
When people first try VR, they usually don’t realize they can move their head in and out, toward and away objects. To prompt this motion, I wanted to create a version of Jenga where a floating tower would appear in front of the user. To reach the pieces and look around, they’d be inclined to move their body, trigging a deeper level of immersion. To find our design boundaries, I ran a design experiment.
Standing in the Oculus demo scene, I tested the limits of head tracking for position and rotation away from the positional tracker. I found that:
We never realized this version of our Jenga project due to time and human resource constraints.
In addition to my UX work, I also created custom sound effects for the game. Because of our minimal aesthetic, bare-bones electronic music seemed the most fitting. I created a short tune with three levels of intensity to correlate with more intense gameplay. Due to development constraints, I created another, longer version to loop in the background during game play.