A Mixed Reality Tour Experience
Environments Studio Mini
Client: Carnegie Mellon University
Target audience: Prospective students and their families.
Context: “There is extensive research and development in wearable mixed reality devices. Unlike virtual reality, which completely immerses you in a virtual world by obscuring your entire view, mixed reality blends real and virtual to create hybrid experiences…For this project we are looking 10 years out, assuming that head-mounted displays, such as Microsoft’s Hololens, are as ubiquitous as smartphones are today on campus.”
Objective: Current tours for prospective students and families at CMU include a student-led tour of campus or virtual tours. The information is broad, focused on what’s on campus, and questions of student life and individual programs are limited to the knowledge of the tour guide. The University recognizes the limitations of the existing tour options and therefore is hiring you to develop an autonomous (no tour guide) augmented reality tour. This tour will allow visitors to explore both campus and the surrounding community that is a vital part of the CMU experience.
Deliverables: Overview of the whole tour experience, Low-fi prototype of a single tour stop
Time Frame: 2–3 weeks
09.28.2021 — Research, Developing Personas & Storyboards
Notes from Expanding the Design Process for Mixed Reality article
- Iterate thinking: approach a problem broadly, share ideas with others effectively, to evaluate those ideas and reach a solution
- Teams are a mix of development, design, research, and management, but all roles can participate in the design process
Techniques for expanding the design process
- Bodystorming: encourage interaction with physical props in a real-world space — Using simple, cheap crafting materials we build physical props to represent digital objects, user interfaces, and animations in a proposed experience
- Physical props: Allows individuals with different skill sets to contribute ideas and uncover opportunities for mixed reality experiences. While technical prototyping or high-fidelity storyboarding requires a skilled 3D developer or artist, a few Styrofoam balls and cardboard can be enough to showcase how an interface might unfold in physical space. Ex: a set of plastic connectors might roughly illustrate the size of holograms that appear in a HoloLens experience or as props to act out interactable elements or motion designs.
- Physical props need only meet the minimum requirements to explore or communicate an idea. Not necessary to fully vet it.
- Acting and expert feedback: Walk through an experience with these objects, staging how a user would move through the experience or a specific interaction
- Capturing ideas with storyboards: best method to introduce your experience concepts to new stakeholders or potential users — convey both the overall flow of an experience (at low fidelities) as well as the aesthetic look and feel (at high fidelities); Can build off mood boards to illustrate the final appearance of a virtual experience
- Expanding your team’s skills: We have witnessed several teams successfully transition into 3D development, and find the biggest advantage comes from team members stepping out of their comfort zone and developing skills that give them more involvement
Ending Takeaways
- Feedback and iteration is key to successful design
- Build design technical skills so designers can better understand the approach of developers and provide more effective feedback during prototyping and developers can understand UX concepts and offer implementation insights
- Mixed reality can change the future with more focus on data and cloud services in holographic experiences — taking advantage of natural language processing and real-time computer vision
Microsoft Mixed Reality Article
Some Important Concepts
- Holographic frame — Understand how users see your content overlaid onto the real world when wearing their headsets
- Spatial mapping — Anchor objects in the user’s world and take advantage of real world’s physical surfaces
- Comfort considerations — Ensure user comfort and safety by creating and presenting immersive content in a way that mimics the natural world
- Interaction models — Provide your users with instinctual interactions through hand, eye, and voice input
- Hands and motion controllers — Learn how to interact with holograms at close range with a user’ hands or at long range with precise interactions
- Voice input — Use voice commands as input in your immersive apps to control surrounding holograms and environments
- Eye Tracking —Context and human understanding in a holographic experience by using information about what your users are looking at
- Color, light, and materials — Design quality assets for Mixed Reality that take color, lighting, and materials into account
- Object scale — Incorporate as many real-world visual cues as possible to us help your users understand where objects are, how big they are, and what they’re made of
- Typography — Use clear, readable text in three-dimensional space to give your users the important information they need
Microsoft HoloLens
- DisplaySee-through display — Allows the user to see the physical environment while wearing the headset.
- Movement — Full six-degrees-of-freedom movement, both rotation and translation.
Our 3 Personas
Since our concept revolves around gamification and engaging our audience, our personas are formatted as gamer profiles.
Developing our Concept
Our revised concept idea from feedback from Peter and Mihika; explains our choice of location, quests for different audiences, and features of the token trail
Storyboard Ideas
09.30.2021: Self- Reflection & Concept Development
Self-reflection: As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?
Following the prevalence and rise of digital media in our physical environments, designers are responsible for creating interactions that ensure that interpersonal human interactions remain intact — that technology does not replace human behavior or relationships — rather complements human interactions. An instance for a positive future was exemplified in the Microsoft 2017 future vision video where technology has the potential to drastically increase workforce efficiency, such as supporting surgeons with brain or heart surgery or aid research fields like technology that facilitates underwater research through scanning and recognizing plants or even drones/ machines that collect samples and data. The purpose should remain to implement technology for a meaningful purpose that doesn't cause harm to humans.
Designers should also consider the sustainability and environmental impact of their designs, to ensure it doesn't further hurt the environment as well as improve it possible— such as smart cars including Tesla where fossil fuels exacerbating climate change is minimized through electrically charging vehicles. The benefit of this model/ appeal includes the long-term cost-benefit where the money saved on fuel over 15 years more than compensates for the initial investment. However, the technology also requires large amounts of energy to produce. Another consideration is the human impact as a result of the technology, such as self-ordering kiosks at restaurants which increases efficiency and lowers expenses for the corporation, but also results in a loss of jobs for humans, potentially increasing unemployment rates.
Overall, I think there are numerous benefits of implementing technology in moderation, especially for increasing accessibility, from biomedical engineering with prosthetics to creating interfaces for the blind and deaf. Developing social media platforms can also foster and facilitate communication between people across nations, bringing diverse communities closer together.
As designers, I also think it's our responsibility to make interactions intuitive and not overwhelming — making interactions natural to ensure users don't feel uncomfortable, creating items/ interfaces that afford (a concept we learned from “how people work”) their use and implementation. When introducing new and unfamiliar technology we should consider our user's opinions and experiences. We should begin with teaching people how to easily use the technology, easing them into it, especially for older populations not accustomed to technological cues. For example, I consistently have to teach my grandparents how to minimize tabs, a concept that is entirely foreign to them but natural to me.
I also believe we should make technology transparent to ensure users aren't scared of what surrounds them. Aligning with the idea of the internet of things, people are constantly surrounded by tech from smart TVs, vacuums, lamps, all connected through Alexa or Google Home. Ensuring transparency behind the controller's decisions and voice responses will prevent new consumers from detering from the idea. Amazon has started to develop this concept further…
“We believe in transparency when it comes to your data. From the basics to a deeper dive, here are some topics to help you learn more about Alexa and your privacy.” — Amazon
The section on their website dedicated to explaining Alexa’s privacy settings to reassure consumers that their personal information is not being exposed.
In the end, I believe immersive technology has vast potential to improve people’s everyday interactions around the world, if designers consider the user’s perspectives and emotional response to the technology while maintaining human interpersonal connections.
Prototyping Features for the Token Trail
Adobe Aero & Reality Composer
Rough idea of creating a trail of circular multicolored objects that spin when approached and can be picked up, enlarged, interacted with, and project important information
Walking through the area and mapping out the path for our tour
Prototyping different types of 3D Coins
Trying Adobe Aero with the exported coin designs
Developing Concept for coin content
Creating an interface for the coins with information to pop up
Photoshopping a rough rendering of the Pausch bridge entrance
- (Proportions of the person are very off)
- Too many tokens featured
- Interface should not be black and should be more transparent with less words
10.05.2021: Developing Concept & Sketch Videos
Writing out a detailed storyboard of the interactions
Experimenting with recording for the After Effects sketch video
10.05.2021: Concept Development & Content Creation
Creating the 3 types of tokens on Adobe Dimension with corresponding icons
Created the first iteration of content for interfaces
Removed unnecessary text elements, changed it to white/gray transparent so it’s not as distracting to the user
Explored methods of putting the coins into after effects
- We tried Aero, reality composer, frame by frame animation, and screen recording on Aero
- Frame by frame animation — lots of work and hard to keep frames consistent as well as it was choppy
- Reality composer doesn’t allow for exporting 3d figures from Dimension
- Tried to record with Aero in the actual scene but the quality was very blurry so we had to rerecord all the scenes without the coins and then add them in later
- In the end, exported Dimension to Aero, Recorded on aero, Rotoscoped to insert into the video scenes
10.07.2021: Video Creation & Content Changes
Feedback from Daphne and Mihika
- Need a better intro to the game — not just jump straight into it —maybe let people pick an avatar, etc
- Weird to have the student’s name in the game
- The interfaces are too text-heavy — don't need most of the information on the interface as well as the coin info (Only need important 1 fact for the token info since that's the purpose of the pop-up)
- Add in-game sounds like “dings,” also consider audio people will hear during the game
Changes to Interfaces & Video
- Don't need more than 2 token interfaces — people understand the concept
- Simplified the information on the coin interfaces — everything besides the token info was deleted
- Added in an intro where people can pick avatars, missions, and language
- Accommodate people who can't see with the audio cues option
- Backpack interface can be simplified— can just be a backpack icon at hip-level that people can drag interface into to save with a counter for the number of tokens they have saved so far
- Added dynamic components to the AR experience — elevators buttons, skateboarding person pop-up
- No need for lots of black text because the actions are self-explanatory
- Added a takeaway component where people can take home the favorite/ most important information they saved in their backpack
New Iterations of Interfaces
10.12.2021: Video Creation & Content Changes
Feedback from Daphe & Tay
- Let information stay on screen longer so that people can read it
- Simplify some info — ex skateboarder not necessary to have so much information that’s difficult to read as someone quickly skates by
- Add some more black text at the end to keep it the same as the beginning
- Prospective students don’t need to know about seat availability — only focus on information that they’ll remember
- Don't need as many tokens on the bridge — can be overwhelming
- Make the intro interfaces consistent with the buttons (language scroll can be hard to click since the lines are so close to each other)
Updated Interfaces
Creating the Hardware/ Software Systems Map
What is the larger system of software and hardware? In a simple diagram, explain the physical components (laptop, tablet, glasses, etc.) and how data is shared/transferred between them
4 Stills of Interactions
Critical components of the token process
Final Sketch Video
Presentation Slides & Graphics
Final Presentation Video
Google Drive Link
Self-reflection. How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?
Thinking back to the physical to digital spectrum we were introduced to in the beginning of the mini course, with this project leaning towards the digital/technological side, I developed more theoretical and abstract design thinking skills about designing spaces that are reliant on technology. Instead of physically mapping out our space in SketchUp or with foam-core models we created for the first project, where we developed an entire controlled space, this project was broader where we designed for a space with no boundaries — focusing more on the experience and people’s interactions with already existing structures. The project was also harder to visualize since it had less tangible components to it. The skills I developed that were similar to the past project include considering a person’s motion in the space/ area and their interactions with technology. While the hybrid exhibit technology supported people’s interactions, our VR experience depends on technology to function. During this project, I learned how to prototype in 3D as well as create motion videos to display someone’s interactions with the entire world around them — creating one main explanatory deliverable. Also since the context was in 2030, there was an apsect of imagination and theorizing what technological advancements would be prevalent 10 years from now. This project was a lot more technologically centered, which I really enjoyed and hope to further explore in E in the future. I think technology has potential for so many impactful immersive experiences. I also found it very interesting to consider technological systems interacting with each other and how ubiquitous this will be in the future.
I understand the role of an environments designer to be someone who designs specific experiences for individuals, considering the different interactions they will have in that space — whether it’s outside in an AR reality experience or in a controlled museum exhibit. I enjoyed how broad the discipline is and the several different avenues that exist to creating spaces catered for certain people and purposes, which can evoke certain emotions. I realized the importance of considering the already existing aspects of the physical world, and purposefully using technology to simply augment and improve the quality of the experience — not overwhelm the audience. I also believe environment designers heavily rely on understanding their users and personas to cater experiences towards their desired emotional responses.
