Innovate with UX: Design User-Friendly AR Experiences

What if your favorite video game characters could come to life or your online shopping could appear right in your living room? Augmented reality (AR) transforms how we interact with digital content. Imagine creating immersive experiences where users can explore 3D models, overlay information on real-world objects or even play interactive games. Learn more about the technology that brings the digital into the physical world in the IxDF UX Design for Augmented Reality course.

It's always a challenge to create a smooth and enjoyable user experience; everyone has different needs, preferences and ways to interact with technology—what works for one person might not work for another.

Things get even trickier when you add new technology—like Augmented Reality (AR)—to the mix. AR blends the digital and real worlds, often in fascinating and fun ways, to offer new possibilities for new apps and experiences. Unlike Virtual reality (VR), which creates an immersive digital environment that feels separate from the real world, AR blends the digital and physical worlds, so it calls for a different design approach for this emerging technology to make AR solutions that resonate with users in the target audience.

Watch Frank Spillers, CEO at Experience Dynamics, discuss AR in this quick video.

AR is all around—including in Google Maps and Ikea’s Place app. How do you create an app that feels natural and intuitive when it overlays digital information in the real world? How do you guide users through an experience that occurs around them, not just on a screen? These are the questions you will face when you work with AR.

If you want to learn how to design better user experiences for augmented reality, then learn about these top 12 things. 

The Fundamentals of UX for AR

Let's understand what UX design for AR is all about and why it's important for the future of this technology.

1. What Makes UX for AR Different?

UX for AR isn’t just about screens and buttons. As part of the design, you create natural and intuitive experiences through the space around you. It's a whole new way to think about design.

Think about using a smartphone app. You tap, swipe and look at a flat screen. Now, imagine an AR app. You move around, use your voice and see digital objects right before you—lots of possibilities there. Let's say, for example, a user tries on a new pair of glasses with an AR app. The app shows them how the glasses look on their face in the mirror. It's like magic, to be sure, but the app loses its magic if it's clunky or the virtual glasses don't fit right. That's why good UX design is important for AR. In a similar way to how UX for mobile device designs needs to make sure users get seamless experiences—where they can forget they’re even using a device to access a brand’s solution—good AR UX makes the experience feel real and enjoyable.

2. How Do You Bring Stories into the Mix?

You can use the power of AR to transform ordinary places into compelling stories. Stories connect memories to real-world places and AR brings them to life. It's like a play or public art, but with AR you can be part of the story.

How do you create a good AR story? Think of the real world as being your stage. Now, imagine your user's journey as the story and use a narrative style in AR design to take your user on a journey. Draw your user with a sense of curiosity.

• Where does it start and end?

• What problems do they face?

• How do they overcome them?

• Why does this happen in this place?

However—as with any good story—you have challenges to overcome. These challenges create tension and excitement. Users who solve problems and interact with the AR elements feel a sense of accomplishment when they overcome this “conflict”—and this positive feeling helps them connect with the experience on a deeper level.

That's what you need to figure out—the story you'd design for AR and the challenges the users would need to overcome, and all the cues and other aspects of content creation that would have to support these. They’re vital considerations for app builders of this sort, long before app development zeroes in on the final product for marketing campaigns to trumpet to legions of users.

When you plan your AR narrative, create a roadmap for the user's journey. This roadmap is what helps you design each step to create specific feelings. Each stage has tasks that fit those feelings.

1. Curiosity/exposition: You introduce the user to the goals. You make them want to explore more. You show them the world of the story.

2. Surprise/rising action: Next, you open up the experience. You show the user what they can do. You teach them how to move and interact. Users may need to move, look around and use controls. You also introduce the problem they need to solve.

3. Crisis/climax: You put an obstacle in the user's way. They need to face the problem and try to fix it. You change the environment to make things feel tense.

4. Success/falling action and resolution: Finally, you let the user solve problems, finish tasks and get rewards.

The user controls the story and the story changes based on their actions. This pulls them into the experience. Use real places as stages and the experience will stay with the user even after they stop using the app.

3. The Role of Social AR

Social AR uses augmented reality on social media apps like Facebook, Instagram and Snapchat. It includes fun camera filters. These filters can help connect customers with brands in really fun ways.

Imagine you send your friend a funny AR filter that turns them into a dancing cat or share a virtual tour of your latest vacation spot—that's social AR in action. When you send something like this, it adds a fun, interactive layer to your social interactions to make them more engaging and memorable.

Social AR is a game-changer because it reaches a massive audience—a mind-bogglingly huge one, in fact. Billions of people use social media apps every day, and that means your AR creations have the potential to go viral and reach far beyond your immediate circle of friends. You get key benefits if you incorporate social AR, such as:

• Engagement: Social AR boosts engagement with your brand or content. People love to share and interact with AR filters and effects.

• Virality: A well-crafted AR experience can quickly spread across social media. It holds the potential to increase your reach and visibility to an exponential—or viral—degree.

• Brand awareness: Social AR creates a fun and memorable way for people to connect with your brand and enjoy the steps of their journey with it—one of the best marketing strategies to harness.

• User-generated content (UGC): Encourage users to create and share their AR content—something that’s more than an added bonus. UGC and shared content amplify your reach and foster a sense of community.

User Research for AR

Before you create any AR experience, you need to understand your users. User research helps you learn what your users want, need and expect from an AR experience. This knowledge helps you design something that they'll love and keep coming back to.

User research for AR has some unique challenges. Even so, with the right approach, you can collect valuable insights to guide your design decisions. So, without further ado, let's explore some key aspects of user research in the AR world.

4. Understand the User's World

To start with, consider the most relevant practicality here—think about where your users will use your AR app. They might be at home, at work or even outside, for instance. The environment matters. It can change how well the app works—not to mention the potential hazards that you’ll need to bear in mind regarding what’s around the user; walls or moving objects, for example.

AR mixes the real world with the digital world. This means you have to think about more than just the app screen. You need to think about the whole environment. Things like lighting, noise and space all affect how people experience AR.

Suppose you want to design an AR app that helps people decorate their homes. The app needs to work in different rooms with different lighting. It needs to show colors and textures accurately, no matter the light.

People also move around when they use AR. They might walk, cycle or stand still. Your design needs to work for everyone. Think about an AR navigation app. It should help people find their way whether they're moving or not—and you’ll have to design for this contextuality so everyone can reap the benefits.

5. Use the Environment in AR Experiences

The environment is an integral part of the design. Lighting, noise and available space influence how users perceive and interact with AR elements.

Consider a museum AR tour—it should be subtle enough not to disturb other visitors, adapt to different light levels throughout the exhibits and avoid directing users into obstacles. Understand and utilize the environment to create AR experiences that feel natural, immersive and considerate of the context.

Here are some things to think about when you design AR experiences:

• User characteristics: Consider age, gender, education and tech experience to design easy-to-use interfaces. Consider how familiar your users are with AR.

• Task characteristics: Make tasks fit the environment. Make the interfaces simple so users can do tasks quickly.

• Environmental characteristics: Think about lighting, noise and space. Design clear interfaces that are easy to understand and safe—safety is particularly paramount, remember, and doesn’t just apply to staircases, roads and low tables. What’s more, think about the field of view (FOV) to avoid discomfort.

• Device characteristics: Consider the device's size, weight, screen and battery life.

• Social context: Think about where people use AR and what others around them expect. Make interfaces respectful and appropriate.

• Cultural context: Think about your users' culture. Consider the many intricacies and situational quirks that users in different parts of the world might see in different ways, and accommodate them on those. Design interfaces that fit different cultures.

6. Focus on Users on the Move

When you design for AR, it's important to activate the spatial part of the brain. Watch Frank Spillers describe what it means to design experiences with spatial cognition on the go.

UX design for AR is similar to mobile UX. Users are often on the move and use the interface for short periods or bursts of activity. This is different from virtual reality (VR), which resembles web or desktop UX and caters to longer use. AR activates spatial cognition, which is how our brains understand the environment around us. It includes:

• Spatial perception: How we see and understand where things are in relation to us.

• Spatial memory: How we remember locations and routes.

• Spatial reasoning: How we predict where things will be.

• Spatial problem-solving: How we use spatial information to solve problems.

AR design should feel like it exists in the same world as the user—it’s where the magic needs to live. The interface should enhance the world, not block it. Here's how you can design AR for people on the go:

• Walk the space: Imagine using the AR experience in the real world. Notice opportunities and constraints.

• Use gestural inputs: Consider how users might interact with AR with their hands, eyes or voice.

• Design adaptive user interface (UI): Adjust contrast and brightness based on lighting conditions.

• Consider user fatigue: Don't make users hold their phones for too long—things can get tiring.

• Limit task load: Don't overload users with too much information at once; they might switch off.

7. Craft Safe and Immersive AR Experiences

Safety first—and this one deserves a second, third and even fourth mention, as it’s impossible to overstate how vital it is. When you design AR experiences for people, it demands a balance of safety and immersion. You want users to feel drawn into the AR world, but not at the expense of their safety in the real world—although the AR-world things won’t necessarily collide with their real-world surroundings, the real-world, real-time user certainly can!

Always remember, users interact with AR in the real world, which has real dangers. You have to keep users safe while they use your app—it’s your primary duty of care, even if they might be responsible for where they actually tread. You must understand potential hazards like obstacles (think knees banging against coffee-table corners), other people (think foreheads smacking in head-on collisions on a street) or traffic (maybe don’t think about this graphically; instead just remember someone might end up getting maimed or losing their life in front of a motorized vehicle if they’re engrossed in their AR experience). So, here's what you must do:

• Don't create experiences that require users to look away from their surroundings for too long or make sudden movements.

• Do design clear visual cues and feedback mechanisms to help users understand where they are and what happens around them.

Immersion doesn't have to mean losing touch with reality. You can create engaging experiences that still prioritize user safety and keep your brand popular among users (and their relatives and friends, who won’t curse it as a reckless thing).

How to Create the Visual Design and Prototypes for AR

When you create the visual design and prototypes for your AR experience, your design really comes to life. It's the bridge between your initial concept and the final product—that vital field experiment where you can test and refine your ideas in a tangible way.

In this stage, you create visually appealing elements and ensure they seamlessly integrate with the real world and enhance the user's experience. It’s a major step on the road to seeing how things will go for users in the wilds of their everyday uses of your ultimately complete version. So, let's understand the key aspects of visual design and prototyping for AR.

8. Make use of Affordances in AR

Affordances are connected to what you can do with an object—they’re not just about the object itself. It's about what the user can do with it. For example, a door with a handle allows you to open it. But the door doesn't offer that option if a small child wants to open it and can't reach the handle. Affordances are all about possibilities—the things you as a designer afford your user to get to use.

People need clues to understand how to use a product. Good design shows the purpose and how to use the product without the need for pictures, labels or instructions.

"When affordances are taken advantage of, the user knows what to do just by looking: no picture, label or instruction needed."

— Don Norman, Grand Old Man of User Experience

AR technology is still developing and has endless potential—many, many places to go with it, and many as yet unimagined, waiting for the taking at some point by creative designers and design teams. If you want to design the best affordance, then you need to apply user-centered design principles to make content experiences more efficient and enjoyable. So, focus on these key considerations:

• Visualize the current state: Provide clear transitions and visual cues to guide users through tasks and indicate completion of these.

• Increase manipulation efficiency: Offer detailed guidelines and interactive instructions for unfamiliar operations to help users through in the moment—remember, a moment’s hesitation or confusion is a moment too long.

• Accurate information delivery: Use intuitive UI elements like icons and menus and make sure that text is easily readable—even when the screen is moving.

• Minimize risk and obstruction: Guide users around obstacles and provide warnings about potential risks—very clear warnings; safety comes first.

• UI design suitable for the environment: Adapt the UI to different lighting conditions and prioritize information on each screen.

• Increased visual immersion: Use realistic graphics and minimize physical fatigue for users.

9. Design Spatial Interfaces for Augmented Reality

UX design for AR goes far beyond traditional screen-based design. You create interfaces that interact with the physical world. That’s where spatial UI comes into play and gets down to work. It focuses on how users perceive and interact with digital content in their surroundings.

You can design a much better spatial user interface when you understand what goes on behind the successful interface. Watch as Frank Spillers discusses the five key factors in AR spatial UI design.

10. Create the Prototype for AR

Prototyping plays a vital role in AR design. It’s what brings your ideas to life and lets you test and refine your concepts before you invest in full development and potentially sink any resources into the wrong assumptions. With AR, you can create both 2D and 3D prototypes to visualize and assess different aspects of the user experience—vital ways to see what works and heed early-warning signs. Here's how you can create a prototype for AR:

1. Define Your AR Experience

Think about what you want your AR app to do. Will it show virtual furniture in a room? Display information about a product? Guide users through a task? Clearly outline the purpose and features of your AR experience to guide your prototype.

2. Sketch Your Ideas

Grab a pen and paper (or a digital drawing tool) and sketch. Draw what you want the user to see on their screen when they use your AR app. Include the virtual objects, buttons or controls and how those elements might interact with the real world.

Many tools are available for AR prototyping that range from simple to complex. For a basic prototype, you could use presentation software like Keynote or PowerPoint or just cut out images and a smartphone. Consider tools like Adobe Aero or Unity if you want more advanced interactions.

You have several AR prototyping methods available, such as:

• 2D prototyping: Use design tools (Adobe Suite, PowerPoint) to create mockups of AR interfaces. Create clickable prototypes with Keynote or PowerPoint to test user interactions.

• 3D prototyping: Use tools like Unity and Vuforia to build 3D environments and interactions for a realistic preview of the AR experience.

• VR prototyping: Use VR tools like Sketchbox to visualize spatial layouts, experiment with UI elements and 3D object placements. Take screenshots and combine them with real-world photos for realistic mockups.

• Low-fidelity user testing: Use printed images or physical objects to represent virtual elements. Conduct user testing in real-world settings to weed out usability issues and gather feedback.

3. Build Your Prototype

Now—drumroll time—it's time to bring your sketches to life. If you use presentation software, create slides that represent different screens of your app. Add images or 3D models of your virtual objects and set up transitions to show how the experience might flow. If you use physical cutouts, arrange them on a table or hold them up to a camera to simulate how they appear in the real world through your phone screen.

4. Test and Refine

Share your prototype with others and get their feedback. Ask them to try out the experience and see if it's intuitive and how easily they take to it. Pay attention to what works well and what might be confusing—and use this feedback to improve your prototype and make it more user-friendly and perhaps even more safe.

Additional Tips

• Keep it simple: Don't try to cram too many features into your prototype; instead, focus on the core functionality you want to test.

• Get creative: You can still create good prototypes with limited tools. AR’s a creative realm, so use your imagination and find ways to simulate the AR experience.

• Iterate: Prototyping is an iterative process; so, don't be afraid to experiment, make changes and refine until you have a prototype that effectively communicates your vision.

Ways to Improve Your AR Experiences

You have your AR vision and a prototype ready for testing. Now, it's time to refine your AR experience and make it truly shine. You can focus on several key areas to create a seamless, intuitive and enjoyable user experience. Let's explore some ways to enhance your AR creation.

11. Leverage AR Heuristics for Better UX Design

User testing is important for usability; however, you can also use "heuristics" guidelines to check your designs. These guidelines are based on years of research and experience, and they help AR designers make the best possible experiences. Experts adapted these guidelines for mobile AR. You can use them as a scorecard to see how well an AR app works and where to improve. These include:

1. Visibility of system status: The app should always tell the user what's happening.

2. Match between system and real world: Virtual objects should look and act real.

3. User control and freedom: The app should let users undo and redo actions.

4. Consistency and standards: Use gestures that make sense. Show how to move objects.

5. Error prevention: The app should help users avoid mistakes.

6. Recognition rather than recall: Show users what they can do, not make them remember.

7. Aesthetic and minimalist design: Don't show too much information at once.

8. Help users recover from errors: The app should help users fix mistakes.

9. Help and Documentation: A simple tutorial can help new users.

12. Test AR Designs with Users: Get Real-World Feedback

User testing gives you valuable insights into how people interact with your creation in the real world—and it’s impossible to overstate how important that is. It helps you find and fix problems before they might surface at launch and word of mouth and reviews might damage the brand: cost effectiveness that’s next to priceless. However, testing AR designs comes with unique challenges:

• Hardware access: Not everyone has AR devices. This can make it tricky to find test participants.

• Environment: AR relies on the real world, which can be unpredictable. You’ll need to test in different locations and lighting.

• Novelty: Many people haven't used AR before. You must guide them through the experience and explain how it works.

Follow these strategies for effective AR user testing:

• Clearly define your goals: What do you want to learn from the test? Focus on specific aspects of the user experience—such as ease of use, navigation or interaction with virtual objects.

• Recruit the right participants: Look for people who fit your target audience and have varied levels of AR experience.

• Choose appropriate testing environments: Test in different locations and lighting conditions to see how the AR experience performs in real-world scenarios.

• Prepare a structured test plan: Outline the tasks you want users to perform and the questions you'll ask them. Consider a combination of observation, interviews and questionnaires to get the most and best insights.

• Think beyond traditional usability testing: AR requires you to observe how users move and interact with their surroundings. You can consider techniques like "think-aloud" protocols, where users verbalize their thoughts as they use the app.

• Be flexible and adaptable: Be prepared to adjust your test plan based on user feedback and unexpected issues that may arise.

The Take Away

UX design for augmented reality—AR—presents exciting challenges and opportunities. It demands a shift in thinking as you have to consider the user's environment, movement and interaction with the real world—taking things up a gear from your typical mobile app concerns. As you understand the unique aspects of AR and apply user-centered design principles, you can create immersive and intuitive experiences that users will love and support with custom and social proof. These should be experiences where you seamlessly blend the digital and physical worlds and make the “magic” of a new world superimposed responsibly on the users’ world.

The IxDFs' UX Design for Augmented Reality is a 5-week course that will help you further understand and build upon the topics we discussed. In this comprehensive course, you’ll get a deep understanding of the principles, techniques and best practices to design user interfaces and interactions in AR applications.

This course is ideal for you if you are:

• A designer who wants to apply their skills in an exciting new field.

• An entrepreneur with a goal to create the next big AR app.

• A business stakeholder or product manager who seeks to augment their brand with AR products or services.

• Anyone who wants to explore cutting-edge technology and grapple with the challenges of an exciting new medium.

UX expert Frank Spillers, CEO and founder of the renowned UX consultancy Experience Dynamics, teaches this course. Frank is an expert in AR and VR and has 22 years of UX experience with Fortune 500 clients.

In the Build Your Portfolio: Interaction Design Project segment of the course, you will engage in practical exercises that give you hands-on experience with the methods taught. These optional exercises will allow you to create a series of case studies for your portfolio. They will help you showcase your skills to potential employers or clients.

References and Where to Learn More

Enroll in the IxDF UX Design for Augmented Reality course. It's included in an IxDF membership. To become a member, sign up here.

Read our topic definition of augmented reality.

Learn about Harnessing the power of Stories.

Understand the power of social AR as you look at Top Social Media Apps/Sites by Monthly Active Users (MAU).

Read our article to learn How to Design for AR Experiences on the go.

Have a look at the Past, Present and Future of Augmented Reality.