Simultaneous Localization and Mapping (SLAM)

SLAM (simultaneous localization and mapping) is the method computers use to perceive their environment. In user experience (UX) design, it is most relevant to AR (Augmented Reality).

Cameras and laser sensors perceive distance and surfaces to project holograms on an AR display as if they were in real space. SLAM’s other applications include self-driving vehicles, drones and other technologies.

How Does SLAM Work?

SLAM's exact process might vary from device to device, as some are not equipped with distance-sensing lasers or dual cameras.

However, the essential function of SLAM is to identify surfaces using a camera and image recognition software. Devices with lasers can judge the distance using focal lengths and can also judge distance by triangulating from multiple perspectives. This helps with spatial cognition and field of view. This is identical to the depth-perceiving binocular vision that humans have. However, as the human eye can be fooled, so can SLAM.

SLAM Errors

SLAM errors must be handled correctly to ensure an excellent overall user experience. When SLAM fails, it should be easy for users to recalibrate the sensor or a UI prompt should instruct them to move around the area or reposition the sensors for a better look. The visual design should communicate the error and how to correct it.

A product or service should also be aware of situations where SLAM cannot perceive something, like a dark or shiny surface that reflects a lot of light. This can be relatively simple for lower-stakes interactions like AR entertainment, as it will be evident to the user that something is not mapped correctly.

© Interaction Design Foundation, CC BY-SA 4.0

However, this can be a more severe issue for other situations like self-driving cars and drones.

SLAM in Augmented Reality

SLAM is the foundation of augmented reality (AR.) It allows AR devices like AR glasses or mobile phones to perceive the world in three dimensions. AR apps can then identify objects or images in the real-world environment and project virtual content on the AR displays so it appears in the real world.

SLAM also provides the necessary information for the display to match the surface’s positioning and perspective needed for the object to look natural —which adds to the level of immersion.

Sophisticated SLAM technologies can recognize human faces by applying virtual filters or makeup. AR headsets often have a way of warning the user if they are too distracted to notice a wall or object they might run into.

Information overlays like Google Lens use SLAM and AI software to identify text, translate it, and display it in AR.

What are some other applications of SLAM besides AR?

In the case of drones and self-driving cars, ensuring that the interface notifies the user to take manual control when SLAM malfunctions or cannot perceive surfaces correctly is even more critical.

Even though virtual reality (VR) uses virtual environments, VR headsets often use SLAM sensors to let players know where they can walk in VR without walking into real walls or chairs. This becomes especially true for mixed reality (MR) or extended reality (XR) programs as well.

More experimental applications of SLAM include photogrammetry, which uses SLAM sensors to scan real objects as 3D models.

Learn More About SLAM

Take our course on UX Design for Augmented Reality.

To discover even more about SLAM, read this in-depth article: Basics of AR: SLAM – Simultaneous Localization and Mapping