Augmented reality and how it works

Augmented reality and how it works

Augmented reality (AR) adds content into a camera feed and make the digital content look like it is part of the physical world around us. This digital content could be anything from overlaying the environment around you to making someone look like a tiger. AR can make you visually see how furniture would look like in your house, or play a board game on a table. For AR to alter your physical environment, the system must first understand the physical world from the camera feed.

How does augmented reality work?

The first step in AR is distinguishing the content of the camera feed from the world around the user. This allows the system to show the digital content that is relevant to what the user wants. This content is then displayed realistically to look like part of the real world. The module augments the environment to ensure it precisely overlaps with your physical environment.

Why does augmented reality need computer vision?

While the human brain understands images easily, it’s very difficult for computers to do so. There is a Computer science branch dedicated to images known as Computer Vision. For AR to work, the system needs to understand the world around the user in terms of 3D geometry and semantics. The geometry answers the question “Where?” while the semantics answers the question, “What?” This infers to where the object is facing and where they are in the 3D world. Without geometry, Augmented Reality content can’t be displayed at the right angle and place, which is a critical process of making it feel like part of the real world. Often. The system will develop new techniques for every domain. For example, the computer vision method used in viewing a house is different from those used for someone’s face.

Geometry and semantics of the world

The computer vision techniques that are used for understanding semantics and geometry are quite different. Thanks to Deep Learning, there is a lot of progress that has been made by experts on semantics. Deep Learning typically figures out what an image is without looking at 3D geometry. When used on its own, Deep Learning enables basic AR forms. Wherever a computer vision identifies an object, it could display the relevant information floating on the screen but will not be anchored to the physical object. The geometric side of Computer Vision is required to build the concept from projective geometry.

Future of Augmented Reality

Augmented reality is an exciting and active field. In the future, it’s expected that there will be many new developments. With advancements in Computer Vision, it will be easier for computers to understand the world around us. This will improve AR experiences making them more exciting and immersive. While AR today mostly lives on our smartphones, it could happen in any device with a camera. In the future, when we will have enough computational power on Augmented Reality glasses, it will enhance the way we shop, live, play and work.