How can augmented reality and video surveillance work together?

Stefan Lundberg

The benefits of daydreaming to creativity are well-documented, and every now and then we’ll allow our minds to wander to some of the “what if?” questions that crop up.

Recently, we were thinking about how augmented reality (AR) might be used more extensively alongside video surveillance, and the types of scenario and use cases where such an integration might align to our vision of creating “a smarter, safer world”.

A brief background on augmented reality

In broad terms AR can be defined as the ability to overlay text, images, and other information onto a live scene or video to provide additional insight to the user. This information can be displayed on whatever device is used to view the live scene, such as monitors and mobile devices, and in some applications through smart glasses and headsets.

It’s worth contrasting this with virtual reality (VR) as they can sometimes be confused. VR – most commonly through a headset – provides a first-person perspective of being in an environment or situation the wearer is not actually physically in.

VR is the illusion of being in a particular environment; AR is actually being somewhere or viewing an actual scene, with additional information overlaid.

A simple example of AR in the consumer environment would be an individual walking down a street in an unfamiliar town, seeing the environment live, but with useful information overlaid on what they’re seeing through AR-enabled glasses or on a mobile device. This information might be directions to a meeting location, or local places of interest. The individual might issue a voice command, “show me the top-rated restaurants on this street” with the AR immediately altering the overlaid information in response.

Some Axis partners are already delivering apps that allow for information to be overlaid on live video. CamStreamer develops apps that allow for live streaming of video from Axis cameras to popular video sites. Its CamOverlay app enables customers to add graphics and information to live video. This example from Prague shows a live camera pivoting to view different views of the city, with the overlaid information changing to suit.

AR can effectively be an “insight superpower”, adding valuable information to what someone’s actually seeing, hearing, touching…even smelling and tasting, without having to use their hands to access it.

When thought about in this way, our imaginations can run wild.

AR in video surveillance

Modern network video surveillance cameras already provide extremely high-quality images to operators which in themselves deliver significant value. Live views of a scene in forensic levels of detail allow operators to monitor and assess situations, and respond in real-time to incidents and activity, alerting first responders when relevant.

In situations when operators are then directing first responders on the ground, the potential benefits in additional information being overlaid on the scene are clear. Some Axis cameras already include an orientation aid, which can overlay street names and compass points to live video footage to assist in providing accurate directions.

But when seconds can make a real difference, there could be many other useful pieces of information delivered through AR overlay. Understanding the location of the closest defibrillator, for instance, and being able to direct people at the scene via live audio could be a life-saving enhancement.

Helping first responders with the layouts of building, entrances and emergency exits, whether remotely from the operations room or locally via mobile devices, could hasten building evacuation or finding people trapped inside. Safely dispersing crowds from areas through the most effective routes might reduce the risk of incident escalation – again, this could be managed remotely from the control room with information overlaid on live video monitors, or locally when delivered on mobile devices.

Gazing even further forwards

If we allow our imaginations to wander a little further into the future, we could see how a combination of existing and live data could be employed to bring vital insight to first responders and security personnel.

Perhaps a police officer could view video with information overlaid using data aggregated from sensors such as installed surveillance cameras, body-wearable cameras, and even cell phone location data. Video could be overlaid on the video to direct the officer to the location of a cell phone that was used to initiate a call to emergency services requesting assistance, speeding up a search for someone in distress.

It is also not beyond the realms of imagination to think that medical staff and paramedics could use AR to help provide assistance and attention to people at the scene of an accident, or that live feed of surveillance from the scene would assist them with relevant preparations while on route.

Beyond emergency services

The use cases above provide some of the most compelling scenarios in which AR might have a significant positive impact on public safety, but examples in other sectors can also be imagined.

For example, personnel in an industrial plant could receive data from connected sensors throughout the site – video surveillance, but also temperature, air quality sensors and smoke detectors as examples – which would allow them to react to issues more rapidly, while also having a clear and live view of the scene.

Alerts related to sudden increases in temperature, breaches of perimeters, specific noises (such as raised voices) could bring live images to video monitors, while mapping directions of the fastest route to the scene, or to locations of fire extinguishers and alarms. In the case of a break-in, video surveillance cameras tracking the intruder might even leave a trail of ‘breadcrumbs’ overlaid on video surveillance for security officers to follow.

The potential applications are endless.

AR and video surveillance – sooner rather than later?

Technologically, there’s very little – if anything – to stop the development of solutions such as those detailed above. Indeed, as we mention, there may well be companies already developing this type of AR application, they just need to find the right partner…

Which might be a good time to direct you to the Axis Developer Community and Application Development Partner (ADP) programme. It’s where the future gets created, and often sooner than you think.

ACAP: A platform for innovation



Access tools to develop ACAP applications
  • The Axis Developer Community is open to anyone interested in developing software for integration with Axis products and technologies. ​Designed to support individual developers as a forum for ideas, collaboration and information sharing, the community gives members immediate access to the tools needed to develop ACAP applications. Join the Developer Community
  • If you are already part of Axis ADP program you find the latest ACAP tools on Axis partner pages. Find the latest ACAP tools on Axis partner pages