Why Augmented Reality is mature for design

The Benefits of Augmenting Reality

We live in an increasingly interconnected world thanks to technological breakthroughs like mobile computing, the internet, and GPS. Yet, the digital realm remains separated, only accessible through entry points like terminals or screens. This means that a person has to decide between being present in the real world or focused on a screen, which can lead to impractical situations where they end up being completely oblivious to the other.

Augmented Reality (AR) is a technology where computer-generated content is integrated into a person’s experience, typically as overlaid images and graphics. This synthesis of the real world and digital content has many advantages and unique use cases, such as allowing the person to remain focused on the real world while they can receive context-specific information, enhancing their situational awareness, providing entertainment and much more. AR is different from Virtual Reality (VR), where the user is completely immersed in a digital world. By having the potential to transform our computing paradigm, AR may represent the next computational paradigm, Spatial Computing, where the world, instead of the screen, is our interface.

The adoption of AR in both the commercial and industrial sector has been slow to take off, mainly due to several technological and design hurdles such as expensive and heavy AR headsets with poor ergonomics, poor battery life and field-of-view and lack of exemplary use-cases. Fortunately, there is much progress within the field, and from what we have seen in AWE 2017, most of these issues are now solved.

Return On Investment of AR

A definite indication that AR is now maturing as a technology, are the evaluations of pilot projects done by different companies. These pilot projects, which were shared at AWE 2017, have been done within industrial contexts to see if AR can improve the efficiency and performance of workers, and the results are, to put it mildly, promising. (Source: General Electric (GE) Healthcare and Skylight AR.)

Increased Worker Performance

• GE Healthcare tested using Skylight's AR platform for warehouse logistics, with the results showing a warehouse worker increasing performance by 42 %. Looking at the video you can see that unlike the worker on the left who has to walk all the way to a stationary computer, the worker with the AR headset (on the right) interacts with the head-mounted display to access and input digital information.

• GE also tested windmill technicians efficiency in wiring wind turbines, using visible instructions and checklists enabled by AR, with performance increasing by 34 % on first use.

• Boeing and UpSkilled have tested wiring harness assembly for aircraft-technicians and showed productivity returns of 25 %.

• Bosch has integrated AR in Car Service workshops, with a time savings of 15 %.

These productivity leap capabilities of AR could be a smart investment that can positively affect the bottom-line for some companies. Furthermore, the error-elimination capabilities seem to be a good fit for the Norwegian industrial sector, that often works with complex and high-risk contexts.

Accessible AR platforms

Another positive change within the AR-landscape is the now prevalence of AR hardware. Where there previously were few and expensive options, there now exists a multitude of different platforms with various advantages.

Ubiquitous smartphone AR

ARKit & ARCore - Perhaps the most interesting development are the recently launched Software Development Kits for both iOS and Android that simplifies the software development pipeline, and enables AR-functionality for many smartphones. By vastly opening up the market for consumers and making it attractive for developers, this might turn into the biggest driver for growth within the field.

It wouldn’t be the first time that the iPhone redefined a medium by creating a more modest alternative. The video game industry was dominated by expensive, complicated console games until the iPhone introduced the concept of low-stakes “casual" games like Candy Crush and Angry Birds.

AR heads-up displays

• • RealWare HMT-1 - This was one of the platforms that impressed us the most at AWE2017. During one of the talks, the presenter dropped the RealWare head-mounted-tablet on the ground, showcasing the device’s ruggedness in contrast to many of the other more fragile glasses. These features make this seem like a good choice for industrial applications: a swappable battery life of up to four hours, an OS running on Android, dust-resistant and waterproof case and a relatively cheap price.

• • DAQRI Smart Glasses - These glasses, shown in the beginning image of this article, have just become available and share many similarities with Microsoft’s Hololens, namely having powerful processing power and sensor-capabilities. Their coolest feature was the large Field of View, which was less noticeable than the Hololens.

• Vuzix - These are one of the oldest AR hardware companies with experience dating back to 1997. One of the cool features was the fact that their head-mounted-display could be mounted to just about anything.

• Other options that probably already are familiar is Microsoft’s Hololens and the relaunch of Google Glass.

Environmental recognition

Many current AR platforms have certain capabilities when it comes to environmental recognition: Apple’s ARKit and Google’s ARCore can currently detect two-dimensional planes and place objects on them. Microsoft’s Hololens can detect walls in addition and use those for content as well.

There is no point in building an AR App unless it interacts with the physical world in some way.

Current Environmental Recognition Capabilities

For AR to truly become general purpose device that people use in everyday life, environmental recognition will to be present. Imagine looking at any object, landmark, species or plant and having image recognition software linked to an encyclopedia, showing you useful information about it. We would no longer need to look things up, but just access relevant information about the environment by looking at it. This is actually possible today with machine learning based on image recognition. Case in point being Blippar where machine learning (github link) will try to identify what the device is looking at, and once identified, can show you information about it.

There are several other environmental-recognition development kits that are available now. Vuforia has developed Unity tools to make object recognition easier to implement, especially with their model tracking. And using their own tracking algorithms they support more devices than ARKit, ARCore and Microsoft Mixed Reality combined, using just one camera sensor. Tracking seemed not quite as stable as ARKit, but that will only improve. VisionLib also showcased similar model tracking technology at the conference.

In the future: mapping a 3D-model of earth

Looking into the future, a really interesting development within AR environmental recognition is Google’s Project Tango, which can map a 3D model of the environment using simple sensors (depth & motion tracking) and image processing. During AWE 2017, a talk by Jürgen Sturm from Google was about how everyone’s 3D-scanned environments could be uploaded into a common library in the cloud, which then could create a 3D-replicate of the earth. This would enable possibilities such as visual global positioning (VPS) without satellites that would be more accurate and also work indoors.

Conclusion - A need for UX and design

We believe we are on the verge of AR finally becoming a feasible technology for many use cases. This is driven by available AR hardware, falling costs of computing & sensors, a vastly expanding market through ARKit and ARCore. We at EGGS are excited about these new opportunities, yet it is also apparent that there still are many challenges when it comes to design and the User Experience. A good AR solution needs to balance usability, desirability, business viability and technological feasibility. We hope you would like to join us designing and developing for the AR future!

AWE Europe 2017 Expo area

See some of our video footage when visiting the AWE Europe 2017. Video by Kenneth Aastrøm.

See also

Eggs Design and the Hololens Augmented Reality Is Already Improving Worker Performance - Harvard Business Review How augmented reality could save tech from itself TechCrunch AWE EU 2017 Developers Track videos AWE EU 2017 Creators Track videos