Tuesday, September 20, 2016

Hololens: What Kind of Applications Can We Build?

The creative potential of Hololens and "Mixed Reality"is huge. There are a wide variety of applications we can develop for it.

I believe the demos shown so far are barely scratching the surface of what we can do with the Hololens. Once the imagination of the development community warms up, we're going to see world changing ideas. Industries will be turned upside down, lives will be changed, and millionaires will be made.

I'm going to make some assumptions on its abilities, but try and highlight some of the many directions we can take in creating for the interface of the future.

Presence Sharing

One of the most interesting uses for the Hololens is allowing others to share your experience, and interact with it. This opens up many prospects for communication and collaboration in virtually every industry.

Think about how cool GoPro cameras are and what we're able to do with them. Now imagine seeing the video live and being able to interact with it.

This particular feature also has plenty of promise for gaming, with users interacting with the same virtual objects, the same reality, or in the same game area.

Reality Overlays

Probably the main purpose of Hololens is its ability to overlay the imagined onto the real. We have seen some examples of this, and there is so much more to come. Interfaces will probably make up the bulk of these overlays, and there is plenty of scope for variety in how these work. But the possibilities go much further than just interfaces.

We've seen game characters and levels merge with the real world. What about augmenting our environment with photo-realistic people, new fixtures and furniture, or movie scenes? Or changing our atmosphere with movement, light, and sound? Virtual Reality might do this better. But having the real world still in view gives a certain edge to the atmosphere.

Gesture Controlled Augmentations

The use of gesture controlled interfaces might have been around for a while, but it takes on a new potency in the realm of artificial reality. Rather than interacting with things on a screen, you'll now be able to "physically" interact with "objects" in your world.

And while the recognisable gestures are still quite simple, this will only get more complex with time. As the Kinect technology gets better at recognising more fingers and more intricate movements, it will allow for more advanced control in much the same way as musical instruments or crafts.

Reality Recognition

Let's not forget that the Hololens cameras can be used to process the real world and the things in it. Measuring distances, identifying objects, detecting movement and more can all lend to the interaction.

How far can we take this real world processing? Imagine, tailoring your experience based on what the Hololens sees, such as popping up an information snippet of a visible landmark, finding your friends in a crowd, or identifying danger.


Like gesture controls, voice finds new opportunities in this context. It's part of the experience, narrating what you see, talking over the internet, recognising commands, sometimes recording. What else?


There are probably several more Hololens abilities that I've missed, so please leave a comment if you can think of any. Each of the items above has the potential for some groundbreaking new applications. Combine two or more concepts and the possibilities increase exponentially.

We just need to start thinking differently about what we can do. Rather than extrapolating our current computers and software to this new paradigm (which we should still do), we should try to think about what new possibilities all these new capabilities afford us, separately as well as combined.

I didn't want to get too deep into these possibilities, I really just wanted to highlight them, to get people thinking about how they might use each of them.

It'll probably take a few years for the world to realise what Augmented Reality can do for it. But all ideas are built on other ideas.

We're just getting started.