Part 2: The Sixth Protocol

“Mixed reality is the eyes and ears of AI.”

• Lili Cheng, VP of Microsoft

When you wake up in the middle of the night to go pee, and you don’t want to wake your partner, you’re lucky to find that you know how to grab the bathroom handle in the dark. Nature has equipped you with complex spatial reasoning, allowing you to navigate the world and pick your nose with your eyes closed.

What makes this possible are often-overlooked senses that we deploy as we move ourselves about the world.

Our sense of proprioception is how we understand the position and movement of our physical body without looking at it, and our sense of direction is how we understand our place in relation to the world we inhabit.

Today, digital devices largely lack these fundamental senses, relying on extremely rudimentary and low-resolution approximations of space.

For Elon’s vision of humanoid robots entering the workforce or Naval’s vision of coordinating self-driving cars, digital devices must acquire a collaborative sense of machine proprioception, sharing, and reasoning about spatial data together.

Microsoft VP Lili Cheng highlighted this truth succinctly when she said that mixed reality is the eyes and ears of AI. Spatial computing, the sixth protocol, is the bridge between the digital world and the physical world we inhabit.

6) Spatial Layer: A universal spatial computing protocol, allowing machines to collaboratively reason about physical space.

Microsoft is not alone in this realization. Bubbling just under the surface of the public’s attention, hidden in the buzz of the metaverse and AI, is one of the greatest capital allocations and arms races in human history.

Every major player in the tech industry is preparing for the advent of the spatial internet, scrambling to solve the problem of machine proprioception so that they can power the AI revolution, smart cities, and the transition from mobile to spatial computers.

Many analysts have noted that Tesla is not really a car company but an AI company. Even this analysis misses what is really going on and what Elon is driving at (pun intended):

The greatest assets that Tesla has are the millions of moving cameras building up a spatial model of the world for its cars and robots to reason about. What would make Tesla’s AI superior to that of OpenAI is that it is being trained on the real world instead of its human-curated internet derivative. Tesla is one of the world’s most successful and ambitious spatial computing companies, giving it a long-term edge in the race for AI dominance.

A purely semantic understanding of the world is insufficient to tackle many of the tasks we dream of AI and robots solving for us. Even simple tasks like “Go fetch the book on my bed” require spatial reasoning, so an AI confined to the internet will be perpetually disappointing.

AI without spatial computing is an anemic vision of the future. No matter its inherent intelligence, an AI confined to the internet is only in its infancy. What kind of machine god could not pick its nose in the dark?