Apple’s secret VR headset just leaked an ingenious idea

Apple’s Reality Pro mixed-reality headset is probably just a few months from launching, but we’re still seeing the company’s top-secret ideas seeping out into the wild. The latest leak shows one way you might be able to control things in Apple’s metaverse — and it’s a pretty unusual concept.

According to a recently granted patent (number 2023/0042447 A1), Apple is exploring the idea of using an Apple Pencil as a sort of virtual reality (VR) controller. The idea is that your hand holding the Apple Pencil could be displayed in the mixed-reality world that you see through the headset, overlaying it onto augmented reality (AR) elements.

Apple says an image of your hand holding an Apple Pencil could be captured by sensors or cameras, for instance, those mounted into a mixed-reality headset. The patent then explains how that would let you generate “pencil-like marks” in the virtual world. One example could be a virtual meeting room where you are able to draw on a whiteboard and see the Apple Pencil making the marks in real time.

Similarly, you could flip the Apple Pencil over and use its blunt end as a kind of virtual eraser, removing the pixel markings you had just made in the mixed-reality space.

Hand-tracking and gesture control

It’s a pretty neat idea and goes some way to revealing what Apple could let you do with your hands while using its Reality Pro headset. So far, we’ve seen a few of Apple’s ideas on this topic, including others than include existing Apple products.

For example, a previously granted patent showed how a pair of Apple Watches could be used to record hand gestures in the metaverse. Elsewhere, another one of Apple’s concepts depicted small, thimble-like devices on the tips of a user’s fingers, which would register hand gestures and transpose them into mixed reality.

The benefit of the Apple Pencil idea is that for many people it might not require any additional purchase, as plenty of Apple fans already have an Apple Pencil. That would make it more accessible than a system that requires you to purchase a set of VR thimbles in addition to the already pricey headset itself.

That’s not to say this is definitely the route Apple will take. Since the information comes from a patent, it might never see the light of day, and it could simply be an idea that Apple is playing around with but will never actually put into practice.

Still, it seems like a clever way to take an existing Apple device and use it to bolster the mixed-reality world Apple’s Reality Pro headset will create. Either way, we should find out soon, as Apple is rumored to be launching the Reality Pro headset at a special spring event. Keep your eyes peeled.

Editors' Recommendations