Apple patent: Pencil lets you feel textures Apfelpatient

A newly granted Apple patent describes a stylus similar to the Apple Pencil that could make the surface of virtual objects tangible. Instead of just drawing, the input device would provide haptic feedback, indicating whether something feels rough, smooth, or uneven – thus making the experience in the Apple Vision Pro more tangible.

There's an old children's toy where two styluses are connected so that the movement of one moves the other. Apple's new patent is reminiscent of this principle – only here, the stylus doesn't transmit a drawing, but rather "surface roughness and other surface features," as well as motion data such as trajectory and speed, into virtual reality. The idea fits with the picture that Apple is continuing development around the Apple Vision Pro despite internal restructuring. The document is titled "Surface texture detection and emulation" and, while explicitly describing a "pencil" system, could, according to its wording, be applied to any handheld device.

An accessory for greater immersion in XR

The final form of the input device is likely of secondary importance: According to the patent, it is primarily an accessory for the Apple Vision Pro, which, according to Apple, is specifically designed to create "enhanced immersion during XR sessions" – i.e., in augmented reality.

This involves two tasks. The device would first have to capture the texture of a virtual object and then transmit it back to the user. Apple describes several ways to achieve this. For example, a gyroscope could be used that changes the surface feel depending on the device's rolling angle.

Haptic feedback designed to create or eliminate friction

The focus is on haptic feedback. The patent describes how an experience such as friction, texture, or roughness on a surface could be "created, replicated, or negated" – via forces, vibrations, or movements that the device transmits to the user.

In nearly 12,000 words and nine pages of drawings, Apple details the possible components. These include linear actuators that could "extend or retract part of the device," as well as an inertial measurement unit and color sensors. The drawings depict an Apple Pencil whose tip, depending on the variant, carries a camera, interferometer, or components simply described as "surface sensors."

Not Apple's first attempt at texture-recognizing styluses

This isn't the first time Apple has filed a patent for an Apple Pencil with texture recognition. However, the previous example targeted real-world applications rather than virtual reality and focused as much on color recognition as on texture.

The new patent shifts the focus: away from simply detecting or measuring, towards ways of making virtual reality feel more realistic. Conveying such feedback via a held stylus would be less cumbersome than the gloves also envisioned by Apple.

The patent is attributed to five inventors, including Nicholas C. Soldner. His previous work for Apple includes several annual patent applications related to ultrasonic sensors and the breathing detection in the Apple Vision Pro.

How resilient a granted patent really is

Even a granted patent is no guarantee that Apple will ever release a product based on it. Patents can be filed for a single component of a long, ongoing research project, and they sometimes serve to secure earlier knowledge in the event of future legal disputes.

Nevertheless, such a patent at least marks a topic to which Apple has dedicated research time. And it is further evidence that the company is pursuing its plans for the Apple Vision Pro and visionOS, contrary to rumors – a course that Ternus publicly reaffirmed with the statement, "We're just getting started." (Image: Shutterstock / Peshkova)