iOS 14 introduced many new features integrated, including under iPadOS 14. But the absolute highlight among the iPad features is the Apple Pencil Scribble function. This is how Scribble was developed for iPads.
With iPadOS 14, it is now possible to use the Apple Pencil to take handwritten notes, which are then automatically converted into typed text. The feature also supports Safari search and much more. But how exactly was this groundbreaking feature developed? Conversation Speaking to Popular Mechanics, Craig Federighi, Apple's senior vice president of software engineering, shared more details about how Scribble works. Federighi explained that the basis of the Scribble feature is data collection.
iPadOS 14 Scribble: Apple follows a different methodology
While the feature was being developed, Apple had many people around the world write things with the Apple Pencil. Based on this data, Scribble was only possible. The interview states:
When it comes to understanding (handwritten) strokes, we collect data. We find people all over the world and let them write things. We give them an Apple Pencil and let them write quickly, we let them write slowly, with a tilt and more. So in different variations.
This methodology differs from the comparatively simple approach of scanning and analyzing existing handwriting. Federighi explained that for Apple's technique, static examples were not enough. They needed to see the strokes that make up each letter.
“The computing power of the iPads must be high”
As with most machine learning features on Apple products, all processing is done on the device. This opens up another set of challenges to overcome. Federighi explains:
It has to happen in real time, right now, on the device you're holding. That means the device has to have enough processing power to run that level of processing locally.
The Scribble feature, which was introduced in iPadOS 14, can currently only be used in English and Chinese. When support for the German language will be introduced remains open for now. AirPods Pro on special offer at Amazon – available now. (Image: Apple)
