Future Apple Pencil could be used to trace 3D objects for easy AR generation
New sensors being researched for a future Apple Pencil could allow users to use the device to trace both 2D and 3D objects, and send the data back to a connected iPad or Mac.
Future Apple Pencils could allow sketching or writing on any surface.
Following multiple patent applications regarding different ways of adding controls to the Apple Pencil, Apple has now looked outside the stylus to the surfaces that it can used on. Currently all Apple Pencils require a touch screen such as an iPad display, but future ones may become much more independent.
"Content creation using electronic input device on non-electronic surfaces," US Patent No. 10,613,666, proposes that the Pencil could be used on any surface at all. If the Pencil can be made to detect and then transmit when it is in contact with a surface, when it is moved, and so on, then it could be used anywhere.
Rather than being confined to a relatively small iPad screen, an artist could sketch out on a whole table, and that drawing would be accurately created on the device. Artists could trace out existing drawings or plans, or could produce 3D drawings of objects by tracing the Pencil over such an object in real life.
"In conventional systems including a touch-sensitive surface," says the patent, "content generation using an input device, such as an active stylus, can be based on a detected path between a touch-down location and a lift-off location."
"[The] touch-down and lift-off on a non-touch-sensitive surface can be monitored by a force sensor of the input device," it continues. "The position and/or motion of the input device can be tracked according to various methods including one or more of a motion and orientation sensor, a camera, or an electromagnetic- or sound-based triangulation scheme."
All of this can be used equally for drawing or writing. "The force data and position/motion data can be processed to generate content, including textual character input and three-dimensional objects," it continues.
It may even be possible to sketch in thin air. "In some examples, the content can be generated based on tracking position and/or motion of the input device without requiring contact with a surface," says the patent.
The proposed sensors even allow for tracing 3D objects
Apple isn't planning to replace the touch-sensitive display, however,, arguing instead that improving the Apple Pencil in this way will be useful in existing situations. "In particular, use of an active stylus capable of generating stylus stimulation signals that can be sensed by the touch-sensitive device can improve the precision of stylus input," it says.
The patent is credited to three inventors, Tyler S. Bushnell, Steven Cardinali, and Katherine E. Tong. Bushnell and Cardinali were previously listed together as two of the inventors on an Apple Watch patent that would see the Digital Crown gain touch-sensitive gestures.
Future Apple Pencils could allow sketching or writing on any surface.
Following multiple patent applications regarding different ways of adding controls to the Apple Pencil, Apple has now looked outside the stylus to the surfaces that it can used on. Currently all Apple Pencils require a touch screen such as an iPad display, but future ones may become much more independent.
"Content creation using electronic input device on non-electronic surfaces," US Patent No. 10,613,666, proposes that the Pencil could be used on any surface at all. If the Pencil can be made to detect and then transmit when it is in contact with a surface, when it is moved, and so on, then it could be used anywhere.
Rather than being confined to a relatively small iPad screen, an artist could sketch out on a whole table, and that drawing would be accurately created on the device. Artists could trace out existing drawings or plans, or could produce 3D drawings of objects by tracing the Pencil over such an object in real life.
"In conventional systems including a touch-sensitive surface," says the patent, "content generation using an input device, such as an active stylus, can be based on a detected path between a touch-down location and a lift-off location."
"[The] touch-down and lift-off on a non-touch-sensitive surface can be monitored by a force sensor of the input device," it continues. "The position and/or motion of the input device can be tracked according to various methods including one or more of a motion and orientation sensor, a camera, or an electromagnetic- or sound-based triangulation scheme."
All of this can be used equally for drawing or writing. "The force data and position/motion data can be processed to generate content, including textual character input and three-dimensional objects," it continues.
It may even be possible to sketch in thin air. "In some examples, the content can be generated based on tracking position and/or motion of the input device without requiring contact with a surface," says the patent.
The proposed sensors even allow for tracing 3D objects
Apple isn't planning to replace the touch-sensitive display, however,, arguing instead that improving the Apple Pencil in this way will be useful in existing situations. "In particular, use of an active stylus capable of generating stylus stimulation signals that can be sensed by the touch-sensitive device can improve the precision of stylus input," it says.
The patent is credited to three inventors, Tyler S. Bushnell, Steven Cardinali, and Katherine E. Tong. Bushnell and Cardinali were previously listed together as two of the inventors on an Apple Watch patent that would see the Digital Crown gain touch-sensitive gestures.
Comments
That would be a a dream come true but probably a stretch. There are very expensive and precise components in existing digitisers that achieve greater accuracy than this but at low volumes and currently targeted and industries where the market will bear $50000.
It will be a game changer if the Apple Pencil and the new LiDAR modules are accurate in iPhones and iPads. If they can, an iPad with Apple Pencil will suddenly become a relatively cheap option for people doing 3D CAD modelling and digitisation.