Adding another arrow to its quiver of augmented reality acquisitions, Apple has reportedly acquired Spektral, a computer vision company with technology for real-time compositing (otherwise know as the "green screen" technique in broadcast TV and film).
According to Danish newspaper Børsen (subscription required), Apple actually bought the company, founded as CloudCutout by Toke Jansen and Henrik Paltoft, late last year for $30 million. Jansen's LinkedIn profile shows that he has served as manager of computational imaging for the company since Dec. 2017.
Spektral's technology applies machine learning and spectral graph theory (the study of connections on a graph) to remove backgrounds from images and videos in real time.
With an equation that represents the layers in an image, the technology solves for unknown values in each pixel. This results in 16.8 million unknown values in each frame, and Spektral solves for these unknowns 60 times every second. Spektral is capable of compensating for fine details and fast motion with the same approach.
"The human visual system is extraordinary at recognition, segmentation, and compositing, simply through thought. Researchers have for decades been intrigued by these remarkable skills, and have therefore tried to build computer systems with similar properties," the company's website states. "This is notoriously a very challenging problem, but recently, computers are able to reach human precision in certain recognition tasks. With no doubt this trend is bound to continue, bridging the gap between human cognition and artificial intelligence. The implications of these advances pave the way for new business models, as tasks that previously required human intervention can now be solved automatically by algorithms."
Apple actually introduced a similar background-removing feature in its Clips app, though the feature requires the TrueDepth camera on the iPhone X. Google has also developed a method for changing video backgrounds using neural networks, while Facebook and Snapchat offer background segmentation.
Nonetheless, Spektral joins a formidable stockpile of Apple's augmented reality talent and tools, including AR display maker Akonia, headset maker Vrvana, and Flyby Media (which developed the code behind ARKit).
Just updated your iPhone to iOS 18? You'll find a ton of hot new features for some of your most-used Apple apps. Dive in and see for yourself:
2 Comments
Great acquisitionSpektral, Apple. I love all the enhancements to Photos and in-camera we can have access to enhance our digital photography experience.
Nice
Share Your Thoughts