Header Banner
Next Reality Logo
Next Reality
Augmented & Mixed Reality News, Rumors & Dev Guides
AppleSnapAR BusinessGoogleInstagram | FacebookNFTHoloLensMagic LeapHands-OnSmartphone ARThe Future of ARNext Reality 30AR GlossaryARKit Dev 101What Is AR?Mixed RealityHoloLens Dev 101Augmented RealityHololens How-TosHoloLens v. Magic Leap v. Meta 2VR v. AR v. MR

Apple iOS-Powered Sesame Street Yourself App Turns Kids into Their Favorite Characters via ARKit 3's New Features

Nov 6, 2019 01:00 PM
Children using fun mobile filters with Elmo and friends from a children's show.

The longest-running children's television show in the United States, Sesame Street, hasn't survived 50 years without leveraging new technology along the way.

The latest innovation from the Sesame Street Workshop is Sesame Street Yourself, a new iOS app that takes advantage of the new features of ARKit 3.

Available now via the Apple App Store, Sesame Street Yourself uses ARKit's multi-face tracking feature to identify and morph users into their favorite Sesame Street characters. The ad-free app also entertains children ages 3-5 with games, singing, and dancing.

"We just thought [ARKit 3] is a fantastic, gigantic toolbox of tricks. [We wanted to see] what we [could do with] the tech for fantastic immersive interactive experiences for children," said Baz Palmer, the co-founder of Weyo (the game's developer), in an interview with Next Reality.

Palmer and his team took advantage of ARKit's feature set for both the rear and front camera. For example, on the rear camera, children can help Cookie Monster bake cookies. On the front camera, they can become Ernie and try on different outfits.

Using the multi-face tracking technology of ARKit 3 gives parents the opportunity to play along with their children, with each face wearing a different character mask that tracks the individual as he or she moves within the camera view.

"Working with the Sesame brands, you've got these lovable characters and have a really clear demographic that will try to engage," said Palmer. "It really focused the way we want to use the technology."

Weyo originally included elements that took advantage of the People Occlusion and Motion Capture features of ARKit 3 as well, but, according to Palmer, the team excluded those features from the final release to keep the app lighter. However, they plan to offer those experiences later as add-ons to the app.

Since the app uses ARKit 3, it will be limited to compatible iPhones and iPads, namely iPhone XS, XS Max, and XR, the 2019 editions of the iPad Air and iPad Mini, and the latest 11-inch and 12.9-inch iPad Pro models.

This isn't the Sesame Street Workshop's first foray into augmented reality. In 2013, the organization integrated the Vuforia AR platform into Big Bird's Words, an educational game that used Vuforia's computer vision to help players find words in the real world. Vuforia also supplied the AR behind a prototype AR playset.

Nonetheless, Sesame Street Yourself is the first household brand name to adopt ARKit 3, which is significant considering that ARKit 2 never really caught on. The uniqueness of People Occlusion and Motion Capture might just be the secret ingredient to getting app developers to build more innovative AR experiences.

Cover image via Weyo

Just updated your iPhone? You'll find updated Apple Intelligence capabilities, new wallpapers, and enhancements to Calculator, PDF cropping, and Live Voicemail, among other useful features. Find out what's new and changed on your iPhone with the iOS 18.3 update.

Related Articles

Comments

No Comments Exist

Be the first, drop a comment!