Samsung Is Still Trying To Reinvent AR Camera Filters

- September 28, 2018

In short: Samsung is looking to push AR stickers and AR filters to the next level by using more advanced facial recognition features, a newly published patent reviewed by AndroidHeadlines suggests. The technology would effectively utilize facial recognition features to detect not only whether a face is present within the field of view of a camera and the boundaries to that face. It would also detect and further define various characteristics of the face, such as forehead or jaw scale, in order to determine how to best "synthesize" with that face. For example, Samsung says in the associated document that those aspects may help some internal algorithm, presumably built around machine learning, determine which AR stickers are available and where on the user's face to place them.

Background: AR features such as dynamic stickers have been growing in popularity since at least last year and Samsung's own take on those were introduced more recently with AR Emoji. This newer patent seems to be building on the underlying technology found in the latter of those. AR Emoji effectively tracks the user using specialized algorithms in order to generate and allow direct manipulation of an avatar with a variety of features based on the user's posture and face. The patent, on the other hand, describes the placement of stickers and similar elements, as well as background manipulation and other features based on attributes of the users face. It can also be applied to more than one user within the same frame if the associated patent images are anything to go off of. That could essentially allow background environment and other objects or AR elements to be added dynamically without relying on an avatar to keep everything smooth. Bearing that in mind, it almost certainly would allow for stickers to be added in a similar fashion but with some extra algorithms in place to stabilize the stickers and apply them more accurately using specific measured characteristics of the user or users' face or faces.

Impact: While there's no guarantee that Samsung will ultimately use this patent since it could always find a better way to accomplish the same goal, it could represent a real step forward for AR and mobile. By building on its own achievements in the past, it would essentially open up AR features in a way that bolsters personalization while expanding on just how comprehensive and dynamic those features can be.