Live Text and Visual Look Up: Use Your iPhone Camera for More Than Just Pics

pcmag.com:

Apple’s Photos app features shockingly powerful tech for what, at a glance, seems like a nondescript photo library. The app has well-known AI capabilities that feature a host of editing features. Not only can Photos crop, enhance, sharpen, and otherwise modify your pics, but the AI recognizes text within pictures with Live Text, letting you highlight, copy, and even translate words. With iOS 16, this feature is even more advanced, giving you the ability to highlight text in videos, too.

This same tech powers Apple’s Visual Look Up, a new iOS 16 feature that lets you identify and isolate a photo's subject from its background. In addition, it lets you discover more information about said subject by pulling data from the internet and Siri. On a superficial level, Visual Look Up can highlight a photo's subject, and copy and paste into another app, such as Mail or Messenger. However, Visual Look Up has tremendous potential; though still in its infancy, its practical applications are impressive.

Live Text and Visual Look Up are easily the most remarkable features coming to iOS 16, and are ones we had the most fun exploring in Apple's mobile operating system. These are our initial thoughts.

Live Text lets you copy and share text located within photographs. The AI identifies key terms within the photo, and offers relevant functions. To test this, I photographed a page from Julia Child’s Mastering the Art of French Cooking. When looking at the images within the Photos app, I tapped the Live Text icon in the screen’s bottom-right corner, which highlighted the text. Measurement conversion options appeared at the photo's bottom for some of the key instructions. They displayed and converted the default Fahrenheit baking temperature