iPhone 16 Pro to get big AI upgrade with Ferret-UI: Know what it is

Apple pioneers AI integration in iPhone 16 Pro to enhance user experience. Discover Ferret-UI's capabilities and upcoming server-side tools for individual iPhones.

Apple's upcoming iPhone 16 Pro promises groundbreaking AI advancements to enhance user experience and accessibility. (AP)

Apple's iPhone has long been renowned for its sleek design and user-friendly interface, but the tech giant is now venturing into the realm of artificial intelligence (AI) to further elevate the user experience.

Ferret-UI Technology

In a recent research collaboration with Cornell University, Apple has revealed a groundbreaking advancement titled "Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs." This innovative system employs a multimodal large language model to comprehensively interpret mobile user interfaces, revolutionising how users interact with their iPhones, Forbes reported. 

Not sure which
mobile to buy?

Ferret-UI, an evolution of the previously introduced Ferret system, boasts an array of capabilities, including icon recognition, text detection, widget parsing, and screen description. This enhanced functionality not only facilitates seamless navigation across various applications but also paves the way for enhanced accessibility features.

Moreover, Ferret-UI holds promise for developers, serving as a versatile testing tool for simulating diverse user scenarios and optimising app performance across different demographics.

Server-Side Tools Unveiled

In a recent development, backend code discoveries by Nicolás Álvarez hint at additional server-side tools dubbed "Safari Browsing Assistant" and "Encrypted Visual Search." While these features align with the principles outlined in the Ferret-UI research, it's important to note that they remain subject to potential modifications or may serve as placeholders for future implementations.

Despite commendable strides in AI, Apple faced criticism for lagging behind Google in embracing AI-driven smartphones. However, with the impending Worldwide Developer Conference in June, Apple is poised to shed light on its AI roadmap, laying the groundwork for the highly anticipated iPhone 16 and iPhone 16 Pro slated for release in September.

As the tech industry eagerly awaits Apple's next move, the academic revelations surrounding Apple's AI endeavours offer a glimpse into the company's trajectory as it strives to remain at the forefront of innovation.