Apple introduces MGIE AI model for image editing; Know how to use it and what it can do

Apple's open-source AI model, MGIE, is capable of performing various image editing tasks using instruction-based editing and Multimodal Large Language Models (MLLMs).

Apple MGIE has been developed in collaboration with researchers from the University of California. (REUTERS)

For years, Apple has been aloof when it comes to artificial intelligence (AI)...but not anymore. The iPhone maker has now released an open-source AI model called MLLM-Guided Image Editing (MGIE), which can deal with various aspects of editing images. Apple has long been touted to be working on several AI projects, but the majority of them have been rumoured to be focused on AI upgrades for its suite of hardware products such as the iPhone and iPad, as well as Siri, its voice assistant. The release of the MGIE AI model is a suggestion that the Cupertino-based tech giant is aiming to compete in not just consumer technology, but on all fronts.

Apple MGIE AI model: What is it?

MGIE is an instruction-based image editing tool that uses Multimodal Large Language Models (MLLMs) as a foundation. It has been developed by Apple in collaboration with researchers from the University of California, Santa Barbara, and was presented as a research paper at the International Conference on Learning Representations (ICLR) 2024.

Instruction-based image editing allows for greater manipulation of images. The process works in two steps. MGIE first derives expressive instructions from natural commands instead of requiring elaborate prompts. For example, a user can give a command “make the colour of the grass more green”. What MGIE understands is “increase the saturation of the grass by X percent”.

In the second step, the AI model uses MLLM to generate a visual representation of the command and even manipulate it at the pixel level. The research paper states that MGIE uses end-to-end training to capture the visual imagination.

Apple MGIE AI model: What can it do?

Harnessing the power of AI, MGIE is capable of performing various tasks including Photoshop-style modification, global photo optimization, and local editing. In layman's terms, MGIE has the potential to manipulate images with expressive and concise commands. Moreover, it can achieve all tasks that Adobe Photoshop can do, including image cropping, resizing, removing objects, applying Gaussian blur, adding filters, and more.

The Apple MGIE AI model can also change various aspects of the image such as its brightness, saturation, sharpness, colour balance, and exposure, while also adding artistic filters such as sketching, watercolouring, pop art, and more. Instead of editing the image as a whole, MGIE can also manipulate certain parts of it, including removing or moving objects, and tweak its attributes such as texture, colour, shape, style, and size.

According to the research paper, experiments have found that expressive instructions are crucial to instruction-based image editing. Using them, MGIE can “lead to a notable improvement in automatic metrics and human evaluation while maintaining competitive inference efficiency.”

Also, read these top stories today:

Cookies are crumbling! The little data files that helped companies stalk users around the web are vanishing. But that doesn't mean a return to privacy. Some interesting details in this article. Check it out here.

Meta will challenge the EU! Meta announced on Wednesday it would challenge in court an EU demand for fees under a content moderation law, which is its legal weaponry to rein in Big Tech. Read all about it here.

Microsoft to cut more jobs! The FTC seeks a response after Microsoft's plans surfaced revealing that the Satya Nadella-led company aims to cut 1900 jobs from the newly acquired Activision Blizzard. Dive in here.

One more thing! We are now on WhatsApp Channels! Follow us there so you never miss any updates from the world of technology. ‎To follow the HT Tech channel on WhatsApp, click here to join now!