JarvisArt: Revolutionizing Photo Editing with AI and Human Collaboration

Bridging Artistic Vision and Technical Expertise

Photo retouching plays a vital role in digital photography, allowing users to enhance tones, exposure, and contrast to create visually stunning images. While professionals rely on advanced tools like Adobe Lightroom, casual users often face steep learning curves. On the other hand, AI-driven photo editing tools tend to oversimplify the process, lacking the nuanced control necessary for refined results.

Challenges with Current AI Photo Editing Solutions

Existing AI models and traditional retouching methods struggle to deliver fine-grained regional edits or maintain high-resolution image quality. Many rely on zeroth- and first-order optimization, reinforcement learning, or diffusion models but fall short in preserving image content or offering precise user control. Even state-of-the-art large models like GPT-4o and Gemini-2-Flash compromise detail fidelity when performing text-based edits.

Introducing JarvisArt: A Multimodal AI Photo Retouching Agent

JarvisArt is an innovative system developed through collaboration among several leading universities and industry partners. It integrates a multimodal large language model that comprehends visual and textual instructions to perform flexible, instruction-guided photo editing. Leveraging over 200 tools within Adobe Lightroom through a custom Agent-to-Lightroom (A2L) protocol, JarvisArt mimics professional artists' decision-making processes.

How JarvisArt Works

The system is built on three core components:

• MMArt Dataset: A comprehensive dataset containing 5,000 standard and 50,000 Chain-of-Thought annotated samples covering diverse editing styles and complexities.
• Two-Stage Training: Initially, supervised fine-tuning develops reasoning and tool-selection abilities. This is followed by Group Relative Policy Optimization for Retouching (GRPO-R), which uses specialized rewards to enhance retouching accuracy and perceptual quality.
• A2L Protocol: A seamless integration protocol that allows JarvisArt to execute Lightroom tools transparently and lets users adjust edits dynamically.

Performance and Real-World Application

Evaluated on the MMArt-Bench benchmark, JarvisArt demonstrated a 60% improvement in pixel-level content fidelity over GPT-4o while maintaining strong instruction adherence. It excels in both global and region-specific edits, capable of modifying elements like skin texture, eye brightness, and hair definition in images of any resolution. This balance of precision and user control makes JarvisArt a powerful tool for creative photo retouching.

Fusing Creativity with Precision

JarvisArt addresses the critical gap between automation and user control in photo editing. By combining data synthesis, reasoning-driven training, and commercial software integration, it offers an accessible yet professional-quality photo retouching experience without requiring expert knowledge.