IPAdapter

IPAdapter is a lightweight, 22M-parameter module that enables image prompting for pre-trained text-to-image diffusion models (e.g., Stable Diffusion) without extensive fine-tuning.

This is the Image Prompt Adapter (IPAdapter), a highly efficient solution from Tencent AI Lab for multimodal image generation. It integrates image conditioning into models like Stable Diffusion, using a decoupled cross-attention mechanism to process both image and text features simultaneously. With only 22M parameters, IPAdapter delivers performance comparable to fully fine-tuned models, making it resource-friendly (under 100MB for SD 1.5). The technology excels at specific tasks: style transfer, composition cloning, and specialized face ID applications, all while maintaining compatibility with existing control tools like ControlNet.

https://github.com/tencent-ailab/IP-Adapter