How to beginner · 3 min read

How to use inpainting with Diffusers

Q: How to use inpainting with Diffusers

Use the Diffusers library's StableDiffusionInpaintPipeline to perform inpainting by providing an input image and a mask image highlighting the area to modify. Load the pipeline, prepare your images, and call the __call__ method with a prompt to generate the inpainted output.

Quick answer

Use the Diffusers library's StableDiffusionInpaintPipeline to perform inpainting by providing an input image and a mask image highlighting the area to modify. Load the pipeline, prepare your images, and call the __call__ method with a prompt to generate the inpainted output.

PREREQUISITES

Python 3.8+
pip install diffusers>=0.19.0
pip install transformers
pip install torch
pip install Pillow

Setup

Install the required packages and import necessary modules. You need diffusers for the inpainting pipeline, torch for tensor operations, and Pillow for image processing.

bash

pip install diffusers transformers torch Pillow

Step by step

This example loads the StableDiffusionInpaintPipeline from Diffusers, prepares an input image and a mask image, and runs inpainting with a text prompt.

python

import os
from PIL import Image
import torch
from diffusers import StableDiffusionInpaintPipeline

# Load the inpainting pipeline
pipe = StableDiffusionInpaintPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    torch_dtype=torch.float16
).to("cuda")

# Load input image and mask image
input_image = Image.open("input.png").convert("RGB")
mask_image = Image.open("mask.png").convert("RGB")

# Define prompt for inpainting
prompt = "A red apple on the table"

# Run inpainting
output = pipe(prompt=prompt, image=input_image, mask_image=mask_image).images[0]

# Save output
output.save("output_inpainted.png")
print("Inpainting complete, saved as output_inpainted.png")

output

Inpainting complete, saved as output_inpainted.png

Common variations

Use CPU by changing .to("cuda") to .to("cpu") if no GPU is available.
Adjust num_inference_steps or guidance_scale parameters in the pipeline call for quality and creativity control.
Use asynchronous calls with asyncio if integrating into async applications.
Try different inpainting models by changing the model name in from_pretrained.

python

output = pipe(
    prompt=prompt,
    image=input_image,
    mask_image=mask_image,
    num_inference_steps=50,
    guidance_scale=7.5
).images[0]
output.save("output_inpainted_variation.png")

Troubleshooting

If you get CUDA out of memory errors, reduce batch size or switch to CPU.
If images do not align, ensure input and mask images have the same dimensions.
For blurry results, increase num_inference_steps or adjust guidance_scale.
Ensure mask_image is a binary mask where white areas represent regions to inpaint.

✅

Key Takeaways

Use StableDiffusionInpaintPipeline from Diffusers for inpainting tasks.
Provide both an input image and a mask image highlighting the area to modify.
Adjust num_inference_steps and guidance_scale for output quality control.
Ensure input and mask images have matching dimensions and correct formats.
Use GPU for faster inference, fallback to CPU if needed.

Verified 2026-04 · runwayml/stable-diffusion-inpainting

Verify ↗