【Diffusers】Stable Diffusion Mixture Tiling を使ってみる

最終更新日：2023年6月3日

はじめに

Diffusersに新たにCommunity Pilelineとして追加された「Stable Diffusion Mixture」を使ってみました。

「家」と「道」と「ロボット」を別々のプロントで一つのモデルに与えるとそれを合成（Mix）した画像を生成してくれました。

環境構築

pip install torch==2.0.1+cu117 torchvision==0.15.2+cu117 --index-url https://download.pytorch.org/whl/cu117
pip install git+https://github.com/huggingface/diffusers
pip install transformers accelerate safetensors
pip install scipy
pip install ligo-segments

Pythonスクリプト

from diffusers import LMSDiscreteScheduler, DiffusionPipeline

scheduler = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", num_train_timesteps=1000)
pipeline = DiffusionPipeline.from_pretrained(
    "./stable-diffusion-v1-4", 
    scheduler=scheduler,
    custom_pipeline="mixture_tiling",
    )
pipeline.to("cuda")

# Mixture of Diffusers generation
image = pipeline(
    prompt=[[
        "A charming house in the countryside, by jakub rozalski, sunset lighting, elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece",
        "A dirt road in the countryside crossing pastures, by jakub rozalski, sunset lighting, elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece",
        "An old and rusty giant robot lying on a dirt road, by jakub rozalski, dark sunset lighting, elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece"
    ]],
    tile_height=640,
    tile_width=640,
    tile_row_overlap=0,
    tile_col_overlap=256,
    guidance_scale=8,
    seed=7178915308,
    num_inference_steps=50,
)["images"][0]

image.save("mix.png")