https://touch-sp.hatenablog.com/entry/2025/03/18/222459

はじめに

以前「Gradio」を使いました。今回は「ChatUI」を使います。
touch-sp.hatenablog.com
Gemma-3-27B-itの実行は前回同様vLLMを使っています。

vllm serve google/gemma-3-27b-it --quantization bitsandbytes --load-format bitsandbytes --max-model-len 2048 --limit-mm-per-prompt image=2

ChatUIの設定

.env.localは以下のようにしました。こちらを参考にしています。

MODELS=`[
      {
        "name": "google/gemma-3-27b-it",
        "id": "google/gemma-3-27b-it",
        "multimodal": true,
        "logoUrl": "https://huggingface.co/datasets/huggingchat/models-logo/resolve/main/google-logo.png",
        "description": "Google's latest open model with great multilingual performance, supports image inputs natively.",
        "websiteUrl": "https://blog.google/technology/developers/gemma-3/",
        "parameters": {
            "max_new_tokens": 1024,
            "stop": ["<eos>", "<pad>"],
        },
        "endpoints": [
          {
            "type": "openai",
            "baseURL": "http://localhost:8000/v1",
            "multimodal": {
              "image": {
                "maxSizeInMB": 10,
                "maxWidth": 560,
                "maxHeight": 560,
                "supportedMimeTypes": ["image/jpeg"],
                "preferredMimeType": "image/jpeg"
              }
            }
          }
        ]
      }
]`

MONGODB_URL=mongodb://localhost:27017

ハマりポイント

System Promptを空欄のまま実行しようとすると以下のエラーが出ました。なにか入力するとエラーは消えます。

[22:12:46.700] ERROR (7668): 400 status code (no body)
    err: {
      "type": "BadRequestError",
      "message": "400 status code (no body)",
      "stack":
          Error: 400 status code (no body)
              at APIError.generate (file:///D:/chatui/gemma-3-4b-it/node_modules/openai/error.mjs:41:20)
              at OpenAI.makeStatusError (file:///D:/chatui/gemma-3-4b-it/node_modules/openai/core.mjs:268:25)
              at OpenAI.makeRequest (file:///D:/chatui/gemma-3-4b-it/node_modules/openai/core.mjs:311:30)
              at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
              at async eval (D:/chatui/gemma-3-4b-it/src/lib/server/endpoints/openai/endpointOai.ts:226:38)
              at async generate (D:/chatui/gemma-3-4b-it/src/lib/server/textGeneration/generate.ts:26:30)
              at async textGenerationWithoutTitle (D:/chatui/gemma-3-4b-it/src/lib/server/textGeneration/index.ts:64:3)
      "status": 400,
      "headers": {
        "content-length": "108",
        "content-type": "application/json",
        "date": "Tue, 18 Mar 2025 13:12:46 GMT",
        "server": "uvicorn"
      }
    }