Skip to content

LLM Connect Issues

Ollama 500 Error

A 500 error from Ollama usually means the model is too large for your available memory.

Diagnosis

# Check which models are installed
ollama list

# Check if a model is loaded and using GPU
ollama ps

# Test the model directly
ollama run qwen3.5:4b

If ollama ps shows 0% GPU, inference runs entirely on CPU and will be very slow.

Fix: Use a Smaller Model

Available RAM/VRAM Recommended Model
4 GB qwen3.5:2b
8 GB qwen3.5:4b
16+ GB (or 8+ GB VRAM) qwen3.5:8b 
ollama pull qwen3.5:4b

Then select the new model in Murmure's LLM Connect settings.

Ollama Not Detected

Murmure may not auto-detect Ollama in some configurations.

Fix: Manually set the Ollama URL in LLM Connect settings:

  • Local Ollama: http://localhost:11434
  • Remote Ollama: http://<server-ip>:11434

LLM Output Contains Quotes or Tags

Some models wrap their output in quotes ("...") or include thinking tags (<think>...</think>).

Fix:

  1. Use recommended models: Qwen 3.5, Ministral
  2. Add to your system prompt: "Output only the result. No quotes, no thinking, no explanation."

LLM Response is Very Slow

  • No GPU: LLM inference on CPU is slow. Consider a smaller model or getting a GPU.
  • Model too large: If the model doesn't fit in VRAM, it falls back to CPU. Use ollama ps to check GPU usage.
  • First request: The first request after launching is slower (model loading). Subsequent requests are faster.

Remote Server Connection Issues

For remote Ollama or OpenAI-compatible servers:

  1. Verify the server is reachable: curl http://<server>:<port>/api/tags
  2. Check firewall settings on both machines
  3. Ensure the URL in Murmure includes the protocol (http:// or https://)
  4. For Ollama, make sure OLLAMA_HOST=0.0.0.0 is set on the server to allow remote connections

Server without a /models endpoint

Some OpenAI-compatible servers, such as the Claude API, do not expose /models. Test Connection then fails on the model list, but the server can still be used: type the exact model name in the model field and the Remote provider stays available. A failed model list never locks the provider.

Proxy support

HTTP proxy for LLM Connect is not yet supported. If you need proxy support in an enterprise environment, please comment on #286.