| Initial Release | 10 July 2023 (Version 1.0) |
|---|---|
| Original Author / Maintainer | Oobabooga |
| GitHub Link | Link |
| License | AGPL-3.0 license |
| Batch Generation |  |
| Chat | |
| Training | |
| Quantization |  |
| Run on CPU alone | |
| Run on GPU / CUDA | |
| GUI | |
| Model Formats | GGML, GPTQ |
| LoRA Formats | |
Oobabooga (also known as Text generation web UI) as a python based web UI for interacting with locally hosted large language models.
Features[edit | edit source]
- 3 interface modes: default, notebook, and chat
- Multiple model backends: transformers, llama.cpp, ExLlama, AutoGPTQ, GPTQ-for-LLaMa
- Dropdown menu for quickly switching between different models
- LoRA support: load and unload LoRAs on the fly, train a new LoRA
- Precise instruction templates for chat mode, including Llama 2, Alpaca, Vicuna, WizardLM, StableLM, and many others
- Multimodal pipelines, including LLaVA and MiniGPT-4
- 8-bit and 4-bit inference through bitsandbytes
- CPU mode for transformers models
- DeepSpeed ZeRO-3 inference
- Extensions
- Custom chat characters
- Very efficient text streaming
- Markdown output with LaTeX rendering, to use for instance with GALACTICA
- Nice HTML output for GPT-4chan
- API, including endpoints for websocket streaming (see the examples)