LLaMA.cpp

LLaMA.cpp

Initial Release	18 March 2023 (2023-03-18)
Original Author / Maintainer	Georgi Gerganov
GitHub Link	Link
License	MIT License
Batch Generation	Y
Chat	Y
Training	N
Quantization	Y
Run on CPU alone	Y
Run on GPU / CUDA	Y
GUI	Via oobabooga or other programs
Code Bindings	Python, .net/C#, Go, NodeJS, Ruby, Scala 3.
Model Formats	GGML

LLaMA.cpp (also written as llama.cpp or LLaMA C++) is an implementation of the transformer model underlying LLaMA and other models written in C++. The implementation was originally targeted at running 4-bit quantized models on the CPU of a MacBook under MacOS, but now also supports running under Linux & Microsoft Windows as well as running models on one or more GPUs.

LLaMA.cpp is notable for being among the first tools available that allowed large language models to be run on personal computers without necessarily depending on GPU capacity.

Related Work

LLaMA.cpp is the main tool used with the GGML model format and library, which were developed in conjunction with it.

See Also

• The project's official GitHub page: [1]