A hyperparameter in a large language model is a parameter that is used for specifying the model architecture, configuring model training or controlling inference. The term hyperparameter is used to distinguish from parameters which in the context of deep learning refers to the model parameters which are generated by the training process.