This gist works as a hands-on note for running llama.cpp on various GPU. It may out-of-date due to the proj update.

This is only a personal record so readers may not have an out-of-box hands-on experience

Record the verified configs. The project is still developing very fast, so the granularity for the record is specified to the commit id. | Imple. | Device | OS | llama.cpp version | 3rd party version | Step | | ------ | --------------------------------- | ------------ | ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------

task (required): forecast | anomaly detection | simulation
training opt (when applied): hyperparameter tuning
inference opt (when applied): onnxruntime | quantization
others (when applied): distributed| customized model

Predict Number of Taxi Passengers with Chronos Forecaster: forecast

Tune a Forecasting Task Automatically: forecast, hyperparameter tuning

background: In Ray AI Runtime (AIR), Trainer is used to scale deep learning/machine learning model's training and tuning process. There is a TorchTrainer for (data parallel / local) pytorch (training / tunning) by providing customized train_loop_per_worker.

motivation1: pytorch-lightning is getting popular among AI researchers and machine learning engineers. The LightningModule in pytorch-lightning contains the customized training loop. It could be user-friendly and more straight-forward if users can only input a LightningModule to a Ray AIR Trainer's initialization process and generate an instance of PytorchLightningTrainer for distributed training / hyperparameter tuning.

motivation2: CPU training is a valid choice for AI models' training, while currently CPU training acceleration is not emphasized. bigdl-nano is a python package to transparently accelerate Pytorch and Tensorflow applicat