You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
|
|
|
# matrix-pygmalion-bot
|
|
|
|
|
|
|
|
|
|
## Setup
|
|
|
|
|
```sh
|
|
|
|
|
# install dependencies
|
|
|
|
|
apt install libolm-dev
|
|
|
|
|
pipenv install --dev
|
|
|
|
|
|
|
|
|
|
pipenv shell
|
|
|
|
|
python3 run.py
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Other
|
|
|
|
|
```sh
|
|
|
|
|
python3 -m venv env
|
|
|
|
|
source env/bin/activate
|
|
|
|
|
pip install -r requirements.txt
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Install KoboldCpp
|
|
|
|
|
```sh
|
|
|
|
|
git clone https://github.com/LostRuins/koboldcpp.git
|
|
|
|
|
apt update && apt-get install libopenblas-dev libclblast-dev libmkl-dev
|
|
|
|
|
cd koboldcpp && make LLAMA_CLBLAST=1 LLAMA_OPENBLAS=1
|
|
|
|
|
cd models && wget https://huggingface.co/concedo/pygmalion-6bv3-ggml-ggjt/resolve/main/pygmalion-6b-v3-ggml-ggjt-q4_0.bin
|
|
|
|
|
python3 koboldcpp.py --unbantokens --smartcontext --stream models/pygmalion-6b-v3-ggml-ggjt-q4_0.bin
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Cloud services
|
|
|
|
|
* runpod.io
|
|
|
|
|
* vast.ai
|
|
|
|
|
* stablehorde.net
|
