You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
|
|
|
## Building
|
|
|
|
|
|
|
|
|
|
Select one of the following models to build:
|
|
|
|
|
|
|
|
|
|
- `gpt-neo-1.3B`
|
|
|
|
|
- `gpt-neo-2.7B`
|
|
|
|
|
- `gpt-neox-20b`
|
|
|
|
|
- `pygmalion-6b`
|
|
|
|
|
- `gpt-j-6b`
|
|
|
|
|
|
|
|
|
|
```BASH
|
|
|
|
|
docker build --build-arg MODEL_NAME={model name} -t repo/image_name:tag .
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
```BASH
|
|
|
|
|
pip install --upgrade pip
|
|
|
|
|
git clone https://github.com/AlpinDale/gptq-gptj.git
|
|
|
|
|
cd gptq-gptj
|
|
|
|
|
pip install -r requirements.txt
|
|
|
|
|
CUDA_VISIBLE_DEVICES=0 python gptj.py PygmalionAI/pygmalion-6b c4 --wbits 4 --groupsize 128 --save pygmalion-6b-8bit-128g.pt
|
|
|
|
|
CUDA_VISIBLE_DEVICES=0 python gptj.py PygmalionAI/pygmalion-6b c4 --wbits 4 --groupsize 128 --save_safetensors pygmalion-6b-8bit-128g.safetensors
|
|
|
|
|
scp -P 22023 pygmalion-6b-8bit-128g.safetensors will@xd0.de:/home/will/
|
|
|
|
|
scp -P 22023 pygmalion-6b-8bit-128g.safetensors will@xd0.de:/home/will/
|
|
|
|
|
```
|
