hendrik
/
matrix-pygmalion-bot
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Chatbot
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
120 Commits
1 Branch
0 Tags
1.0 MiB
 
 
 
 
Tree: d112b273df
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'd112b273df'
${ noResults }
matrix-pygmalion-bot/runpod/runpod-worker-oobabooga-api
Hendrik Langer b8d21fbb40
remote worker 		2 years ago
..
Dockerfile remote worker 2 years ago
README.md add runpod oobabooga worker 2 years ago
runpod_infer.py add runpod oobabooga worker 2 years ago
start.sh add deepspeed 2 years ago
test_input.json add runpod oobabooga worker 2 years ago

README.md

Building

Select one of the following models to build:

  • gpt-neo-1.3B
  • gpt-neo-2.7B
  • gpt-neox-20b
  • pygmalion-6b
  • gpt-j-6b
docker build --build-arg MODEL_NAME={model name} -t repo/image_name:tag .

pip install --upgrade pip
git clone https://github.com/AlpinDale/gptq-gptj.git
cd gptq-gptj
pip install -r requirements.txt
CUDA_VISIBLE_DEVICES=0 python gptj.py PygmalionAI/pygmalion-6b c4 --wbits 4 --groupsize 128 --save pygmalion-6b-8bit-128g.pt
CUDA_VISIBLE_DEVICES=0 python gptj.py PygmalionAI/pygmalion-6b c4 --wbits 4 --groupsize 128 --save_safetensors pygmalion-6b-8bit-128g.safetensors
scp -P 22023 pygmalion-6b-8bit-128g.safetensors will@xd0.de:/home/will/
scp -P 22023 pygmalion-6b-8bit-128g.safetensors will@xd0.de:/home/will/