hendrik
/
matrix-pygmalion-bot
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Chatbot
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
47 Commits
1 Branch
0 Tags
1.0 MiB
 
 
 
 
Tree: 698d59bd34
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '698d59bd34'
${ noResults }
matrix-pygmalion-bot/runpod/runpod-worker-transformers
Hendrik Langer 698d59bd34
rewrite remote transformers container 		2 years ago
..
Dockerfile rewrite remote transformers container 2 years ago
README.md add experimental and wip worker containers 2 years ago
model_fetcher.py rewrite remote transformers container 2 years ago
runpod_infer.py rewrite remote transformers container 2 years ago
test_input.json work on remote workers 2 years ago

README.md

Building

Select one of the following models to build:

  • gpt-neo-1.3B
  • gpt-neo-2.7B
  • gpt-neox-20b
  • pygmalion-6b
  • gpt-j-6b
docker build --build-arg MODEL_NAME={model name} -t repo/image_name:tag .

pip install --upgrade pip
git clone https://github.com/AlpinDale/gptq-gptj.git
cd gptq-gptj
pip install -r requirements.txt
CUDA_VISIBLE_DEVICES=0 python gptj.py PygmalionAI/pygmalion-6b c4 --wbits 4 --groupsize 128 --save pygmalion-6b-8bit-128g.pt
CUDA_VISIBLE_DEVICES=0 python gptj.py PygmalionAI/pygmalion-6b c4 --wbits 4 --groupsize 128 --save_safetensors pygmalion-6b-8bit-128g.safetensors
scp -P 22023 pygmalion-6b-8bit-128g.safetensors will@xd0.de:/home/will/
scp -P 22023 pygmalion-6b-8bit-128g.safetensors will@xd0.de:/home/will/