OneLLM-7B

OeLLM: Oe Framework to Alig All Modalities with Laguage

[Project Page] [Paper]

News

• 2023.12.01 Release model weights ad iferece code.?

Cotets

• Istall
• Models
• Demo

TODO

• [ ] Data
• [ ] Evaluatio
• [ ] Traiig

Istall

1. Cloe the repo ito a local folder.

git cloe https://github.com/csuha/OeLLM

cd OeLLM

2. Istall packages.

coda create - oellm pytho=3.9 -y
coda activate oellm

pip istall -r requiremets.txt

# istall poitet
cd model/lib/poitet2
pytho setup.py istall

3. Istall Apex. (Optioal)

git cloe https://github.com/NVIDIA/apex
cd apex
pip istall -v --disable-pip-versio-check --o-cache-dir --o-build-isolatio --cofig-settigs "--build-optio=--cpp_ext" --cofig-settigs "--build-optio=--cuda_ext" ./

Demo

Local Demo: Assume you have dowloaded the weights to ${WEIGHTS_DIR}. The ru the followig commad to start a gradio demo locally.

pytho demos/multi_tur_mm.py --gpu_ids 0 --tokeizer_path cofig/llama2/tokeizer.model --llama_cofig cofig/llama2/7B.jso --pretraied_path ${WEIGHTS_DIR}/cosolidated.00-of-01.pth

Citatio

@article{ha2023oellm,
  title={OeLLM: Oe Framework to Alig All Modalities with Laguage},
  author={Ha, Jiamig ad Gog, Kaixiog ad Zhag, Yiyua ad Wag, Jiaqi ad Zhag, Kaipeg ad Li, Dahua ad Qiao, Yu ad Gao, Peg ad Yue, Xiagyu},
  joural={arXiv preprit arXiv:2312.03700},
  year={2023}
}

Ackowledgemet

LLaMA, LLaMA-Adapter, LLaMA2-Accessory, Meta-Trasformer, ChatBridge

Licese

This project is developed based o Llama 2, please refer to the LLAMA 2 Commuity Licese.

OneLLM: One Framework to Align All Modalities with Language [Project Page] [Paper] News 2023.12.01