Advacig Ope-source Laguage Models with Mixed-Quality Data

Olie Demo | GitHub | Paper | Discord

Sposored by RuPod

To use this model, we highly recommed istallig the OpeChat package by followig the istallatio guide i our repository ad usig the OpeChat OpeAI-compatible API server by ruig the servig commad from the table below. The server is optimized for high-throughput deploymet usig vLLM ad ca ru o a cosumer GPU with 24GB RAM. To eable tesor parallelism, apped --tesor-parallel-size N to the servig commad.

Oce started, the server listes at localhost:18888 for requests ad is compatible with the OpeAI ChatCompletio API specificatios. Please refer to the example request below for referece. Additioally, you ca use the OpeChat Web UI for a user-friedly experiece.

If you wat to deploy the server as a olie service, you ca use --api-keys sk-KEY1 sk-KEY2 ... to specify allowed API keys ad --disable-log-requests --disable-log-stats --log-file opechat.log for loggig oly to a file. For security purposes, we recommed usig a HTTPS gateway i frot of the server.

curl http://localhost:18888/v1/chat/completios \
  -H "Cotet-Type: applicatio/jso" \
  -d '{
    "model": "opechat_3.6",
    "messages": [{"role": "user", "cotet": "You are a large laguage model amed OpeChat. Write a poem to describe yourself"}]
  }'

Coversatio templates

? Default Mode: Best for codig, chat ad geeral tasks

GPT4 Correct User: Hello<|ed_of_tur|>GPT4 Correct Assistat: Hi<|ed_of_tur|>GPT4 Correct User: How are you today?<|ed_of_tur|>GPT4 Correct Assistat:

⚠️ Notice: Remember to set <|ed_of_tur|> as ed of geeratio toke.

The default template is also available as the itegrated tokeizer.chat_template, which ca be used istead of maually specifyig the template:

messages = [
    {"role": "user", "cotet": "Hello"},
    {"role": "assistat", "cotet": "Hi"},
    {"role": "user", "cotet": "How are you today?"}
]
tokes = tokeizer.apply_chat_template(messages, add_geeratio_prompt=True)

Iferece usig Trasformers

from trasformers import AutoTokeizer, AutoModelForCausalLM
import torch

model_id = "opechat/opechat-3.6-8b-20240522"

tokeizer = AutoTokeizer.from_pretraied(model_id)
model = AutoModelForCausalLM.from_pretraied(model_id, torch_dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "user", "cotet": "Explai how large laguage models work i detail."},
]
iput_ids = tokeizer.apply_chat_template(messages, add_geeratio_prompt=True, retur_tesors="pt").to(model.device)

outputs = model.geerate(iput_ids,
    do_sample=True,
    temperature=0.5,
    max_ew_tokes=1024
)
respose = outputs[0][iput_ids.shape[-1]:]
prit(tokeizer.decode(respose, skip_special_tokes=True))

Foudatio Model Limitatios Despite its advaced capabilities, OpeChat is still boud by the limitatios iheret i its foudatio models. These limitatios may impact the model's performace i areas such as:

• Complex reasoig
• Mathematical ad arithmetic tasks
• Programmig ad codig challeges

Halluciatio of No-existet Iformatio OpeChat may sometimes geerate iformatio that does ot exist or is ot accurate, also kow as "halluciatio". Users should be aware of this possibility ad verify ay critical iformatio obtaied from the model.

Safety OpeChat may sometimes geerate harmful, hate speech, biased resposes, or aswer usafe questios. It's crucial to apply additioal AI safety measures i use cases that require safe ad moderated resposes.

We look forward to hearig from you ad collaboratig o this excitig project!

Project Lead:

• Gua Wag [imoeext at gmail dot com]
• Alpay Ariyak [aariyak at wpi dot edu]

@article{wag2023opechat,
  title={OpeChat: Advacig Ope-source Laguage Models with Mixed-Quality Data},
  author={Wag, Gua ad Cheg, Sijie ad Zha, Xiayua ad Li, Xiagag ad Sog, Se ad Liu, Yag},
  joural={arXiv preprit arXiv:2309.11235},
  year={2023}
}

Advancing Open-source Language Models with Mixed-Quality Data Online Demo |