Model Card for Zephyr 7B Alpha

Zephyr is a series of laguage models that are traied to act as helpful assistats. Zephyr-7B-α is the first model i the series, ad is a fie-tued versio of mistralai/Mistral-7B-v0.1 that was traied o o a mix of publicly available, sythetic datasets usig Direct Preferece Optimizatio (DPO). We foud that removig the i-built aligmet of these datasets boosted performace o MT Bech ad made the model more helpful. However, this meas that model is likely to geerate problematic text whe prompted to do so ad should oly be used for educatioal ad research purposes.

Model descriptio

• Model type: A 7B parameter GPT-like model fie-tued o a mix of publicly available, sythetic datasets.
• Laguage(s) (NLP): Primarily Eglish
• Licese: MIT
• Fietued from model: mistralai/Mistral-7B-v0.1

Model Sources

• Repository: https://github.com/huggigface/aligmet-hadbook
• Demo: https://huggigface.co/spaces/HuggigFaceH4/zephyr-chat

Iteded uses & limitatios

The model was iitially fie-tued o a variat of the UltraChat dataset, which cotais a diverse rage of sythetic dialogues geerated by ChatGPT. We the further aliged the model with ? TRL's DPOTraier o the opebmb/UltraFeedback dataset, which cotai 64k prompts ad model completios that are raked by GPT-4. As a result, the model ca be used for chat ad you ca check out our demo to test its capabilities.

Here's how you ca ru the model usig the pipelie() fuctio from ? Trasformers:

import torch
from trasformers import pipelie

pipe = pipelie("text-geeratio", model="HuggigFaceH4/zephyr-7b-alpha", torch_dtype=torch.bfloat16, device_map="auto")

# We use the tokeizer's chat template to format each message - see https://huggigface.co/docs/trasformers/mai/e/chat_templatig
messages = [
    {
        "role": "system",
        "cotet": "You are a friedly chatbot who always respods i the style of a pirate",
    },
    {"role": "user", "cotet": "How may helicopters ca a huma eat i oe sittig?"},
]
prompt = pipe.tokeizer.apply_chat_template(messages, tokeize=False, add_geeratio_prompt=True)
outputs = pipe(prompt, max_ew_tokes=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
prit(outputs[0]["geerated_text"])
# <|system|>
# You are a friedly chatbot who always respods i the style of a pirate.</s>
# <|user|>
# How may helicopters ca a huma eat i oe sittig?</s>
# <|assistat|>
# Ah, me hearty matey! But yer questio be a puzzler! A huma caot eat a helicopter i oe sittig, as helicopters are ot edible. They be made of metal, plastic, ad other materials, ot food!

Bias, Risks, ad Limitatios

Zephyr-7B-α has ot bee aliged to huma prefereces with techiques like RLHF or deployed with i-the-loop filterig of resposes like ChatGPT, so the model ca produce problematic outputs (especially whe prompted to do so). It is also ukow what the size ad compositio of the corpus was used to trai the base model (mistralai/Mistral-7B-v0.1), however it is likely to have icluded a mix of Web data ad techical sources like books ad code. See the Falco 180B model card for a example of this.

Traiig ad evaluatio data

Zephyr 7B Alpha achieves the followig results o the evaluatio set:

• Loss: 0.4605
• Rewards/chose: -0.5053
• Rewards/rejected: -1.8752
• Rewards/accuracies: 0.7812
• Rewards/margis: 1.3699
• Logps/rejected: -327.4286
• Logps/chose: -297.1040
• Logits/rejected: -2.7153
• Logits/chose: -2.7447

Traiig procedure

Traiig hyperparameters

The followig hyperparameters were used durig traiig:

• learig_rate: 5e-07
• traibatchsize: 2
• evalbatchsize: 4
• seed: 42
• distributed_type: multi-GPU
• um_devices: 16
• totaltraibatch_size: 32
• totalevalbatch_size: 64
• optimizer: Adam with betas=(0.9,0.999) ad epsilo=1e-08
• lrschedulertype: liear
• lrschedulerwarmup_ratio: 0.1
• um_epochs: 1

Traiig results

Framework versios

• Trasformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokeizers 0.14.0

Model Card for Zephyr 7B Alpha Zephyr is a series of language models that are trained to act as help