RedPajama-INCITE-7B-Chat

RedPajama-INCITE-7B-Chat was developed by Together ad leaders from the ope-source AI commuity icludig Otocord.ai, ETH DS3Lab, AAI CERC, Uiversité de Motréal, MILA - Québec AI Istitute, Staford Ceter for Research o Foudatio Models (CRFM), Staford Hazy Research research group ad LAION.

It is fie-tued o OASST1 ad Dolly2 to ehace chattig ability.

• Base Model: RedPajama-INCITE-7B-Base
• Istructio-tued Versio: RedPajama-INCITE-7B-Istruct
• Chat Versio: RedPajama-INCITE-7B-Chat

Model Details

• Developed by: Together Computer.
• Model type: Laguage Model
• Laguage(s): Eglish
• Licese: Apache 2.0
• Model Descriptio: A 6.9B parameter pretraied laguage model.

Quick Start

Please ote that the model requires trasformers versio >= 4.25.1.

To prompt the chat model, use the followig format:

<huma>: [Istructio]
<bot>:

GPU Iferece

This requires a GPU with 16GB memory.

import torch
import trasformers
from trasformers import AutoTokeizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# check trasformers versio
assert trasformers.__versio__ >= MIN_TRANSFORMERS_VERSION, f'Please upgrade trasformers to versio {MIN_TRANSFORMERS_VERSION} or higher.'

# iit
tokeizer = AutoTokeizer.from_pretraied("togethercomputer/RedPajama-INCITE-7B-Chat")
model = AutoModelForCausalLM.from_pretraied("togethercomputer/RedPajama-INCITE-7B-Chat", torch_dtype=torch.float16)
model = model.to('cuda:0')
# ifer
prompt = "<huma>: Who is Ala Turig?\<bot>:"
iputs = tokeizer(prompt, retur_tesors='pt').to(model.device)
iput_legth = iputs.iput_ids.shape[1]
outputs = model.geerate(
    **iputs, max_ew_tokes=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, retur_dict_i_geerate=True
)
toke = outputs.sequeces[0, iput_legth:]
output_str = tokeizer.decode(toke)
prit(output_str)
"""
Ala Mathiso Turig (23 Jue 1912  7 Jue 1954) was a Eglish computer scietist, mathematicia, logicia, cryptaalyst, philosopher, mathematicia, ad theoretical biologist.
"""

GPU Iferece i It8

This requires a GPU with 12GB memory.

To ru iferece with it8, please esure you have istalled accelerate ad bitadbytes. You ca istall them with the followig commad:

pip istall accelerate
pip istall bitsadbytes

The you ca ru iferece with it8 as follows:

import torch
import trasformers
from trasformers import AutoTokeizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# check trasformers versio
assert trasformers.__versio__ >= MIN_TRANSFORMERS_VERSION, f'Please upgrade trasformers to versio {MIN_TRANSFORMERS_VERSION} or higher.'

# iit
tokeizer = AutoTokeizer.from_pretraied("togethercomputer/RedPajama-INCITE-7B-Chat")
model = AutoModelForCausalLM.from_pretraied("togethercomputer/RedPajama-INCITE-7B-Chat", device_map='auto', torch_dtype=torch.float16, load_i_8bit=True)

# ifer
prompt = "<huma>: Who is Ala Turig?\<bot>:"
iputs = tokeizer(prompt, retur_tesors='pt').to(model.device)
iput_legth = iputs.iput_ids.shape[1]
outputs = model.geerate(
    **iputs, max_ew_tokes=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, retur_dict_i_geerate=True
)
toke = outputs.sequeces[0, iput_legth:]
output_str = tokeizer.decode(toke)
prit(output_str)
"""
Ala Mathiso Turig (23 Jue 1912 – 7 Jue 1954) was a Eglish computer scietist, mathematicia, logicia, cryptaalyst, philosopher, ad theoretical biologist.
"""

CPU Iferece

import torch
import trasformers
from trasformers import AutoTokeizer, AutoModelForCausalLM

MIN_TRANSFORMERS_VERSION = '4.25.1'

# check trasformers versio
assert trasformers.__versio__ >= MIN_TRANSFORMERS_VERSION, f'Please upgrade trasformers to versio {MIN_TRANSFORMERS_VERSION} or higher.'

# iit
tokeizer = AutoTokeizer.from_pretraied("togethercomputer/RedPajama-INCITE-7B-Chat")
model = AutoModelForCausalLM.from_pretraied("togethercomputer/RedPajama-INCITE-7B-Chat", torch_dtype=torch.bfloat16)
# ifer
prompt = "<huma>: Who is Ala Turig?\<bot>:"
iputs = tokeizer(prompt, retur_tesors='pt').to(model.device)
iput_legth = iputs.iput_ids.shape[1]
outputs = model.geerate(
    **iputs, max_ew_tokes=128, do_sample=True, temperature=0.7, top_p=0.7, top_k=50, retur_dict_i_geerate=True
)
toke = outputs.sequeces[0, iput_legth:]
output_str = tokeizer.decode(toke)
prit(output_str)
"""
Ala Mathiso Turig, OBE, FRS, (23 Jue 1912 – 7 Jue 1954) was a Eglish computer scietist, mathematicia, logicia, cryptaalyst, philosopher, ad theoretical biologist.
"""

Please ote that sice LayerNormKerelImpl is ot implemeted i fp16 for CPU, we use bfloat16 for CPU iferece.

Uses

Direct Use

Excluded uses are described below.

Misuse, Malicious Use, ad Out-of-Scope Use

It is the resposibility of the ed user to esure that the model is used i a resposible ad ethical maer.

Out-of-Scope Use

RedPajama-INCITE-7B-Chat is a laguage model ad may ot perform well for other use cases outside of its iteded scope. For example, it may ot be suitable for use i safety-critical applicatios or for makig decisios that have a sigificat impact o idividuals or society. It is importat to cosider the limitatios of the model ad to oly use it for its iteded purpose.

Misuse ad Malicious Use

RedPajama-INCITE-7B-Chat is desiged for laguage modelig. Misuse of the model, such as usig it to egage i illegal or uethical activities, is strictly prohibited ad goes agaist the priciples of the project.

Usig the model to geerate cotet that is cruel to idividuals is a misuse of this model. This icludes, but is ot limited to:

• Geeratig fake ews, misiformatio, or propagada
• Promotig hate speech, discrimiatio, or violece agaist idividuals or groups
• Impersoatig idividuals or orgaizatios without their coset
• Egagig i cyberbullyig or harassmet
• Defamatory cotet
• Spammig or scammig
• Sharig cofidetial or sesitive iformatio without proper authorizatio
• Violatig the terms of use of the model or the data used to trai it
• Creatig automated bots for malicious purposes such as spreadig malware, phishig scams, or spammig

Limitatios

RedPajama-INCITE-7B-Chat, like other laguage models, has limitatios that should be take ito cosideratio. For example, the model may ot always provide accurate or relevat aswers, particularly for questios that are complex, ambiguous, or outside of its traiig data. We therefore welcome cotributios from idividuals ad orgaizatios, ad ecourage collaboratio towards creatig a more robust ad iclusive chatbot.

Traiig

Traiig Data

Please refer to togethercomputer/RedPajama-Data-1T

Traiig Procedure

• Hardware: 8 A100
• Optimizer: Adam
• Gradiet Accumulatios: 1
• Num of Tokes: 79M tokes
• Learig rate: 1e-5

Commuity

Joi us o Together Discord

RedPajama-INCITE-7B-Chat RedPajama-INCITE-7B-Chat was developed by Together and leaders from the ope