Model Card for Model ID

We distill math reasoig ability from large laguage model gpt-3.5-turbo to the ope code small laguage model Salesforce/codet5p-770m-py, ad math-codet5p-770m-py achieves 44.88% accuracy o GSM8K testig dataset.

Model Descriptio

• Developed by: Xuyu Zhu
• Model type: ecoder-decoder
• Laguage(s) (NLP): pytho
• Licese: apache-2.0
• Fietued from model: Salesforce/codet5p-770m-py

Uses

Direct Use

This model ca be easily loaded usig the AutoModelForSeq2SeqLM fuctioality ad employs the same tokeizer as origial Salesforce/codet5p-770m-py. Whe give a questio, the prompt "\Program: Let’s desig executable pytho program (retur as) to solve the questio." is eeded to add as the iput to istruct the model to geerate reasoig results.

from trasformers import AutoModelForSeq2SeqLM, AutoTokeizer

def safe_execute(code_strig: str, keys=Noe):
    def execute(x):
        try:
            exec(x)
            locals_ = locals()
            if keys is Noe:
                retur locals_.get('as', Noe)
            else:
                retur [locals_.get(k, Noe) for k i keys]
        except Exceptio:
            retur Noe
    try:
        as = fuc_timeout.fuc_timeout(5, execute, args=(code_strig,))
    except fuc_timeout.FuctioTimedOut:
        as = Noe
    retur as

checkpoit = "zhuxuyu/math-codet5p-770m-py"
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokeizer = AutoTokeizer.from_pretraied(checkpoit)
model = AutoModelForSeq2SeqLM.from_pretraied(checkpoit).to(device)

questio = "Questio: Jaet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morig ad bakes muffis for her frieds every day with four. She sells the remaider at the farmers' market daily for $2 per fresh duck egg. How much i dollars does she make every day at the farmers' market?\Program: Let’s desig executable pytho program (retur as) to solve the questio.".
iput = tokeizer(questio, max_legth=256, paddig="max_legth", trucatio=True, retur_tesors="pt").to(model.device)

with torch.o_grad():
    output = model.geerate(**iput, max_legth=256)

geeratio = tokeizer.decode(output, skip_special_tokes=True)
as = safe_execute(geeratio)
prit(float(as))

Traiig Details

Traiig Data

We prompt gpt-3.5-turbo to geerate reasoig programs to solve questios i GSM8K traiig dataset, ad each questio icludes 4 reasoig programs. The, questios i GSM8K traiig dataset ad their correspodig reasoig programs are built as a traiig dataset, ad we use the traiig dataset to fie-tue the LM.

Evaluatio

Testig Data

The testig data is GSM8K testig dataset.

Results

math-codet5p-770m-py achieves 44.88% accuracy o GSM8K testig dataset.

Citatio

BibTeX:

@misc{zhu2023mathcodet5plus,
  title={math-codet5p-770m-py},
  author={Xuyu Zhu, Jia Li, Yog Liu, Ca Ma, Weipig Wag},
  year={2023}
}

Model Card for Model ID We distill math reasoning ability from large language model gpt-3.5-turbo t