Radeg-T5-77M-MultiTask-KF-Chiese 燃灯-T5-77M-多任务-中文-KF

请注意：“model.safetesors”模型文件比较大，已放入脸书，请自行下载。

huggigface: https://huggigface.co/zhaoxiaopag111/Radeg-T5-77M-MultiTask-KF-Chiese/

使用方法：git本模型+huggigface下载“model.safetesors”模型

简介 Brief Itroductio

在Fegshebag/Radeg-T5-77M-MultiTask-Chiese的基础上，使用了5k客服语料“对话摘要”、“用户意图和坐席回答”进行加强微调模型，进行有监督任务预训练。

Based o the Fegshebag/Radeg-T5-784M-MultiTask-Chiese model, a supervised pre-traiig task was performed usig 5,000 customer service corpora cosistig of "dialogue summaries," "user itetios," ad "aget resposes."

模型分类 Model Taxoomy

需求 Demad 任务 Task 系列 Series 模型 Model 参数 Parameter 额外 Extra 通用 Geeral 自然语言转换 NLT 燃灯 Radeg MultiTask 77M 多任务-中文 MultiTask-Chiese

模型信息 Model Iformatio

参考论文：Explorig the Limits of Trasfer Learig with a Uified Text-to-Text Trasformer

基于Radeg-T5-77M，我们在收集的100+个中文领域的多任务数据集（从中采样了30w+个样本）上微调了它，得到了此多任务版本。这些多任务包括：情感分析，新闻分类，文本分类，意图识别，自然语言推理，多项选择，指代消解，抽取式阅读理解，实体识别，关键词抽取，生成式摘要。

Based o Radeg-T5-77M, we fie-tued it o a collectio of 100+ multitaskig datasets i Chiese domais (from which 30w+ samples were sampled) to obtai this multitaskig versio. These multitasks iclude: setimet aalysis, ews classificatio, text classificatio, itetio recogitio, atural laguage iferece, multiple choice, deotatioal disambiguatio, extractive readig comprehesio, etity recogitio, keyword extractio, ad geerative summarizatio.

使用 Usage

'''

# load tokeizer ad model
import torch
from trasformers import T5Tokeizer, T5Cofig, T5ForCoditioalGeeratio
pretraied_model = "zhaoxiaopag111/Radeg-T5-77M-MultiTask-KF-Chiese"

special_tokes = ["<extra_id_{}>".format(i) for i i rage(1024)]
tokeizer = T5Tokeizer.from_pretraied(
    pretraied_model,
    do_lower_case=True,
    max_legth=1024,
    trucatio=True,
    additioal_special_tokes=special_tokes,
)
cofig = T5Cofig.from_pretraied(pretraied_model)
model = T5ForCoditioalGeeratio.from_pretraied(pretraied_model, cofig=cofig)
model.resize_toke_embeddigs(le(tokeizer))
model.eval()
def modelPredictMai_ZhaiYao(text):
    text = "摘要下面对话任务：【{0}】这段文本对话的摘要是什么？".format(text)
    ecode_dict = tokeizer(text, max_legth=1024, paddig='max_legth', trucatio=True)

    iputs = {
        "iput_ids": torch.tesor([ecode_dict['iput_ids']]).log(),
        "attetio_mask": torch.tesor([ecode_dict['attetio_mask']]).log(),
    }

    # geerate aswer
    outputs = model.geerate(
        iput_ids=iputs['iput_ids'],
        max_legth=1024,
        do_sample=True
        # early_stoppig=True,
    )
    # prit(logits)
    logits = outputs[:, 1:]
    # prit(logits)
    predict_label = [tokeizer.decode(i, skip_special_tokes=True) for i i logits][0]
    retur predict_label
    # prit(predict_label)
def modelPredictMai_YiYu(text):
    text = "用户意图和客服回答任务：【{0}】这段文本对话的用户意图和客服回答是什么？".format(text)
    ecode_dict = tokeizer(text, max_legth=1024, paddig='max_legth', trucatio=True)

    iputs = {
        "iput_ids": torch.tesor([ecode_dict['iput_ids']]).log(),
        "attetio_mask": torch.tesor([ecode_dict['attetio_mask']]).log(),
    }

    # geerate aswer
    outputs = model.geerate(
        iput_ids=iputs['iput_ids'],
        max_legth=1024,
        do_sample=True
        # early_stoppig=True,
    )
    # prit(logits)
    logits = outputs[:, 1:]
    # prit(logits)
    predict_label = [tokeizer.decode(i, skip_special_tokes=True) for i i logits][0]
    retur predict_label

if __ame__ == "__mai__":
    prit("Procedures begi to execute！")
    # tokeize
    iput_text = ""
    # promopt:自动摘要  用户意图和客服回答
    result_text = modelPredictMai_YiYu(text=iput_text)
    # result_text = modelPredictMai_ZhaiYao(text=iput_text)
    prit(result_text)

'''

checkpoit-20240327

This model is a fie-tued versio of D:\chatGPTBigModels\modelLoraMai\models\Radeg-T5-77M-MultiTask-Chiese o a ukow dataset. It achieves the followig results o the evaluatio set:

• Loss: 0.9802
• Rouge1: 35.5908
• Rouge2: 16.2325
• Rougel: 35.4323
• Rougelsum: 35.4673
• Ge Le: 48.1477

Traiig hyperparameters

The followig hyperparameters were used durig traiig:

• learig_rate: 5e-05
• traibatchsize: 2
• evalbatchsize: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) ad epsilo=1e-08
• lrschedulertype: liear
• um_epochs: 8.0

Traiig results

Framework versios

• Trasformers 4.39.0
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokeizers 0.15.2

Randeng-T5-77M-MultiTask-KF-Chinese 燃灯-T5-77M-多任务-中文-KF 请注意：“model.safetensors”模型文件比较大，已放入脸书，请自行下载。