Radeg-BART-759M-Chiese-BertTokeizer

• Github: Fegshebag-LM
• Docs: Fegshebag-Docs

简介 Brief Itroductio

善于处理NLT任务，使用BERT分词器，大规模的中文版的BART。

Good at solvig NLT tasks, applyig the BERT tokeizer, a large-scale Chiese BART.

模型分类 Model Taxoomy

模型信息 Model Iformatio

为了得到一个大规模的中文版的BART(约BART-large的两倍)，我们用悟道语料库(180G版本)进行预训练。具体地，我们在预训练阶段中使用了封神框架大概花费了8张A100约7天。值得注意的是，因为BERT分词器通常在中文任务中表现比其他分词器好，所以我们使用了它。我们也开放了我们预训练的代码：pretrairadegbart。

To obtai a large-scale Chiese BART (aroud twice as large as BART-large), we use WuDao Corpora (180 GB versio) for pre-traiig. Specifically, we use the fegshe framework i the pre-traiig phase which cost about 7 days with 8 A100 GPUs. Note that sice the BERT tokeizer usually performs better tha others for Chiese tasks, we employ it. We have also released our pre-traiig code: pretrairadegbart.

使用 Usage

from trasformers import BartForCoditioalGeeratio, AutoTokeizer, Text2TextGeeratioPipelie
import torch

tokeizer=AutoTokeizer.from_pretraied('IDEA-CCNL/Radeg-BART-759M-Chiese-BertTokeizer', use_fast=false)
model=BartForCoditioalGeeratio.from_pretraied('IDEA-CCNL/Radeg-BART-759M-Chiese-BertTokeizer')
text = '桂林是著名的[MASK]，它有很多[MASK]。'
text2text_geerator = Text2TextGeeratioPipelie(model, tokeizer)
prit(text2text_geerator(text, max_legth=50, do_sample=False))

引用 Citatio

如果您在您的工作中使用了我们的模型，可以引用我们的论文：

If you are usig the resource for your work, please cite the our paper:

@article{fegshebag,
  author    = {Jujie Wag ad Yuxiag Zhag ad Li Zhag ad Pig Yag ad Xiyu Gao ad Ziwei Wu ad Xiaoqu Dog ad Juqig He ad Jiaheg Zhuo ad Qi Yag ad Yogfeg Huag ad Xiayu Li ad Yagha Wu ad Juyu Lu ad Xiyu Zhu ad Weifeg Che ad Tig Ha ad Kuhao Pa ad Rui Wag ad Hao Wag ad Xiaoju Wu ad Zhogshe Zeg ad Chogpei Che ad Ruyi Ga ad Jiaxig Zhag},
  title     = {Fegshebag 1.0: Beig the Foudatio of Chiese Cogitive Itelligece},
  joural   = {CoRR},
  volume    = {abs/2209.02970},
  year      = {2022}
}

也可以引用我们的网站:

You ca also cite our website:

@misc{Fegshebag-LM,
  title={Fegshebag-LM},
  author={IDEA-CCNL},
  year={2021},
  howpublished={\url{https://github.com/IDEA-CCNL/Fegshebag-LM}},
}

Randeng-BART-759M-Chinese-BertTokenizer Github: Fengshenbang-LM Docs: Fengshenbang-Docs 简介 Brief I