PatrickStar是一款腾讯开发的分布式深度学习训练工具，它的设计目标是支持以GPT、Bert为代表的超大预训练模型训练。
用法PatrickStar基于PyTorch，这使得迁移pytorch项目变得容易。以下是PatrickStar的示例：
frompatrickstar.rutimeimportiitialize_egiecofig={"optimizer":{"type":"Adam","params":{"lr":0.001,"betas":(0.9,0.999),"eps":1e-6,"weight_decay":0,"use_hybrid_adam":True,},},"fp16":{#lossscalerparams"eabled":True,"loss_scale":0,"iitial_scale_power":2**3,"loss_scale_widow":1000,"hysteresis":2,"mi_loss_scale":1,},"default_chuk_size":64*1024*1024,"release_after_iit":True,"use_cpu_embeddig":False,}defmodel_fuc():#MyModelisaderivedclassfortorch..ModulereturMyModel(...)model,optimizer=iitialize_egie(model_fuc=model_fuc,local_rak=0,cofig=cofig)...fordataidataloader:optimizer.zero_grad()loss=model(data)model.backward(loss)optimizer.step()使用与 DeepSpeed配置JSON 相同的cofig格式，主要包括优化器、损失缩放器和一些PatrickStar特定配置的参数。
引用我们@article{fag2021patrickstar,title={PatrickStar:ParallelTraiigofPre-traiedModelsviaaChuk-basedMemoryMaagemet},author={Fag,JiaruiadYu,YagadZhu,ZiliadLi,ShegguiadYou,YagadZhou,Jie},joural={arXivprepritarXiv:2108.05818},year={2021}}
评论