这是一个本地托管版本的GitHubCopilot。它在英伟达的Trito推理服务器中使用了SalesForceCodeGe模型和FasterTrasformer后端。
前提条件Dockerdocker-compose>=1.28一台计算能力大于6.0的英伟达GPU，以及足够的VRAM来运行你想要的模型vidia-dockercurl和zstd，用于下载和解包模型Copilot插件你可以配置官方VSCodeCopilot插件来使用你的本地服务器。只要编辑你的settigs.jso来添加。
"github.copilot.advaced":{"debug.overrideEgie":"codege","debug.testOverrideProxyUrl":"https://localhost:5000","debug.overrideProxyUrl":"https://localhost:5000"}设置运行设置脚本以选择要使用的模型。这将从Huggigface下载模型，然后将其转换为与FasterTrasformer一起使用。
$./setup.shModelsavailable:[1]codege-350M-moo(2GBtotalVRAMrequired;Pytho-oly)[2]codege-350M-multi(2GBtotalVRAMrequired;multi-laguage)[3]codege-2B-moo(7GBtotalVRAMrequired;Pytho-oly)[4]codege-2B-multi(7GBtotalVRAMrequired;multi-laguage)[5]codege-6B-moo(13GBtotalVRAMrequired;Pytho-oly)[6]codege-6B-multi(13GBtotalVRAMrequired;multi-laguage)[7]codege-16B-moo(32GBtotalVRAMrequired;Pytho-oly)[8]codege-16B-multi(32GBtotalVRAMrequired;multi-laguage)Eteryourchoice[6]:2EterumberofGPUs[1]:1Wheredoyouwattosavethemodel[/home/moyix/git/fauxpilot/models]?/fastdata/mymodelsDowloadigadcovertigthemodel,thiswilltakeawhile...Covertigmodelcodege-350M-multiwith1GPUsLoadigCodeGemodelDowloadigcofig.jso:100%|██████████|996/996[00:00<00:00,1.25MB/s]Dowloadigpytorch_model.bi:100%|██████████|760M/760M[00:11<00:00,68.3MB/s]CreatigemptyGPTJmodelCovertig...Coversiocomplete.Savigmodeltocodege-350M-multi-hf...===============Argumet===============saved_dir:/models/codege-350M-multi-1gpu/fastertrasformer/1i_file:codege-350M-multi-hftraied_gpu_um:1ifer_gpu_um:1processes:4weight_data_type:fp32========================================trasformer.wte.weighttrasformer.h.0.l_1.weight[...morecoversiooutputtrimmed...]trasformer.l_f.weighttrasformer.l_f.biaslm_head.weightlm_head.biasDoe!Nowru./lauch.shtostarttheFauxPilotserver.
评论