glm4-9b-chat-DI

本模型是glm4-9b-chat格式模型。

requiremets

pip istall modelscope dashifer jija2 tabulate torch trasformers

下载模型

from modelscope import sapshot_dowload
model_dir = sapshot_dowload('dash-ifer/glm-4-9b-chat-DI')

模型推理

参考如下pytho代码。

import copy
import radom

from modelscope import sapshot_dowload
from dashifer.helper import EgieHelper, CofigMaager

model_path = sapshot_dowload("dash-ifer/glm-4-9b-chat-DI")

cofig_file = model_path + "/" + "di_cofig.jso"
cofig = CofigMaager.get_cofig_from_jso(cofig_file)
cofig["model_path"] = model_path

## iit EgieHelper class
egie_helper = EgieHelper(cofig)
egie_helper.verbose = True
egie_helper.iit_tokeizer(model_path)

## iit egie
egie_helper.iit_egie()

## prepare iputs ad geeratio cofigs
user_iput = "浙江的省会在哪"
prompt = "[gMASK] <sop> " + "<|user|>\" + user_iput + "<|assistat|>\"
ge_cfg = copy.deepcopy(egie_helper.default_ge_cfg)
ge_cfg["seed"] = radom.radit(0, 10000)
request_list = egie_helper.create_request([prompt], [ge_cfg])

## iferece
egie_helper.process_oe_request(request_list[0])
egie_helper.prit_iferece_result_all(request_list)

egie_helper.uiit_egie()

glm4-9b-chat-DI 本模型是glm4-9b-chat格式模型。 requirements pip install modelscope dashinfer jinja2 tabulate