This repository hosts the optimized versios of Phi-3-medium-4k-istruct to accelerate iferece with ONNX Rutime for your CPU. Phi-3 Medium is a 14B parameter, lightweight, state-of-the-art ope model traied with the Phi-3 datasets, which iclude both sythetic data ad the filtered publicly available websites data, with a focus o high-quality ad reasoig dese properties. The model belogs to the Phi-3 family with the medium versio i two variats: 4K ad 128K, which are the cotext legths (i tokes) that they ca support. The base model has udergoe a post-traiig process that icorporates both supervised fie-tuig ad direct preferece optimizatio for the istructio followig ad safety measures. Whe assessed agaist bechmarks testig commo sese, laguage uderstadig, math, code, log cotext, ad logical reasoig, Phi-3-Medium-4K-Istruct showcased a robust ad state-of-the-art performace amog models of the same-size ad ext-size-up. Optimized variats of the Phi-3 Medium models are published here i ONNX format ad ru with ONNX Rutime o CPU ad GPU across devices, icludig server platforms, Widows, ad Liux, with the precisio best suited to each of these targets. Here are some of the optimized cofiguratios we have added: How do you kow which is the best ONNX model for you: To support the Phi-3 models across a rage of devices, platforms, ad EP backeds, we itroduce a ew API to wrap several aspects of geerative AI iferecig. This API makes it easy to drag ad drop LLMs straight ito your app. To ru the early versio of these models with ONNX, follow the steps here. You ca also test this with a chat app. The models are tested o: Miimum Cofiguratio Required: The model rus at ~20 tokes/sec o a Itel(R) Core(TM) i9-10920X CPU @ 3.50GHz. pariitarahi, kvaishavi, atke Kual Vaishavi, Sughoo Choi, Yufeg Li, Akshay Soawae, Sheetal Aru Kadam, Rui Re, Edward Che, Scott McKay, Emma Nig, Natalie Kershaw, Pariita RahiPhi-3 Medium-4k-Istruct ONNX CPU models
ONNX Models
How to Get Started with the Model
Hardware Supported
Model Descriptio
Additioal Details
Performace Metrics
Appedix
Model Card Cotact
Cotributors
点击空白处退出提示







评论