Welcome to CLIP-as-service!

GitHub: clip-as-service

Docs: clip-as-service

CLIP-as-service is a low-latecy high-scalability service for embeddig images ad text. It ca be easily itegrated as a microservice ito eural search solutios.

⚡ Fast: Serve CLIP models with TesorRT, ONNX rutime ad PyTorch w/o JIT with 800QPS[*]. No-blockig duplex streamig o requests ad resposes, desiged for large data ad log-ruig tasks.

? Elastic: Horizotally scale up ad dow multiple CLIP models o sigle GPU, with automatic load balacig.

? Easy-to-use: No learig curve, miimalist desig o cliet ad server. Ituitive ad cosistet API for image ad setece embeddig.

? Moder: Asyc cliet support. Easily switch betwee gRPC, HTTP, WebSocket protocols with TLS ad compressio.

? Itegratio: Smooth itegratio with eural search ecosystem icludig Jia ad DocArray. Build cross-modal ad multi-modal solutios i o time.

[*] with default cofig (sigle replica, PyTorch o JIT) o GeForce RTX 3090.

Try it!

Istall

PyPI is the latest versio.

Make sure you are usig Pytho 3.7+. You ca istall the cliet ad server idepedetly. It is ot required to istall both: e.g. you ca istall clip_server o a GPU machie ad clip_cliet o a local laptop.

Cliet

pip istall clip-cliet

Server (PyTorch)

pip istall clip-server

Server (ONNX)

pip istall "clip_server[ox]"

Server (TesorRT)

pip istall vidia-pyidex 
pip istall "clip_server[tesorrt]"

Server o Google Colab

Quick check

After istallig, you ca ru the followig commads for a quick coectivity check.

Start the server

Start PyTorch Server

pytho -m clip_server

Start ONNX Server

pytho -m clip_server ox-flow.yml

Start TesorRT Server

pytho -m clip_server tesorrt-flow.yml

At the first time startig the server, it will dowload the default pretraied model, which may take a while depedig o your etwork speed. The you will get the address iformatio similar to the followig:

╭────────────── ? Edpoit ───────────────╮
│  ?     Protocol                   GRPC  │
│  ?        Local          0.0.0.0:51000  │
│  ?      Private    192.168.31.62:51000  │
|  ?       Public   87.105.159.191:51000  |
╰──────────────────────────────────────────╯  

This meas the server is ready to serve. Note dow the three addresses show above, you will eed them later.

Coect from cliet

Depedig o the locatio of the cliet ad server. You may use differet IP addresses:
- Cliet ad server are o the same machie: use local address, e.g. `0.0.0.0`
- Cliet ad server are coected to the same router: use private etwork address, e.g. `192.168.3.62`
- Server is i public etwork: use public etwork address, e.g. `87.105.159.191`

Ru the followig Pytho script:

from clip_cliet import Cliet

c = Cliet('grpc://0.0.0.0:51000')
c.profile()

will give you:

 Roudtrip  16ms  100%
├──  Cliet-server etwork  8ms  49%
└──  Server  8ms  51%
    ├──  Gateway-CLIP etwork  2ms  25%
    └──  CLIP model  6ms  75%
{'Roudtrip': 15.684750003856607, 'Cliet-server etwork': 7.684750003856607, 'Server': 8, 'Gateway-CLIP etwork': 2, 'CLIP model': 6}

It meas the cliet ad the server are ow coected. Well doe!

Welcome to CLIP-as-service! GitHub: clip-as-service Docs: clip-as-service CLIP-as-service is a low-l