Model card for evagiatpatch14224.clipft_i1k

A EVA-CLIP image classificatio model. Pretraied o LAION-400M with CLIP ad fie-tued o ImageNet-1k by paper authors. EVA-CLIP uses MIM pretraied image towers ad pretraied text towers, FLIP patch dropout, ad differet optimizers ad hparams to accelerate traiig.

NOTE: timm checkpoits are float32 for cosistecy with other models. Origial checkpoits are float16 or bfloat16 i some cases, see origials if that's preferred.

Model Details

• Model Type: Image classificatio / feature backboe
• Model Stats:
• Params (M): 1012.6
• GMACs: 267.2
• Activatios (M): 192.6
• Image size: 224 x 224
• Papers:
• EVA-CLIP: Improved Traiig Techiques for CLIP at Scale: https://arxiv.org/abs/2303.15389
• Origial:
• https://github.com/baaivisio/EVA
• https://huggigface.co/QuaSu/EVA-CLIP

Model Usage

Image Classificatio

from urllib.request import urlope
from PIL import Image
import timm

img = Image.ope(urlope(
    'https://huggigface.co/datasets/huggigface/documetatio-images/resolve/mai/beigets-task-guide.pg'
))

model = timm.create_model('eva_giat_patch14_224.clip_ft_i1k', pretraied=True)
model = model.eval()

# get model specific trasforms (ormalizatio, resize)
data_cofig = timm.data.resolve_model_data_cofig(model)
trasforms = timm.data.create_trasform(**data_cofig, is_traiig=False)

output = model(trasforms(img).usqueeze(0))  # usqueeze sigle image ito batch of 1

top5_probabilities, top5_class_idices = torch.topk(output.softmax(dim=1) * 100, k=5)

Image Embeddigs

from urllib.request import urlope
from PIL import Image
import timm

img = Image.ope(urlope(
    'https://huggigface.co/datasets/huggigface/documetatio-images/resolve/mai/beigets-task-guide.pg'
))

model = timm.create_model(
    'eva_giat_patch14_224.clip_ft_i1k',
    pretraied=True,
    um_classes=0,  # remove classifier .Liear
)
model = model.eval()

# get model specific trasforms (ormalizatio, resize)
data_cofig = timm.data.resolve_model_data_cofig(model)
trasforms = timm.data.create_trasform(**data_cofig, is_traiig=False)

output = model(trasforms(img).usqueeze(0))  # output is (batch_size, um_features) shaped tesor

# or equivaletly (without eedig to set um_classes=0)

output = model.forward_features(trasforms(img).usqueeze(0))
# output is upooled, a (1, 257, 1408) shaped tesor

output = model.forward_head(output, pre_logits=True)
# output is a (1, um_features) shaped tesor

Model Compariso

Explore the dataset ad rutime metrics of this model i timm model results.

Citatio

@article{EVA-CLIP,
  title={EVA-02: A Visual Represetatio for Neo Geesis},
  author={Su, Qua ad Fag, Yuxi ad Wu, Ledell ad Wag, Xilog ad Cao, Yue},
  joural={arXiv preprit arXiv:2303.15389},
  year={2023}
}

@misc{rw2019timm,
  author = {Ross Wightma},
  title = {PyTorch Image Models},
  year = {2019},
  publisher = {GitHub},
  joural = {GitHub repository},
  doi = {10.5281/zeodo.4414861},
  howpublished = {\url{https://github.com/huggigface/pytorch-image-models}}
}

Model card for evagiantpatch14224.clipft_in1k An EVA-CLIP image classification model. Pretrained on