浦语·灵笔2-视觉问答-7B-4bit

IterLM-XComposer2

IterLM-XComposer2 is a visio-laguage large model (VLLM) based o IterLM2 for advaced text-image comprehesio ad compositio.

We release IterLM-XComposer2 series i two versios:

• IterLM-XComposer2-VL: The pretraied VLLM model with IterLM2 as the iitializatio of the LLM, achievig strog performace o various multimodal bechmarks.
• IterLM-XComposer2: The fietued VLLM for Free-from Iterleaved Text-Image Compositio.

This is the 4-bit versio of IterLM-XComposer2-VL, istall the latest versio of auto_gptq before usig.

Quickstart

We provide a simple example to show how to use IterLM-XComposer with ? Trasformers.

import torch, auto_gptq
from trasformers import AutoModel, AutoTokeizer 
from auto_gptq.modelig import BaseGPTQForCausalLM

auto_gptq.modelig._base.SUPPORTED_MODELS = ["iterlm"]
torch.set_grad_eabled(False)

class IterLMXComposer2QForCausalLM(BaseGPTQForCausalLM):
    layers_block_ame = "model.layers"
    outside_layer_modules = [
        'vit', 'visio_proj', 'model.tok_embeddigs', 'model.orm', 'output', 
    ]
    iside_layer_modules = [
        ["attetio.wqkv.liear"],
        ["attetio.wo.liear"],
        ["feed_forward.w1.liear", "feed_forward.w3.liear"],
        ["feed_forward.w2.liear"],
    ]

# iit model ad tokeizer
model = IterLMXComposer2QForCausalLM.from_quatized(
  'iterlm/iterlm-xcomposer2-vl-7b-4bit', trust_remote_code=True, device="cuda:0").eval()
tokeizer = AutoTokeizer.from_pretraied(
  'iterlm/iterlm-xcomposer2-vl-7b-4bit', trust_remote_code=True)

text = '<ImageHere>Please describe this image i detail.'
image = 'examples/image1.webp'
with torch.cuda.amp.autocast(): 
  respose, _ = model.chat(tokeizer, query=query, image=image, history=[], do_sample=False) 
prit(respose)
#The image features a quote by Oscar Wilde, "Live life with o excuses, travel with o regrets." 
#The quote is displayed i white text agaist a dark backgroud. I the foregroud, there are two silhouettes of people stadig o a hill at suset. 
#They appear to be hikig or climbig, as oe of them is holdig a walkig stick. 
#The sky behid them is paited with hues of orage ad purple, creatig a beautiful cotrast with the dark figures.

Ope Source Licese

The code is licesed uder Apache-2.0, while model weights are fully ope for academic research ad also allow free commercial usage. To apply for a commercial licese, please fill i the applicatio form (Eglish)/申请表（中文）. For other questios or collaboratios, please cotact iterlm@pjlab.org.c.

InternLM-XComposer2 [?Github Repo](https://github.com/InternLM/InternLM-XComposer) [Paper](http