本文主要是介绍CogVLM 本地部署体验（问题解决），希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
硬件要求（模型推理）：
INT4 ： RTX30901，显存24GB，内存32GB，系统盘200GB
INT4 ： RTX40901或RTX3090*2，显存24GB，内存32GB，系统盘200GB
模型微调硬件要求更高。一般不建议个人用户环境使用
如果要运行官方web界面streamlit run composite_demo/main.py显存需要40G以上，至少需两张RTX3090显卡。否则基本无法体验
环境准备
模型准备
手动下载以下几个模型（体验时几个模型不一定需全下载）
下载地址：https://hf-mirror.com/THUDM
lmsys/vicuna-7b-v1.5
THUDM/cogagent-chat-hf
THUDM/cogvlm-chat-hf
THUDM/cogvlm-grounding-generalist-hf
下载模型源码
git clone https://github.com/THUDM/CogVLM.git;
cd CogVLM
创建conda环境
conda create -n cogvlm python=3.11 -y
source activate cogvlm
修改本国内源
pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
pip config set install.trusted-host mirrors.aliyun.com
安装依赖库
安装torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
安装 cuda-runtime
(cogvlm) develop@itserver03:/u01/workspace/cogvlm/CogVLM$: conda install -y -c "nvidia/label/cuda-12.1.0" cuda-runtimeThe following NEW packages will be INSTALLED:cuda-cudart nvidia/label/cuda-12.1.0/linux-64::cuda-cudart-12.1.55-0 cuda-libraries nvidia/label/cuda-12.1.0/linux-64::cuda-libraries-12.1.0-0 cuda-nvrtc nvidia/label/cuda-12.1.0/linux-64::cuda-nvrtc-12.1.55-0 cuda-opencl nvidia/label/cuda-12.1.0/linux-64::cuda-opencl-12.1.56-0 cuda-runtime nvidia/label/cuda-12.1.0/linux-64::cuda-runtime-12.1.0-0 libcublas nvidia/label/cuda-12.1.0/linux-64::libcublas-12.1.0.26-0 libcufft nvidia/label/cuda-12.1.0/linux-64::libcufft-11.0.2.4-0 libcufile nvidia/label/cuda-12.1.0/linux-64::libcufile-1.6.0.25-0 libcurand nvidia/label/cuda-12.1.0/linux-64::libcurand-10.3.2.56-0 libcusolver nvidia/label/cuda-12.1.0/linux-64::libcusolver-11.4.4.55-0 libcusparse nvidia/label/cuda-12.1.0/linux-64::libcusparse-12.0.2.55-0 libnpp nvidia/label/cuda-12.1.0/linux-64::libnpp-12.0.2.50-0 libnvjitlink nvidia/label/cuda-12.1.0/linux-64::libnvjitlink-12.1.55-0 libnvjpeg nvidia/label/cuda-12.1.0/linux-64::libnvjpeg-12.1.0.39-0
Downloading and Extracting Packages:
libcublas-12.1.0.26 | 329.0 MB | | 0%
libcusparse-12.0.2.5 | 163.0 MB | | 0%
libnpp-12.0.2.50 | 139.8 MB | | 0%
libcufft-11.0.2.4 | 102.9 MB | | 0%
libcusolver-11.4.4.5 | 98.3 MB | | 0%
libcurand-10.3.2.56 | 51.7 MB | | 0%
cuda-nvrtc-12.1.55 | 19.7 MB | | 0%
libnvjitlink-12.1.55 | 16.9 MB | | 0%
libnvjpeg-12.1.0.39 | 2.5 MB | | 0%
libcufile-1.6.0.25 | 763 KB | | 0%
cuda-cudart-12.1.55 | 189 KB | | 0%
cuda-opencl-12.1.56 | 11 KB | | 0%
cuda-libraries-12.1. | 2 KB | | 0%
cuda-runtime-12.1.0 | 1 KB | | 0%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(cogvlm) develop@itserver03:/u01/workspace/cogvlm/CogVLM$
安装CogVLM依赖库
pip install -r requirements.txt
在安装后，启动web界面时，会出现报错，可能碰到如下安装包依赖库问题。huggingface_hub版本不要用最新版。这里制定版本huggingface_hub==0.21.4。bitsandbytes，chardet 这两库可能会需要单独在安装以便，这里至少我是碰到了错误。
pip install bitsandbytes
pip install chardet
pip install huggingface_hub==0.21.4
安装语言模型（非必须）
python -m spacy download en_core_web_sm
运行
运行web界面
运行前请先修改模型地址，编辑composite_demo/client.py 文件中默认的模型地址
models_info = {'tokenizer': {#'path': os.environ.get('TOKENIZER_PATH', 'lmsys/vicuna-7b-v1.5'),'path': os.environ.get('TOKENIZER_PATH', '/u01/workspace/cogvlm/models/vicuna-7b-v1.5'),},'agent_chat': {#'path': os.environ.get('MODEL_PATH_AGENT_CHAT', 'THUDM/cogagent-chat-hf'),'path': os.environ.get('MODEL_PATH_AGENT_CHAT', '/u01/workspace/cogvlm/models/cogagent-chat-hf'),'device': ['cuda:0']},'vlm_chat': {#'path': os.environ.get('MODEL_PATH_VLM_CHAT', 'THUDM/cogvlm-chat-hf'),'path': os.environ.get('MODEL_PATH_VLM_CHAT', '/u01/workspace/cogvlm/models/cogvlm-chat-hf'),'device': ['cuda:0']},'vlm_grounding': {#'path': os.environ.get('MODEL_PATH_VLM_GROUNDING','THUDM/cogvlm-grounding-generalist-hf'),'path': os.environ.get('MODEL_PATH_VLM_GROUNDING','/u01/workspace/cogvlm/models/cogvlm-grounding-generalist-hf'),'device': ['cuda:']}
}
执行启动命令
streamlit run composite_demo/main.py
成功后可以打开界面
控制台交互式运行
在python basic_demo/cli_demo_hf.py中运行代码,注意替换模型地址
python cli_demo_hf.py --from_pretrained /u01/workspace/cogvlm/models/cogvlm-chat-hf --fp16 --quant 4
OpenAI 方式 Restful API 运行
运行服务端
python openai_demo/openai_api.py
客户端请求
请编辑openai_demo/openai_api_request.py中的图片地址以及你需要提的问题，例如
messages = [{"role": "user","content": [{"type": "text","text": "What’s in this image?",},{"type": "image_url","image_url": {"url": img_url},},],},{"role": "assistant","content": "The image displays a wooden boardwalk extending through a vibrant green grassy wetland. The sky is partly cloudy with soft, wispy clouds, indicating nice weather. Vegetation is seen on either side of the boardwalk, and trees are present in the background, suggesting that this area might be a natural reserve or park designed for ecological preservation and outdoor recreation. The boardwalk allows visitors to explore the area without disturbing the natural habitat.",},{"role": "user","content": "Do you think this is a spring or winter photo?"},
]
if __name__ == "__main__":simple_image_chat(use_stream=False, img_path="/u01/workspace/cogvlm/CogVLM/openai_demo/demo.jpg")
运行客户端请求命令
python openai_demo/openai_api_request.py
【Qinghub Studio 】更适合开发人员的低代码开源开发平台
【QingHub企业级应用统一部署】
【QingHub企业级应用开发管理】
【QingHub** 演示】**
【https://qingplus.cn】
这篇关于CogVLM 本地部署体验（问题解决）的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
