ComfyUI 集成混元DIT(comfyui-hydit)

2024-06-14 09:44
文章标签 集成 comfyui 混元 dit hydit

本文主要是介绍ComfyUI 集成混元DIT(comfyui-hydit),希望对大家解决编程问题提供一定的参考价值,需要的开发者们随着小编来一起学习吧!

最近腾讯官方推出了ComfyUI插件comfyui-hydit 。是一个专门为腾讯的 Hunyuan-DiT 模型设计的自定义节点和工作流。本文主要介绍如何通过ComfyUI来运行腾讯新出的支持中文提示词的混元文生图大模型Hunyuan-DiT

环境准备

插件

从腾讯混元DIT 源码库获取插件源码:
https://github.com/Tencent/HunyuanDiT.git
从该仓库中获取 :comfyui-hydit

模型

所有模型下载后保存在需要挂载的路径下。

所有模型下载后保存在需要挂载的路径下:/u01/workspace/models/hunyuan

克隆HunyuanDiT模型

有多个模型可选择,我们这里采用了第三个模型:
Tencent-Hunyuan/HunyuanDiT 基础模型
Tencent-Hunyuan/HunyuanDiT-Diffusers 加入Diffusers格式
Tencent-Hunyuan/Distillation 加速版

git clone https://huggingface.co/Tencent-Hunyuan/HunyuanDiT
git clone https://huggingface.co/Tencent-Hunyuan/Distillation

如果想要加速版,以上两个地址的文件都要下载
注意下载完Distillation 模型后,需要拷贝模型pytorch_model_distill.pt到: HunyuanDiT/t2i/model/目录下

下载OpenAI视觉模型

git clone https://hf-mirror.com/openai/clip-vit-large-patch14-336
:::danger
保存位置:/u01/workspace/models/clip-vit-large-patch14-336
源码修改方式请参照混元模型的部署过程。
:::

容器化部署ComfyUI

复制comfyui-hydit插件目录到ComfyUI custom_nodes目录:

├── AIGODLIKE-ComfyUI-Translation
├── comfyui_controlnet_aux
├── ComfyUI_Custom_Nodes_AlekPet
├── ComfyUI-Custom-Scripts
├── comfyui-hydit
├── ComfyUI-Manager
├── ComfyUI_TiledKSampler
├── ComfyUI_UltimateSDUpscale
├── Derfuu_ComfyUI_ModdedNodes
├── efficiency-nodes-comfyui
├── example_node.py.example
├── __pycache__
├── sdxl_prompt_styler
└── websocket_image_save.py

准备Dockerfile文件

准备Dockerfile可以安装预装插件和不预装两种不同的方式,如果不预装插件,可以把插件挂载到宿主机上等启动之在安装插件,本文采用的是预装的方式准备的Dockerfile

FROM pytorch/pytorch:2.3.0-cuda12.1-cudnn8-runtime
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
ENV LANG=zh_CN.UTF-8
WORKDIR /appRUN apt-get update 
RUN apt-get install -y libgl1-mesa-glx libglib2.0-0 libsm6 libxrender1 libxext6 git
RUN rm -rf /var/lib/apt/lists/*
ENV GIT_PYTHON_REFRESH=quiet RUN pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
RUN pip config set install.trusted-host mirrors.aliyun.comCOPY ComfyUI/ /app/ComfyUI/
WORKDIR /app/ComfyUIRUN pip install --use-pep517 -r requirements.txt
RUN pip install -r  custom_nodes/comfyui_controlnet_aux/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI-Manager/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI_Custom_Nodes_AlekPet/ArgosTranslateNode/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI_Custom_Nodes_AlekPet/DeepTranslatorNode/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI_Custom_Nodes_AlekPet/GoogleTranslateNode/requirements.txt
RUN pip install -r  custom_nodes/efficiency-nodes-comfyui/requirements.txt
RUN pip install -r  custom_nodes/comfyui-hydit/requirements.txtEXPOSE 8188
CMD [ "python","main.py ","--port 8188"]

custom_nodes/comfyui-hydit/requirements.txt 修改:

#--extra-index-url https://pypi.ngc.nvidia.com
#--extra-index-url https://download.pytorch.org/whl/cu117
timm
diffusers
peft
protobuf
accelerate
loguru
sentencepiece
cuda-python
polygraphy
pandas
omegaconf
# torch>=2.3.0
# torchvision>=0.18.0
# torchaudio>=2.3.0
xformers>=0.0.26.post1
pytorch_lightning

执行构建

docker buile -t qingcloudtech/comfyui:v1.1 .

启动容器

docker run -it --gpus all \
-p 8188:8188 \
-v /u01/workspace/models/stablediffusion:/app/ComfyUI/models \
-v /u01/workspace/comfyui/output/:/app/ComfyUI/output \
- /u01/workspace/ComfyUI/extra_model_paths.yaml:/app/ComfyUI/extra_model_paths.yaml
qingcloudtech/comfyui:v1.5 python main.py

启动容器(docker-compose方式)

准备配置文件
version: '3.8'
services:comfyui:image: qingcloudtech/comfyui:v1.1container_name: comfyuicommand: python /app/ComfyUI/main.pydeploy:resources:reservations:devices:- driver: nvidiacount: allcapabilities: [gpu]volumes:- /u01/workspace/models/stablediffusion/:/app/ComfyUI/models/- /u01/workspace/comfyui/output/:/app/ComfyUI/output- /u01/workspace/ComfyUI/extra_model_paths.yaml:/app/ComfyUI/extra_model_paths.yamlnetwork_mode: host
执行启动命令:
docker-compose up -d

浏览器访问:

演示访问:

1、浏览器访问:127.0.0.1:8188
2、从web界面中加载如下测试json:

{"last_node_id": 21,"last_link_id": 6,"nodes": [{"id": 19,"type": "DiffusersClipTextEncode","pos": [74,277],"size": {"0": 400,"1": 200},"flags": {},"order": 0,"mode": 0,"outputs": [{"name": "positive","type": "STRINGC","links": [5],"shape": 3,"label": "positive"},{"name": "negative","type": "STRINGC","links": [6],"shape": 3,"label": "negative"}],"properties": {"Node name for S&R": "DiffusersClipTextEncode"},"widgets_values": ["castle,landscape,1girl,loli,silver hair,school_uniform,crystal_earrings,kind_smile,arm_support,in summer,mini_witch_hat,frilled,strappy_heels,book,\n","错误的眼睛,糟糕的人脸,毁容,糟糕的艺术,变形,多余的肢体,模糊的颜色,模糊,重复,病态,残缺,"]},{"id": 16,"type": "DiffusersModelMakeup","pos": [526,393],"size": {"0": 304.79998779296875,"1": 46},"flags": {},"order": 3,"mode": 0,"inputs": [{"name": "pipeline","type": "PIPELINE","link": 2,"label": "pipeline"},{"name": "scheduler","type": "SCHEDULER","link": 3,"label": "scheduler"}],"outputs": [{"name": "MAKED_PIPELINE","type": "MAKED_PIPELINE","links": [4],"shape": 3,"label": "MAKED_PIPELINE"}],"properties": {"Node name for S&R": "DiffusersModelMakeup"}},{"id": 18,"type": "DiffusersSampler","pos": [842,132],"size": [315,454],"flags": {},"order": 4,"mode": 0,"inputs": [{"name": "maked_pipeline","type": "MAKED_PIPELINE","link": 4,"label": "maked_pipeline"},{"name": "positive","type": "STRINGC","link": 5,"label": "positive"},{"name": "negative","type": "STRINGC","link": 6,"label": "negative"}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [1],"shape": 3,"label": "IMAGE"}],"properties": {"Node name for S&R": "DiffusersSampler"},"widgets_values": [1,1024,1024,30,6,2705109163,"randomize"]},{"id": 6,"type": "PreviewImage","pos": [1186,181],"size": [210,246],"flags": {},"order": 5,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 1,"label": "图像"}],"title": "Preview Image","properties": {"Node name for S&R": "PreviewImage"}},{"id": 21,"type": "DiffusersPipelineLoader","pos": [81,544],"size": {"0": 315,"1": 190},"flags": {},"order": 1,"mode": 0,"outputs": [{"name": "PIPELINE","type": "PIPELINE","links": [2],"shape": 3,"label": "PIPELINE"},{"name": "MODEL","type": "MODEL","links": null,"shape": 3,"label": "MODEL"},{"name": "CLIP","type": "CLIP","links": null,"shape": 3,"label": "CLIP"},{"name": "VAE","type": "VAE","links": null,"shape": 3,"label": "VAE"}],"properties": {"Node name for S&R": "DiffusersPipelineLoader"},"widgets_values": ["ckpts","pytorch_model_distill.pt","disable","diffusers"]},{"id": 15,"type": "DiffusersSchedulerLoader","pos": [73,129],"size": {"0": 315,"1": 58},"flags": {},"order": 2,"mode": 0,"outputs": [{"name": "SCHEDULER","type": "SCHEDULER","links": [3],"shape": 3,"label": "SCHEDULER"}],"properties": {"Node name for S&R": "DiffusersSchedulerLoader"},"widgets_values": ["ddim"]}],"links": [[1,18,0,6,0,"IMAGE"],[2,21,0,16,0,"PIPELINE"],[3,15,0,16,1,"SCHEDULER"],[4,16,0,18,0,"MAKED_PIPELINE"],[5,19,0,18,1,"STRINGC"],[6,19,1,18,2,"STRINGC"]],"groups": [],"config": {},"extra": {"ds": {"scale": 0.9090909090909098,"offset": {"0": 163.4918776991919,"1": 16.755363890844446}}},"version": 0.4
}

3、查看结果:
image.png


【Qinghub Studio 】更适合开发人员的低代码开源开发平台
QingHub企业级应用统一部署
QingHub企业级应用开发管理
QingHub演示
https://qingplus.cn

这篇关于ComfyUI 集成混元DIT(comfyui-hydit)的文章就介绍到这儿,希望我们推荐的文章对编程师们有所帮助!



http://www.chinasem.cn/article/1060049

相关文章

【区块链 + 人才服务】区块链集成开发平台 | FISCO BCOS应用案例

随着区块链技术的快速发展,越来越多的企业开始将其应用于实际业务中。然而,区块链技术的专业性使得其集成开发成为一项挑战。针对此,广东中创智慧科技有限公司基于国产开源联盟链 FISCO BCOS 推出了区块链集成开发平台。该平台基于区块链技术,提供一套全面的区块链开发工具和开发环境,支持开发者快速开发和部署区块链应用。此外,该平台还可以提供一套全面的区块链开发教程和文档,帮助开发者快速上手区块链开发。

从0到1,AI我来了- (7)AI应用-ComfyUI-II(进阶)

上篇comfyUI 入门 ,了解了TA是个啥,这篇,我们通过ComfyUI 及其相关Lora 模型,生成一些更惊艳的图片。这篇主要了解这些内容:         1、哪里获取模型?         2、实践如何画一个美女?         3、附录:               1)相关SD(稳定扩散模型的组成部分)               2)模型放置目录(重要)

【Shiro】Shiro 的学习教程(三)之 SpringBoot 集成 Shiro

目录 1、环境准备2、引入 Shiro3、实现认证、退出3.1、使用死数据实现3.2、引入数据库,添加注册功能后端代码前端代码 3.3、MD5、Salt 的认证流程 4.、实现授权4.1、基于角色授权4.2、基于资源授权 5、引入缓存5.1、EhCache 实现缓存5.2、集成 Redis 实现 Shiro 缓存 1、环境准备 新建一个 SpringBoot 工程,引入依赖:

系统架构师-ERP+集成

ERP   集成平台end:就懒得画新的页

Spring Boot集成Tess4J实现OCR

1.什么是Tess4j? Tesseract是一个开源的光学字符识别(OCR)引擎,它可以将图像中的文字转换为计算机可读的文本。支持多种语言和书面语言,并且可以在命令行中执行。它是一个流行的开源OCR工具,可以在许多不同的操作系统上运行。Tess4J是一个基于Tesseract OCR引擎的Java接口,可以用来识别图像中的文本,说白了,就是封装了它的API,让Java可以直接调用。 Tess

使用Spring Boot集成Spring Data JPA和单例模式构建库存管理系统

引言 在企业级应用开发中,数据库操作是非常重要的一环。Spring Data JPA提供了一种简化的方式来进行数据库交互,它使得开发者无需编写复杂的JPA代码就可以完成常见的CRUD操作。此外,设计模式如单例模式可以帮助我们更好地管理和控制对象的创建过程,从而提高系统的性能和可维护性。本文将展示如何结合Spring Boot、Spring Data JPA以及单例模式来构建一个基本的库存管理系统

Spring Boot集成PDFBox实现电子签章

概述 随着无纸化办公的普及,电子文档的使用越来越广泛。电子签章作为一种有效的身份验证方式,在很多场景下替代了传统的纸质文件签名。Apache PDFBox 是一个开源的Java库,可以用来渲染、生成、填写PDF文档等操作。本文将介绍如何使用Spring Boot框架结合PDFBox来实现电子签章功能。 准备工作 环境搭建:确保你的开发环境中安装了JDK 8或更高版本,并且配置好了Maven或

springboot+maven搭建的项目,集成单元测试

springboot+maven搭建的项目,集成单元测试 1.在pom.xml文件中引入单元测试的依赖包 <!--单元测试依赖--><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></depen

Java消息队列:RabbitMQ与Kafka的集成与应用

Java消息队列:RabbitMQ与Kafka的集成与应用 大家好,我是微赚淘客返利系统3.0的小编,是个冬天不穿秋裤,天冷也要风度的程序猿! 在现代的分布式系统中,消息队列是实现系统间通信、解耦和提高可扩展性的重要组件。RabbitMQ和Kafka是两个广泛使用的消息队列系统,它们各有特点和优势。本文将介绍如何在Java应用中集成RabbitMQ和Kafka,并展示它们的应用场景。 消息队

java后端服务监控与告警:Prometheus与Grafana集成

Java后端服务监控与告警:Prometheus与Grafana集成 大家好,我是微赚淘客返利系统3.0的小编,是个冬天不穿秋裤,天冷也要风度的程序猿! 在现代的微服务架构中,监控和告警是确保服务稳定性的关键组成部分。Prometheus和Grafana是两个强大的工具,它们可以集成在一起,为Java后端服务提供实时监控和可视化告警。 服务监控的重要性 服务监控可以帮助我们实时了解服务的健