ComfyUI 集成混元DIT（comfyui-hydit）

2024-06-14 09:44
文章标签 集成 comfyui 混元 dit hydit

本文主要是介绍ComfyUI 集成混元DIT（comfyui-hydit），希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

最近腾讯官方推出了ComfyUI插件comfyui-hydit 。是一个专门为腾讯的 Hunyuan-DiT 模型设计的自定义节点和工作流。本文主要介绍如何通过ComfyUI来运行腾讯新出的支持中文提示词的混元文生图大模型Hunyuan-DiT

环境准备

插件

从腾讯混元DIT 源码库获取插件源码：
https://github.com/Tencent/HunyuanDiT.git
从该仓库中获取 ：comfyui-hydit

模型

所有模型下载后保存在需要挂载的路径下。

所有模型下载后保存在需要挂载的路径下：/u01/workspace/models/hunyuan

克隆HunyuanDiT模型

有多个模型可选择，我们这里采用了第三个模型：
Tencent-Hunyuan/HunyuanDiT 基础模型
Tencent-Hunyuan/HunyuanDiT-Diffusers 加入Diffusers格式
Tencent-Hunyuan/Distillation 加速版

git clone https://huggingface.co/Tencent-Hunyuan/HunyuanDiT
git clone https://huggingface.co/Tencent-Hunyuan/Distillation

如果想要加速版，以上两个地址的文件都要下载
注意下载完Distillation 模型后，需要拷贝模型pytorch_model_distill.pt到： HunyuanDiT/t2i/model/目录下

下载OpenAI视觉模型

git clone https://hf-mirror.com/openai/clip-vit-large-patch14-336
:::danger
保存位置：/u01/workspace/models/clip-vit-large-patch14-336
源码修改方式请参照混元模型的部署过程。
:::

容器化部署ComfyUI

复制comfyui-hydit插件目录到ComfyUI custom_nodes目录：

├── AIGODLIKE-ComfyUI-Translation
├── comfyui_controlnet_aux
├── ComfyUI_Custom_Nodes_AlekPet
├── ComfyUI-Custom-Scripts
├── comfyui-hydit
├── ComfyUI-Manager
├── ComfyUI_TiledKSampler
├── ComfyUI_UltimateSDUpscale
├── Derfuu_ComfyUI_ModdedNodes
├── efficiency-nodes-comfyui
├── example_node.py.example
├── __pycache__
├── sdxl_prompt_styler
└── websocket_image_save.py

准备Dockerfile文件

准备Dockerfile可以安装预装插件和不预装两种不同的方式，如果不预装插件，可以把插件挂载到宿主机上等启动之在安装插件，本文采用的是预装的方式准备的Dockerfile

FROM pytorch/pytorch:2.3.0-cuda12.1-cudnn8-runtime
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
ENV LANG=zh_CN.UTF-8
WORKDIR /appRUN apt-get update 
RUN apt-get install -y libgl1-mesa-glx libglib2.0-0 libsm6 libxrender1 libxext6 git
RUN rm -rf /var/lib/apt/lists/*
ENV GIT_PYTHON_REFRESH=quiet RUN pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
RUN pip config set install.trusted-host mirrors.aliyun.comCOPY ComfyUI/ /app/ComfyUI/
WORKDIR /app/ComfyUIRUN pip install --use-pep517 -r requirements.txt
RUN pip install -r  custom_nodes/comfyui_controlnet_aux/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI-Manager/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI_Custom_Nodes_AlekPet/ArgosTranslateNode/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI_Custom_Nodes_AlekPet/DeepTranslatorNode/requirements.txt
RUN pip install -r  custom_nodes/ComfyUI_Custom_Nodes_AlekPet/GoogleTranslateNode/requirements.txt
RUN pip install -r  custom_nodes/efficiency-nodes-comfyui/requirements.txt
RUN pip install -r  custom_nodes/comfyui-hydit/requirements.txtEXPOSE 8188
CMD [ "python","main.py ","--port 8188"]

custom_nodes/comfyui-hydit/requirements.txt 修改：

#--extra-index-url https://pypi.ngc.nvidia.com
#--extra-index-url https://download.pytorch.org/whl/cu117
timm
diffusers
peft
protobuf
accelerate
loguru
sentencepiece
cuda-python
polygraphy
pandas
omegaconf
# torch>=2.3.0
# torchvision>=0.18.0
# torchaudio>=2.3.0
xformers>=0.0.26.post1
pytorch_lightning

执行构建

docker buile -t qingcloudtech/comfyui:v1.1 .

启动容器

docker run -it --gpus all \
-p 8188:8188 \
-v /u01/workspace/models/stablediffusion:/app/ComfyUI/models \
-v /u01/workspace/comfyui/output/:/app/ComfyUI/output \
- /u01/workspace/ComfyUI/extra_model_paths.yaml:/app/ComfyUI/extra_model_paths.yaml
qingcloudtech/comfyui:v1.5 python main.py

启动容器（docker-compose方式）

准备配置文件
version: '3.8'
services:comfyui:image: qingcloudtech/comfyui:v1.1container_name: comfyuicommand: python /app/ComfyUI/main.pydeploy:resources:reservations:devices:- driver: nvidiacount: allcapabilities: [gpu]volumes:- /u01/workspace/models/stablediffusion/:/app/ComfyUI/models/- /u01/workspace/comfyui/output/:/app/ComfyUI/output- /u01/workspace/ComfyUI/extra_model_paths.yaml:/app/ComfyUI/extra_model_paths.yamlnetwork_mode: host
执行启动命令：
docker-compose up -d

浏览器访问：

演示访问：

1、浏览器访问：127.0.0.1:8188
2、从web界面中加载如下测试json：

{"last_node_id": 21,"last_link_id": 6,"nodes": [{"id": 19,"type": "DiffusersClipTextEncode","pos": [74,277],"size": {"0": 400,"1": 200},"flags": {},"order": 0,"mode": 0,"outputs": [{"name": "positive","type": "STRINGC","links": [5],"shape": 3,"label": "positive"},{"name": "negative","type": "STRINGC","links": [6],"shape": 3,"label": "negative"}],"properties": {"Node name for S&R": "DiffusersClipTextEncode"},"widgets_values": ["castle,landscape,1girl,loli,silver hair,school_uniform,crystal_earrings,kind_smile,arm_support,in summer,mini_witch_hat,frilled,strappy_heels,book,\n","错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，"]},{"id": 16,"type": "DiffusersModelMakeup","pos": [526,393],"size": {"0": 304.79998779296875,"1": 46},"flags": {},"order": 3,"mode": 0,"inputs": [{"name": "pipeline","type": "PIPELINE","link": 2,"label": "pipeline"},{"name": "scheduler","type": "SCHEDULER","link": 3,"label": "scheduler"}],"outputs": [{"name": "MAKED_PIPELINE","type": "MAKED_PIPELINE","links": [4],"shape": 3,"label": "MAKED_PIPELINE"}],"properties": {"Node name for S&R": "DiffusersModelMakeup"}},{"id": 18,"type": "DiffusersSampler","pos": [842,132],"size": [315,454],"flags": {},"order": 4,"mode": 0,"inputs": [{"name": "maked_pipeline","type": "MAKED_PIPELINE","link": 4,"label": "maked_pipeline"},{"name": "positive","type": "STRINGC","link": 5,"label": "positive"},{"name": "negative","type": "STRINGC","link": 6,"label": "negative"}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [1],"shape": 3,"label": "IMAGE"}],"properties": {"Node name for S&R": "DiffusersSampler"},"widgets_values": [1,1024,1024,30,6,2705109163,"randomize"]},{"id": 6,"type": "PreviewImage","pos": [1186,181],"size": [210,246],"flags": {},"order": 5,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 1,"label": "图像"}],"title": "Preview Image","properties": {"Node name for S&R": "PreviewImage"}},{"id": 21,"type": "DiffusersPipelineLoader","pos": [81,544],"size": {"0": 315,"1": 190},"flags": {},"order": 1,"mode": 0,"outputs": [{"name": "PIPELINE","type": "PIPELINE","links": [2],"shape": 3,"label": "PIPELINE"},{"name": "MODEL","type": "MODEL","links": null,"shape": 3,"label": "MODEL"},{"name": "CLIP","type": "CLIP","links": null,"shape": 3,"label": "CLIP"},{"name": "VAE","type": "VAE","links": null,"shape": 3,"label": "VAE"}],"properties": {"Node name for S&R": "DiffusersPipelineLoader"},"widgets_values": ["ckpts","pytorch_model_distill.pt","disable","diffusers"]},{"id": 15,"type": "DiffusersSchedulerLoader","pos": [73,129],"size": {"0": 315,"1": 58},"flags": {},"order": 2,"mode": 0,"outputs": [{"name": "SCHEDULER","type": "SCHEDULER","links": [3],"shape": 3,"label": "SCHEDULER"}],"properties": {"Node name for S&R": "DiffusersSchedulerLoader"},"widgets_values": ["ddim"]}],"links": [[1,18,0,6,0,"IMAGE"],[2,21,0,16,0,"PIPELINE"],[3,15,0,16,1,"SCHEDULER"],[4,16,0,18,0,"MAKED_PIPELINE"],[5,19,0,18,1,"STRINGC"],[6,19,1,18,2,"STRINGC"]],"groups": [],"config": {},"extra": {"ds": {"scale": 0.9090909090909098,"offset": {"0": 163.4918776991919,"1": 16.755363890844446}}},"version": 0.4
}

3、查看结果：
image.png

【Qinghub Studio 】更适合开发人员的低代码开源开发平台
【QingHub企业级应用统一部署】
【QingHub企业级应用开发管理】
【QingHub演示】
【https://qingplus.cn】

这篇关于ComfyUI 集成混元DIT（comfyui-hydit）的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！


http://www.chinasem.cn/article/1060049。

相关文章

Spring Boot集成Druid实现数据源管理与监控的详细步骤

Spring Boot集成Druid实现数据源管理与监控的详细步骤

《SpringBoot集成Druid实现数据源管理与监控的详细步骤》本文介绍如何在SpringBoot项目中集成Druid数据库连接池,包括环境搭建、Maven依赖配置、SpringBoot配置文件... 目录1. 引言1.1 环境准备1.2 Druid介绍2. 配置Druid连接池3. 查看Druid监控
阅读更多...
在Spring Boot中集成RabbitMQ的实战记录

在Spring Boot中集成RabbitMQ的实战记录

《在SpringBoot中集成RabbitMQ的实战记录》本文介绍SpringBoot集成RabbitMQ的步骤,涵盖配置连接、消息发送与接收,并对比两种定义Exchange与队列的方式:手动声明(... 目录前言准备工作1. 安装 RabbitMQ2. 消息发送者（Producer）配置1. 创建 Spr
阅读更多...
如何在Spring Boot项目中集成MQTT协议

如何在Spring Boot项目中集成MQTT协议

《如何在SpringBoot项目中集成MQTT协议》本文介绍在SpringBoot中集成MQTT的步骤,包括安装Broker、添加EclipsePaho依赖、配置连接参数、实现消息发布订阅、测试接口... 目录1. 准备工作2. 引入依赖3. 配置MQTT连接4. 创建MQTT配置类5. 实现消息发布与订阅
阅读更多...
SpringBoot集成LiteFlow工作流引擎的完整指南

SpringBoot集成LiteFlow工作流引擎的完整指南

《SpringBoot集成LiteFlow工作流引擎的完整指南》LiteFlow作为一款国产轻量级规则引擎/流程引擎,以其零学习成本、高可扩展性和极致性能成为微服务架构下的理想选择,本文将详细讲解Sp... 目录一、LiteFlow核心优势二、SpringBoot集成实战三、高级特性应用1. 异步并行执行2
阅读更多...
SpringBoot3应用中集成和使用Spring Retry的实践记录

SpringBoot3应用中集成和使用Spring Retry的实践记录

《SpringBoot3应用中集成和使用SpringRetry的实践记录》SpringRetry为SpringBoot3提供重试机制,支持注解和编程式两种方式,可配置重试策略与监听器,适用于临时性故... 目录1. 简介2. 环境准备3. 使用方式3.1 注解方式 基础使用自定义重试策略失败恢复机制注意事项
阅读更多...
SpringBoot集成LiteFlow实现轻量级工作流引擎的详细过程

SpringBoot集成LiteFlow实现轻量级工作流引擎的详细过程

《SpringBoot集成LiteFlow实现轻量级工作流引擎的详细过程》LiteFlow是一款专注于逻辑驱动流程编排的轻量级框架,它以组件化方式快速构建和执行业务流程,有效解耦复杂业务逻辑,下面给大... 目录一、基础概念1.1 组件（Component）1.2 规则（Rule）1.3 上下文（Conte
阅读更多...
使用vscode搭建pywebview集成vue项目实践

使用vscode搭建pywebview集成vue项目实践

《使用vscode搭建pywebview集成vue项目实践》：本文主要介绍使用vscode搭建pywebview集成vue项目实践,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地... 目录环境准备项目源码下载项目说明调试与生成可执行文件核心代码说明总结本节我们使用pythonpywebv
阅读更多...
Maven项目中集成数据库文档生成工具的操作步骤

Maven项目中集成数据库文档生成工具的操作步骤

《Maven项目中集成数据库文档生成工具的操作步骤》在Maven项目中,可以通过集成数据库文档生成工具来自动生成数据库文档,本文为大家整理了使用screw-maven-plugin（推荐）的完... 目录1. 添加插件配置到 pom.XML2. 配置数据库信息3. 执行生成命令4. 高级配置选项5. 注意事
阅读更多...
Java集成Onlyoffice的示例代码及场景分析

Java集成Onlyoffice的示例代码及场景分析

《Java集成Onlyoffice的示例代码及场景分析》：本文主要介绍Java集成Onlyoffice的示例代码及场景分析,本文给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要... 需求场景：实现文档的在线编辑，团队协作总结：两个接口 + 前端页面 + 配置项接口1：一个接口，将o
阅读更多...
Swagger2与Springdoc集成与使用详解

Swagger2与Springdoc集成与使用详解

《Swagger2与Springdoc集成与使用详解》：本文主要介绍Swagger2与Springdoc集成与使用方式,具有很好的参考价值,希望对大家有所帮助,如有错误或未考虑完全的地方,望不吝赐... 目录1. 依赖配置2. 基础配置2.1 启用 Springdoc2.2 自定义 OpenAPI 信息3.
阅读更多...

Copyright China编程 23002807@qq.com

沪ICP备2023033072号-2