Deepseek's Janus-pro-7b Model and My Tinkering

2 2 月, 2025 20点热度 0人点赞 0条评论

背景：

春节期间 Deepseek又放了一个大招，发布了 Janus-pro-7b 多模态的大模型，闲来无事，必须折腾一下。

先后折腾了2个环境，卡分别是 T4 16G 和 4090 ,24G . 因为 Janus-pro-7b 跑生图流程时会oom，跑 Janus-pro-1b，生成图片凑合用，于是想办法从T4换成了4090后跑通了，实际观察显存也就是16G超过一点点，估计改改参数T4也能用，暂时还不会优化，有空继续。

过程如下：

安装步骤

1，安装 https://github.com/comfyanonymous/ComfyUI

2，在 ComfyUI/custom_nodes 里面安装 https://github.com/ltdrdata/ComfyUI-Manager

3，在 ComfyUI/ custom_nodes 里面安装 https://github.com/CY-CHENYUE/ComfyUI-Janus-Pro/

4, 在 huggingface.co 下载模型 https://huggingface.co/deepseek-ai/Janus-Pro-1B 和 https://huggingface.co/deepseek-ai/Janus-Pro-7B/ 把模型文件放在 ComfyUI/models/Janus-Pro 文件夹中

5，回到 ComfyUI 目录，执行 python main.py, 会提示一个 http://127.0.0.1:8188 url暴露出来了

6，访问 http://127.0.0.1:8188 web页面，加载来自 https://github.com/CY-CHENYUE/ComfyUI-Janus-Pro/ 的流程模版（分析图片和生成图片），等待结果

7，以上所谓安装，基本都是执行 git clone，然后在目录中执行

python -m pip install -r requirements.txt 即可，具体每一步细节可以看各个链接的安装介绍。

开始体验

启动过程日志记录一下

python main.py 
[START] Security scan
[DONE] Security scan
## ComfyUI-Manager: installing dependencies. (GitPython)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
## ComfyUI-Manager: installing dependencies done.
** ComfyUI startup time: 2025-01-30 12:53:11.310
** Platform: Linux
** Python version: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0]
** Python executable: /opt/conda/bin/python
** ComfyUI Path: /home/jovyan/ComfyUI
** ComfyUI Base Folder Path: /home/jovyan/ComfyUI
** User directory: /home/jovyan/ComfyUI/user
** ComfyUI-Manager config path: /home/jovyan/ComfyUI/user/default/ComfyUI-Manager/config.ini
** Log path: /home/jovyan/ComfyUI/user/comfyui.log
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Prestartup times for custom nodes:
  22.6 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Manager

Checkpoint files will always be loaded safely.
Total VRAM 24210 MB, total RAM 257560 MB
pytorch version: 2.6.0+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
Using pytorch attention
ComfyUI version: 0.3.12
[Prompt Server] web root: /home/jovyan/ComfyUI/web
### Loading: ComfyUI-Manager (V3.9.5)
### ComfyUI Version: v0.3.12-28-g13fd4d6e | Released on '2025-01-28'

Import times for custom nodes:
   0.0 seconds: /home/jovyan/ComfyUI/custom_nodes/websocket_image_save.py
   0.0 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Janus-Pro
   0.0 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts
   0.2 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Manager

Starting server

To see the GUI go to: http://127.0.0.1:8188
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json

python main.py

[START] Security scan

[DONE] Security scan

## ComfyUI-Manager: installing dependencies. (GitPython)

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

## ComfyUI-Manager: installing dependencies done.

** ComfyUI startup time: 2025-01-30 12:53:11.310

** Platform: Linux

** Python version: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0]

** Python executable: /opt/conda/bin/python

** ComfyUI Path: /home/jovyan/ComfyUI

** ComfyUI Base Folder Path: /home/jovyan/ComfyUI

** User directory: /home/jovyan/ComfyUI/user

** ComfyUI-Manager config path: /home/jovyan/ComfyUI/user/default/ComfyUI-Manager/config.ini

** Log path: /home/jovyan/ComfyUI/user/comfyui.log

Prestartup times for custom nodes:

22.6 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Manager

Checkpoint files will always be loaded safely.

Total VRAM 24210 MB, total RAM 257560 MB

pytorch version: 2.6.0+cu124

Set vram state to: NORMAL_VRAM

Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync

Using pytorch attention

ComfyUI version: 0.3.12

[Prompt Server] web root: /home/jovyan/ComfyUI/web

### Loading: ComfyUI-Manager (V3.9.5)

### ComfyUI Version: v0.3.12-28-g13fd4d6e | Released on '2025-01-28'

Import times for custom nodes:

0.0 seconds: /home/jovyan/ComfyUI/custom_nodes/websocket_image_save.py

0.0 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Janus-Pro

0.0 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts

0.2 seconds: /home/jovyan/ComfyUI/custom_nodes/ComfyUI-Manager

Starting server

To see the GUI go to: http://127.0.0.1:8188

[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/model-list.json

[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json

执行效果图

遇到的几个问题

下载 https://huggingface.co/deepseek-ai/Janus-Pro-7B 比较费时，里面还用到了 git-lfs，折腾完了才发现可以用魔搭modelscope 加速下载。
跑工作流生图时基本在排障，oom ，改成1b模型，提示词不支持中文，生成的图片是同一张，换成英文基本可行，描述图片比较准确，堪用。
显存不足时的报错日志如下

File "/root/miniforge3/lib/python3.10/site-packages/transformers/cache_utils.py", line 450, in update
    self.value_cache[layer_idx] = torch.cat([self.value_cache[layer_idx], value_states], dim=-2)
torch.OutOfMemoryError: Allocation on device 

2025-01-29T10:56:28.826798 - Got an OOM, unloading all loaded models.