As promised in my last post, today I am testing out Thibaud Zamora’s SDXL-controlnet: OpenPose (v2) model using ComfyUI. I keep saying I’ll keep my posts short but never do...

About OpenPose and ControlNet

Quoting from the OpenPose Git, “OpenPose has represented the first real-time multi-person system to jointly detect human body, hand, facial, and foot keypoints (in total 135 keypoints) on single images.” In my very layman view, this means that given a reference image of the full body of a person, OpenPose can generate what I’d call a “skeleton wireframe” (keypoints) to describe the position of the persons limbs in space.

And as noted in my previous post, SDXL 1.0 with SDXL-ControlNet: Canny, ControlNet “is a neural network structure to control diffusion models by adding extra conditions.” With ControlNet, we can train an AI model to “understand” OpenPose data (i.e. the position of a person’s limbs in a reference image) and then apply these conditions to Stable Diffusion XL when generating our own images, according to a pose we define.

Install controlnet-openpose-sdxl-1.0

And we have Thibaud Zamora to thank for providing us such a trained model!

• Head over to HuggingFace and download OpenPoseXL2.safetensors from the controlnet-openpose-sdxl-1.0 repository, under Files and versions
• Place the file in the ComfyUI folder models\controlnet.

No-Code Workflow

First, we need a posed “skeleton wireframe” - a great and beautiful source is OpenPoses.com. Here you can clearly see what I mean by a “skeleton wireframe!” What a wonderful, generous resource. Go ahead and download any.

You know the drill:

• Start with a ControlNetLoader node and load the downloaded model.
• Use a LoadImage node to load the posed “skeleton” downloaded.
• Wire these up to up to a ControlNetApply node.
• The rest of the flow is the typical SDXL Base workflow...

Install ComfyUI-OpenPose-Editor

• To create custom poses, downloaded the Custom Node ComfyUI-OpenPose-Editor by space-nuko.
• Unzip to the custom_nodes folder.
• Re-start ComfyUI.

No-Code Workflow

• Replace the LoadImage node with the node called Nui.OpenPoseEditor.
• Click the Open Editor button and in the popup editor, draw your pose(s).
• Close the popup editor, and run the flow!

Here are two poses I created myself. It is a challenge to get proportions right, and shame about the odd guns... but as an illustration, you can see how well ControlNet applies multiple poses to create a dynamic image.

The author of this custom node perhaps has a much larger monitor than I do! The bottom row of buttons was off the screen and impossible to reach!

• My solution is to edit the file custom_node\OpenPose-Editor\js\openpose.js.
• At line 91 or thereabouts, edit the line that specifies the hard-coded height of the dialog box, adjusting the value to one that suits you.
• Save and re-start ComfyUI.

        const height = 800; // originally, const height = 1000;

A final note: You may wish to try another Custom Node set, Fannovel16/comfyui_controlnet_aux v1.1 or the archived v1, which has a node called OpenposePreprocessor. I think this node uses OpenPose to “extract” “skeleton wireframe” from a reference image but I have not tried it...