❌

Vista Normal

Hay nuevos artículos disponibles. Pincha para refrescar la página.
Ayer — 22 Noviembre 2024StableDiffusion

challenge: reduce Txx5 style drift

✇StableDiffusion
Por: ／u／Winter_unmuted
22 Noviembre 2024 at 05:19
challenge: reduce Txx5 style drift

I want to talk about something that has fallen off of this sub's radar ever since Flux dropped.

SDXL base had iffy prompt adherence, but its style decay with increasing tokens wasn't that bad. Depending on the strength of the art style prompt, you could cram a bunch of other stuff in there and still have recognizable style influence

Consider this example, made with SDXL base (vae fixed) with the simple prompt "style of ART COMP" then doing find/replace on ART and COMP:

SDXL base, \"style of ART COMP\" with find/replace as labeled

Here's the same thing, all same settings and seed but with SD3.5:

SD3.5 Large, \"style of ART COMP\" with find/replace as labeled

Even though the pure artist tag on the left is more vivid and detailed, you can see how fast the style decays into photographic. This is even worse with Flux1.d, which has very little style knowledge compared to SD3.5L.

How can we fix this? One thing I've been trying is several iterations of img2img to try and mold a simple art style into my composition, which doesn't work that well. Another thought is to generate the composition, then controlnet into SDXL for style, then img2img back into SD3.5. Each of these ideas requires a lot of work in ComfyUI and a fair bit if time, trial/error, and computing power.

So what do you think? How can we get back to the artistic flexibility of SDXL while keeping some of the strengths of SD3.5? Aside from massively training a finetune or huge LoRA, which would require a massive dataset and time (both person hours and GPU time).

Love to hear your thoughts and find out who else is working on this problem.

submitted by /u/Winter_unmuted
[link] [comments]

Flux Redux is cool but very strong, so here are a couple workflows to lower its weight

✇StableDiffusion
Por: ／u／afinalsin
21 Noviembre 2024 at 21:18

So Flux redux is very cool, but running the default workflow in Comfy felt like I was just getting variations of the same image, and I suppose that's what it's billed as. That's way too limiting though, so here's a couple twine and duct tape workflows to combine an image and an actual prompt. Workflows below, examples first.

Here is the input image I used for testing. Here is the result of using the default workflow with the prompt: "cute anime girl with massive fluffy fennec ears". It's a little cuter, but it's still mostly a variation.

Here's how my workflow did, all seed 90210:

drawing of a dragon wearing a bowler hat

Cinematic film still of a chihuahua demon

illustration of a creature that resembles a giant duck

Screenshot from 1990s anime tv show of a dragon, red background

pixar style render of a cute dragon

That input accidentally turned out to be the perfect test, since green scales and red spikes are easy to see carry over to all the prompts.

Here is the workflow to control strength.

It's basically the default workflow (I use gguf, so switch to regular nodes if you don't run them), but with a ConditioningAverage node combining the output of the Apply Style Model node in the conditioning_to slot, and the output from the FluxGuidance node in the conditioning_from slot.

From the very early testing I've done, any higher than like 0.07 and Redux will overwhelm the prompt and just produce a variation. 0.04 - 0.07 seems to be the sweet spot, although it may be fluid depending on the prompt and the input image. I haven't been the most thorough here.

And here is a bonus workflow to control the end step of Redux, like a normal controlnet or IPadapter. Flux is insane at predicting noise, so you only need a tiny amount of conditioning right at the start to affect the overall gen. This is the Pixar dragon above with Redux ending at 0.01. Still green scales, still red spikes. If you wanna use this workflow make sure the top ConditioningSetTimeStepRange node's "end" is synced with the bottom node's "start". I should have connected a primitive to both widgets but eh, I forgot.

Have fun.

submitted by /u/afinalsin
[link] [comments]
❌
❌