A little over a year ago I made a similar clip with the same footage. It took me about a day as I was motion tracking, facial mocapping, blender overlaying and using my old TokyoJab method on each element of the scene (head, shirt, hands, backdrop).

This new one took about 40 minutes in total, 20 minutes of maxing out the card with Wan Vace and a few minutes repairing the mouth with LivePortrait as the direct output from Comfy/Wan wasn't strong enough.

The new one is obviously better. Especially because of the physics on the hair and clothes.

All locally made on an RTX3090.

Wan Vace is insane. This is the amount of control I always hoped for. Makes my method utterly obsolete. Loving it.

I started experimenting after watching this tutorial.. Well worth a look.

This is a another training diary for different captioning methods and training with Flux.

Here I am using a public domain tarot card dataset, and experimenting how different captions affect the style of the output model.

The Captioning Types

With this exploration I tested 6 different captioning types. They start from number 3 due to my dataset setup. Apologies for any confusion.

• 3-triggerword
• 4-notriggerword
• 5-toriigatebrief
• 6-toriigatedetailed
• 7-funnycaptions
• 8-funnycaptionshort

Let's cover each one, what the captioning is like, and the results from it. After that, we will go over some comparisons. Lots of images coming up! Each model is also available in the links above.

Original Dataset

I used the 1920 Raider Waite Tarot deck dataset by user multimodalart on Huggingface.

The fantastic art is created by Pamela Colman Smith.

https://huggingface.co/datasets/multimodalart/1920-raider-waite-tarot-public-domain

The individual datasets are included in each model under the Training Data zip-file you can download from the model.

Cleaning up the dataset

I spent a couple of hours cleaning up the dataset. As I wanted to make an art style, and not a card generator, I didn't want any of the card elements included. So the first step was to remove any tarot card frames, borders, text and artist signature.

Training data clean up, removing the text and card layout

I also removed any text or symbols I could find, to keep the data as clean as possible.

Note the artists signature in the bottom right of the Ace of Cups image. The artist did a great job hiding the signature in interesting ways in many images. I don't think I even found it in "The Fool".

Apologies for removing your signature Pamela. It's just not something I wanted the model to pick learn.

Training Settings

Each model was trained locally with the ComfyUI-FluxTrainer node-pack by Jukka Seppänen (kijai).

The different versions were each trained using the same settings.

Resolution: 512

Scheduler: cosine_with_restarts

LR Warmup Steps: 50

LR Scheduler Num Cycles: 3

Learning Rate: 7.999999999999999e-05

Optimizer: adafactor

Precision: BF16

Network Dim: 2

Network Alpha: 16

Training Steps: 1000

V3: Triggerword

This first version is using the original captions from the dataset. This includes the trigger word trtcrd.

The captions mention the printed text / title of the card, which I did not want to include. But I forgot to remove this text, so it is part of the training.

Example caption:

a trtcrd of a bearded man wearing a crown and red robes, sitting on a stone throne adorned with ram heads, holding a scepter in one hand and an orb in the other, with mountains in the background, "the emperor"

I tried generating images with this model both with and without actually using the trained trigger word.

I found no noticeable differences in using the trigger word and not.

Here are some samples using the trigger word:

Trigger word version when using the trigger word

Here are some samples without the trigger word:

Trigger word version without using the trigger word

They both look about the same to me. I can't say that one method of prompting gives a better result.

Example prompt:

An old trtcrd illustration style image with simple lineart, with clear colors and scraggly rough lines, historical colored lineart drawing of a An ethereal archway of crystalline spires and delicate filigree radiates an auroral glow amidst a maelstrom of soft, iridescent clouds that pulse with an ethereal heartbeat, set against a backdrop of gradated hues of rose and lavender dissolving into the warm, golden light of a rising solstice sun. Surrounding the celestial archway are an assortment of antique astrolabes, worn tomes bound in supple leather, and delicate, gemstone-tipped pendulums suspended from delicate filaments of silver thread, all reflecting the soft, lunar light that dances across the scene.

The only difference in the two types is including the word trtcrd or not in the prompt.

V4: No Triggerword

This second model is trained without the trigger word, but using the same captions as the original.

Example caption:

a figure in red robes with an infinity symbol above their head, standing at a table with a cup, wand, sword, and pentacle, one hand pointing to the sky and the other to the ground, "the magician"

Sample images without any trigger word in the prompt:

Sample images of the model trained without trigger words

Something I noticed with this version is that it generally makes worse humans. There are a lot of body horror limb merging. I really doubt it had anything to do with the captioning type, I think it was just the randomness of model training and that the final checkpoint happened to be trained to a point where the bodies were often distorted.

It also has a smoother feel to it than the first style.

V5: Toriigate - Brief Captioning

For this I used the excellent Toriigate captioning model. It has a couple of different settings for caption length, and here I used the BRIEF setting.

Links:

Toriigate Batch Captioning Script

Toriigate Gradio UI

Original model: Minthy/ToriiGate-v0.3

I think Toriigate is a fantastic model. It outputs very strong results right out of the box, and has both SFW and not SFW capabilities.

But the key aspect of the model is that you can include an input to the model, and it will use information there for it's captioning. It doesn't mean that you can ask it questions and it will answer you. It's not there for interrogating the image. Its there to guide the caption.

Example caption:

A man with a long white beard and mustache sits on a throne. He wears a red robe with gold trim and green armor. A golden crown sits atop his head. In his right hand, he holds a sword, and in his left, a cup. An ankh symbol rests on the throne beside him. The background is a solid red.

If there is a name, or a word you want the model to include, or information that the model doesn't have, such as if you have created a new type of creature or object, you can include this information, and the model will try to incorporate it.

I did not actually utilize this functionality for this captioning. This is most useful when introducing new and unique concepts that the model doesn't know about.

For me, this model hits different than any other and I strongly advice you to try it out.

Sample outputs using the Brief captioning method:

Sample images using the Toriigate BRIEF captioning method

Example prompt:

An old illustration style image with simple lineart, with clear colors and scraggly rough lines, historical colored lineart drawing of a A majestic, winged serpent rises from the depths of a smoking, turquoise lava pool, encircled by a wreath of delicate, crystal flowers that refract the fiery, molten hues into a kaleidoscope of prismatic colors, as it tosses its sinuous head back and forth in a hypnotic dance, its eyes gleaming with an inner, emerald light, its scaly skin shifting between shifting iridescent blues and gold, its long, serpent body coiled and uncoiled with fluid, organic grace, surrounded by a halo of gentle, shimmering mist that casts an ethereal glow on the lava's molten surface, where glistening, obsidian pools appear to reflect the serpent's shimmering, crystalline beauty.

Side Quest: How to use trained data from Flux LoRAs

If trigger words are not working in Flux, how do you get the data from the model? Just loading the model does not always give you the results you want. Not when you're training a style like this.

The trick here is to figure out what Flux ACTUALLY learned from your images. It doesn't care too much about your training captions. It feels like it has an internal captioning tool which compares your images to its existing knowledge, and assigns captions based on that.

Possibly, it just uses its vast library of visual knowledge and packs the information in similar embeddings / vectors as the most similar knowledge it already has.

But once you start thinking about it this way, you'll have an easier time to actually figure out the trigger words for your trained model.

To reiterate, these models are not trained with a trigger word, but you need to get access to your trained data by using words that Flux associates with the concepts you taught it in your training.

Sample outputs looking for the learned associated words:

Sample outputs looking for the learned associated words

I started out by using:

An illustration style image of

This gave me some kind of direction, but it has not yet captured the style. You can see this in the images of the top row. They all have some part of the aesthetics, but certainly not the visual look.

I extended this prefix to:

An illustration style image with simple clean lineart, clear colors, historical colored lineart drawing of a

Now we are starting to cook. This is used in the images in the bottom row. We are getting much more of our training data coming through. But the results are a bit too smooth. So let's change the simple clean lineart part of the prompt out.

Let's try this:

An old illustration style image with simple lineart, with clear colors and scraggly rough lines, historical colored lineart drawing of a

And now I think we have found most of the training. This is the prompt I used for most of the other output examples.

The key here is to try to describe your style in a way that is as simple as you can, while being clear and descriptive.

If you take away anything from this article, let it be this.

V6: Toriigate - Detailed Captioning

Similar to the previous model, I used the Toriigate model here, but I tried the DETAILED captioning settings. This is a mode you choose when using the model.

Sample caption:

The image depicts a solitary figure standing against a plain, muted green background. The figure is a tall, gaunt man with a long, flowing beard and hair, both of which are predominantly white. He is dressed in a simple, flowing robe that reaches down to his ankles, with wide sleeves that hang loosely at his sides. The robe is primarily a light beige color, with darker shading along the folds and creases, giving it a textured appearance. The man's pose is upright and still, with his arms held close to his body. One of his hands is raised, holding a lantern that emits a soft, warm glow. The lantern is simple in design, with a black base and a metal frame supporting a glass cover. The light from the lantern casts a gentle, circular shadow on the ground beneath the man's feet. The man's face is partially obscured by his long, flowing beard, which covers much of his lower face. His eyes are closed, and his expression is serene and contemplative. The overall impression is one of quiet reflection and introspection. The background is minimalistic, consisting solely of a solid green color with no additional objects or scenery. This lack of detail draws the viewer's focus entirely to the man and his actions. The image has a calm, almost meditative atmosphere, enhanced by the man's peaceful demeanor and the soft glow of the lantern. The muted color palette and simple composition contribute to a sense of tranquility and introspective solitude.

This is the caption for ONE image. It can get quite expressive and lengthy.

Note: We trained with the setting t5xxl_max_token_length of 512. The above caption is ~300 tokens. You can check it using the OpenAI Tokenizer website, or using a tokenizer node I added to my node pack.

OpenAI's Tokenizer

OpenAI's Tokenizer

Tiktoken Tokenizer from mnemic's node pack

Tiktoken Tokenizer from mnemic's node pack

Sample outputs using v6:

Sample outputs using Toriigate Captioning DETAILED mode

Quite expressive and fun, but no real improvement over the BRIEF caption type. I think the results of the brief captions were in general more clean.

Sidenote: The bottom center image is what happens when a dragon eat too much burrito.

V7: Funnycaptions

"What the hell is funnycaptions? That's not a thing!" You might say to yourself.

You are right. This was just a stupid idea I had. I was thinking "Wouldn't it be funny to caption each image with a weird funny interpretation, as if it was a joke, to see if the model would pick up on this behavior and create funnier interpretations of the input prompt?"

I believe I used an LLM to create a joking caption for each image. I think I used OpenAI's API using my GPT Captioning Tool. I also spent a bit of time modernizing the code and tool to be more useful. It now supports local files uploading and many more options.

Unfortunately I didn't write down the prompt I used for the captions.

Example Caption:

A figure dangles upside down from a bright red cross, striking a pose more suited for a yoga class than any traditional martyrdom. Clad in a flowing green robe and bright red tights, this character looks less like they’re suffering and more like they’re auditioning for a role in a quirky circus. A golden halo, clearly making a statement about self-care, crowns their head, radiating rays of pure whimsy. The background is a muted beige, making the vibrant colors pop as if they're caught in a fashion faux pas competition.

A figure dangles upside down from a bright red cross, striking a pose more suited for a yoga class than any traditional martyrdom. Clad in a flowing green robe and bright red tights, this character looks less like they’re suffering and more like they’re auditioning for a role in a quirky circus. A golden halo, clearly making a statement about self-care, crowns their head, radiating rays of pure whimsy. The background is a muted beige, making the vibrant colors pop as if they're caught in a fashion faux pas competition.

It's quite wordy. Let's look at the result:

It looks good. But it's not funny. So experiment failed I guess? At least I got a few hundred images out of it.

But what if the problem was that the caption was too complex, or that the jokes in the caption was not actually good? I just automatically processed them all without much care to the quality.

V8: Funnycaptionshort

Just in case the jokes weren't funny enough in the first version, I decided to give it one more go, but with more curated jokes. I decided to explain the task to Grok, and ask it to create jokey captions for it.

It went alright, but it would quickly and often get derailed and the quality would get worse. It would also reuse the same descriptory jokes over and over. A lot of frustration, restarts and hours later, I had a decent start. A start...

The next step was to fix and manually rewrite 70% of each caption, and add a more modern/funny/satirical twist to it.

Example caption:

A smug influencer in a white robe, crowned with a floral wreath, poses for her latest TikTok video while she force-feeds a large bearded orange cat, They are standing out on the countryside in front of a yellow background.

A smug influencer in a white robe, crowned with a floral wreath, poses for her latest TikTok video while she force-feeds a large bearded orange cat, They are standing out on the countryside in front of a yellow background.

The goal was to have something funny and short, while still describing the key elements of the image. Fortunately the dataset was only of 78 images. But this was still hours of captioning.

Sample Results:

Sample results from the funnycaption method, where each image is described using a funny caption

Interesting results, but nothing more funny about them.

Conclusion? Funny captioning is not a thing. Now we know.

Conclusions & Learnings

It's all about the prompting. Flux doesn't learn better or worse from any input captions. I still don't know for sure that they even have a small impact. From my testing it's still no, with my training setup.

The key takeaway is that you need to experiment with the actual learned trigger word from the model. Try to describe the outputs with words like traditional illustration or lineart if those are applicable to your trained style.

Let's take a look at some comparisons.

Comparison Grids

I used my XY Grid Maker tool to create the sample images above and below.

https://github.com/MNeMoNiCuZ/XYGridMaker/

It is a bit rough, and you need to go in and edit the script to choose the number of columns, labels and other settings. I plan to make an optional GUI for it, and allow for more user-friendly settings, such as swapping the axis, having more metadata accessible etc.

The images are 60k pixels in height and up to 80mb each. You will want to zoom in and view on a large monitor. Each individual image is 1080p vertical.

All images in one (resized down)

All images without resizing - part 1

All images without resizing - part 2

All images without resizing - part 3

A sample of the samples:

A sample of samples of the different captioning methods

Use the links above to see the full size 60k images.

My Other Training Articles

Below are some other training diaries in a similar style.

Flux World Morph Wool Style part 1

Flux World Morph Wool Style part 2

Flux Character Captioning Differences

Flux Character Training From 1 Image

Flux Font Training

And some other links you may find interesting:

Datasets / Training Data on CivitAI

Dataset Creation with: Bing, ChatGPT, OpenAI API

Website: https://reachomk.github.io/gen2seg/

HuggingFace Demo: https://huggingface.co/spaces/reachomk/gen2seg

What do you guys think? Does it work on images you guys tried?

What is this?
This workflow turns any video into a seamless loop using Wan2.1 VACE. Of course, you could also hook this up with Wan T2V for some fun results.

It's a classic trick—creating a smooth transition by interpolating between the final and initial frames of the video—but unlike older methods like FLF2V, this one lets you feed multiple frames from both ends into the model. This seems to give the AI a better grasp of motion flow, resulting in more natural transitions.

It also tries something experimental: using Qwen2.5 VL to generate a prompt or storyline based on a frame from the beginning and the end of the video.

Workflow: Loop Anything with Wan2.1 VACE

Side Note:
I thought this could be used to transition between two entirely different videos smoothly, but VACE struggles when the clips are too different. Still, if anyone wants to try pushing that idea further, I'd love to see what you come up with.

A year ago, a message on this subreddit was posted introducing an advanced image upscale method called SILVI v2. The method left many (myself included) impressed and sent me on a search for ways to improve on it, using a modified approach and more up to date tools. A year later, I am happy to share my results here and - hopefully - revive the discussion. Also, answer more general questions that are still important to many, judging by the questions people continue to post here.

Can we enhance images with open source, locally-generating tools with the quality on par with commercial online services like Magnific of Leonardo, or even better? Can it be done with a consumer-grade GPU and which processing times can be expected? What is the most basic, bare bone approach to upscaling and enhancing images locally? My article on CivitAI has some answers, and more. Your comments will be appreciated.

https://preview.redd.it/hftivcbc4q2f1.jpg?width=2876&format=pjpg&auto=webp&s=b0e5f801cfcf1ebc6e2a3e55f72ba192993730b3

I created this simple 3D render in Blender! I experimented with the Grease Pencil Line Art modifier. However, I didn’t quite achieve the result I was aiming for. Is there a way to convert my 3D render into 2D vector-style line art—something that resembles hand-drawn animation—using only my local computer hardware?

After some time of testing and research, I finally finished this article on LayerDiffuse, a method to generate images with built-in transparency (RGBA) directly from the prompt, no background removal needed.

I explain a bit how it works at a technical level (latent transparency, transparent VAE, LoRA guidance), and also compare it to traditional background removal so you know when to use each one. I’ve included lots of real examples like product visuals, UI icons, illustrations, and sprite-style game assets. There’s also a section with prompt tips to get clean edges.

It’s been a lot of work but I’m happy with how it turned out. I hope you find it useful or interesting!

Any feedback is welcome 🙂

👉 https://runware.ai/blog/introducing-layerdiffuse-generate-images-with-built-in-transparency-in-one-step

🛸A vision of the future rooted in timeless beauty. This humanoid-alien fashion figure stands before a silent Fontana di Trevi, wrapped in clouds of smoke colored like the Italian flag — a cinematic collision between heritage and post-human elegance.

🇮🇹What could Italian identity look like in 2050? When style, technology and mythology converge, the result might not just be unexpected… It might be spectacular.

🔴Visual concept generated with Imagen 4 by Gemini; prompt design by Spectacular Vintage Ultra-realistic detailing, fashion-forward design, and an urban-futurist mood.