Vista de Lectura
New Hunyuan tweet (possible model teaser)
![]() | This tweet comes from the Official Hunyuan twitter account. They're hinting at what's probably a new model, but it doesn't seem to be i2v. Based on this I think it's most likely a fully open source t2i model. We'll have to wait and see [link] [comments] |
We don't know!
![]() | submitted by /u/the_bollo [link] [comments] |
Hey everyone! Let's revisit SD 1.5! test tri-structured prompts
![]() | Since I have weaker hardware, I never really stopped using it, and sometimes I come across techniques I’ve never seen before. For example, I recently found a new model (Contra Base) and decided to run it through my standard set of prompts for comparison (I have a special spreadsheet for that). The author mentioned something called a tri-structured prompt for image generation, which they used during training. I didn’t recall seeing this approach before, so I decided to check it out. Here’s a link to the article. In short:
I'll assume you roughly understand this logic and won't dive into the finer details.
However, having everything in a single text field is much more convenient - both for reading and editing, especially when using multiple Let’s play around and see what results we get. First, we need to set up the basics. Let's start with the default pipeline. For some reason, I wanted to generate a clown in an unusual environment, so I wrote this: Not very readable, right? And we still need to write the negative prompt, as described in the article. A more readable format could look something like this: After some thought, I decided the prompt format should be more intuitive and visually clear while still being easy to process. So I restructured it like this: The first line is the main prompt, with clarifying details listed below as keyword branches. I added underscores for better readability. They don’t affect generation significantly, but I’ll remove them before processing. For comparison, of course, I decided to test what would be generated without BREAK commands to see how much of an impact they have. Let's begin! The resolution I want to make more than 768 points, which will give us repetitions and duplications without additional “dodging” of the model... As expected! One noticeable difference: the To keep it simple, I added:
Much better results! Not perfect, but definitely more interesting. Then I remembered that in SD 1.5, I could use an external encoder instead of the one from the loaded model. Flux works well for this. Using this one, I got these results: What conclusions can be drawn about using this prompting method? "It's not so simple." I think no one would argue with that. Some things improved, while others were lost. By the way, my clown just kept juggling, no matter how much I tweaked the prompt. But I didn’t stress over it too much. One key takeaway: increasing the number of “layers” indefinitely is a bad idea. The more nested branches there are, the less weight each one carries in the overall prompt, which leads to detail loss. So in my opinion, 3 to 4 clarifications are the optimal amount. a smaller number of branches gives a clearer following Now, let’s try other prompt variations for better comparison. While working with this, I discovered that Kohya Deep Shrink sometimes swaps colors - turning the dress green and the car red. It seems to depend on the final image resolution. Different samplers also handle this prompt differently (who would’ve thought, right?). Another interesting detail: I clearly specified that the dress should be fluffy with large knots. In the general prompt, this token is considered, but since there are many layers, its weight is diluted, resulting in just a plain red dress. Also, the base prompt tends to generate a doll-like figure, while the branches produce a more realistic image. Let’s try another one: No cats here. And no painterly effect from the branches. My guess? Since the painting-style tokens are placed in just one out of five branches, their total weight is only one-fifth of the overall prompt. Let’s test this by increasing the weight of that branch. With a small boost, no visible changes. But if we overdo it (e.g., 1.6), abstract painting tokens dominate, making the image completely off-topic. weight of painting brunch is 1.55 Conclusion: this method is not suitable for defining overall art style. And finally, let’s wrap up with a cat holding a sign. Of course, SD 1.5 won’t magically generate perfect text, but splitting into branches does improve results. Final thoughtsIn my opinion, this prompting technique can be useful for refining a few specific elements, but it doesn't work as the original article described. More branches = less influence per branch = loss of control. Right now, I think there are better ways to add complexity and detail to SD 1.5 models. For example, ELLA handles more intricate prompts much better. To test this, I used the same prompts with ELLA and the same seed values: If anyone wants to experiment, I’ve uploaded my setup here. Let me know your thoughts or if you see any flaws in my approach. Happy generating! 🎨🚀 [link] [comments] |
What is the currently the best SDXL model for realistic scenary in your opinion?
I am not interested in portrait and human, i use it for urban photos ,landscape, buildings... Etc. My pic is juggernaut, what about you?
[link] [comments]
Flux Tech Details by Robin Rombach (CEO, Black Forest Labs)
https://www.youtube.com/watch?v=nrKKLJXBSw0
I made a summary, I can't digest it myself.
FLUX: Flow Matching for Content Creation at Scale - Detailed Summary (Formatted)
Speaker:
Robin Rombach (Creator of Latent Diffusion, CEO of Black Forest Labs)
Lecture Topic:
Flux - Content Creation Model using Flow Matching
Focus of Lecture:
Detailed methodology of Flux, comparison of flow matching vs. diffusion models, and future directions in generative modeling.
Context:
TUM AI Lecture Series
Key Highlights:
- Latent Diffusion Influence: Rombach emphasized the impact of Latent Diffusion (15,000+ citations) and its role in establishing text-to-image generation as a standard.
- Dual Impact: Rombach's contributions span both academia and industry, notably including his work on Stable Diffusion at Stability AI.
Flux: Methodology and Foundations
- Developed by: Black Forest Labs
- Core Techniques: Flow Matching and Distillation for efficient content creation.
- Latent Generative Modeling Paradigm:
- Motivation: Separates perceptually relevant information into a lower-dimensional space.
- Benefit: Improves computational efficiency and simplifies the generative task.
- Contrast: Compared to end-to-end learning and auto-regressive latent models (e.g., Gemini 2 image generation).
- Flux Architecture (Two-Stage):
- Adversarial Autoencoder:
- Function: Compresses images into latent space.
- Key Feature: Removes imperceptible details and separates texture from structure.
- Addresses: "Getting lost in details" issue of likelihood-based models.
- Advantage: Adversarial component ensures sharper reconstructions than standard autoencoders.
- Flow Matching based Generative Model (in Latent Space):
- Technique: Rectified Flow Matching.
- Goal: Transforms noise samples (normal distribution) into complex image samples.
- Adversarial Autoencoder:
Flux's Flow Matching Implementation:
- Simplified Training: Direct interpolation between data and noise samples.
- Benefit: Concise loss function and implementation.
- Optimized Time-Step Sampling: Log-normal distribution for time-steps (t).
- Down-weights: Trivial time steps (t=0, t=1).
- Focuses Computation: On informative noise levels.
- Resolution-Aware Training & Inference:
- Adaptation: Adjusts noise schedules and sampling steps based on image dimensionality.
- Improvement: Enhanced high-resolution generation.
- Addresses Limitation: Suboptimal uniform Euler step sampling for varying resolutions.
Architectural Enhancements in Flux:
- Parallel Attention (Transformer Blocks):
- Inspiration: Vision Transformers.
- Benefit: Hardware efficiency via fused attention and MLP operations (single matrix multiplication).
- RoPE Embeddings (Relative Positional Embeddings):
- Advantage: Flexibility across different aspect ratios and resolutions.
- Impact: Improved generalization.
Flux Model Variants & Distillation:
- Flux Pro: Proprietary API model.
- Flux Dev: Open-weights, distilled.
- Flux Schnell: Open-source, 4-step distilled.
- Differentiation: Trade-offs between quality and efficiency.
- Adversarial Distillation for Acceleration:
- Technique: Distills pre-trained diffusion model (teacher) into faster student model.
- Loss Function: Adversarial Loss.
- Latent Adversarial Diffusion Distillation: Operates in latent space, avoiding pixel-space decoding.
- Benefits: Scalability to higher resolutions, retains teacher model flexibility.
- Addresses: Quality-diversity trade-off, potentially improving visual quality.
Applications & Future Directions:
- Practical Applications:
- Image Inpainting (Flux Fill)
- Iterative Image Enlargement
- Scene Composition
- Retexturing (Depth Maps, etc.)
- Image Variation (Flux Redux)
- Future Research:
- Zero-Shot Personalization & Text-Based Editing (Customization)
- Streaming & Controllable Video Generation
- Interactive 3D Content Creation
Black Forest Labs - Startup Learnings:
- Critical Importance of Model Scaling: For real-world deployment.
- Emphasis on: Robust Distillation Techniques and Efficient Parallelization (ZeRO, FSDP).
- Evaluation Shift: Application-specific performance and user preference are prioritized over traditional metrics (FID).
- Methodological Simplicity: Key for practical scalability and debugging.
Conclusion:
- Flux represents a significant advancement in content creation through efficient flow matching and distillation techniques.
- Future research directions promise even more powerful and versatile generative models.
- Black Forest Labs emphasizes practical scalability and user-centric evaluation in their development process.
[link] [comments]
Civitai all down
![]() | submitted by /u/kjbbbreddd [link] [comments] |
Zluda, NVidia to AMD takes more Vram?
So I recently got a 7900xtx and figured jumping from (rtx2070)8gb to 24gb Vram would be great but im finding im even more restricted when it comes to resolutions now.
I was wondering if this is normal for the AMD windows experience or if i did something wrong somewhere.
Also It''d be great to know if linux has the same issue.
For example trying to get 1600x1600 gets me: Tried to allocate 47.69 GiB. GPU
Using /likelovewant/'s forge guide for the AMD install since it was the only one that seems to work for me on windows.
[link] [comments]
I wanted to see how far I could push Pony diffusion V6
![]() | I used pony diffusion V6 base model with illumination Lora I wanted to see how far I could push the image to make it look close to Flux. This was the result. [link] [comments] |
Disney/Pixar 'competition' - prize to be won
Hi all,
I'm looking for someone that can turn the picture I have included in this post of me and my friends into Disney/Pixar art style.
I would like the text 'Congratulations' written somewhere on the image, similar to you may see on a celebration card or a movie poster.
I'll leave the rest up to your imagination.
The 'prize' I am offering is £15/$20 USD for the best picture, and a further £100/$125 USD if you're willing to teach me your technique of how you create the image.
[link] [comments]
hi i need your recommendations on Checkpoint for stablediffusion
"Hi, I’d love to get your opinion on a Stable Diffusion checkpoint. I’m looking for something that works well for both realistic and fantasy images, as well as everything in between. Do you have any recommendations?
[link] [comments]
Problems with LORA in FLUX Forge
![]() | Can anyone help me figure out how to get a decent render from FLUX using Forge if I want to apply LORA to my renders? I'm running FLUX Dev fp16 GGUF (flux1-dev-Q1_0.gguf) on Forge and I have thousands of LORA downloaded but I can't use a single one of them because each and every one seems to completely overcook the model, even when I drop the weight to "0.1". The moment I remove the LORA, the images render cleanly but not with the results I want....because I can't use the LORAs. I have pretty much preserved all base defaults for FLUX Forge, and have my CFG at 1. In terms of processing power, I'm running with RTX-4090 on i9-14900 with 48GB RAM. I setup my Forge using this guide: Getting Started with Flux & Forge I've attached an image comparison of my typical experience (the reference image is based on THIS RENDER from Civit, which uses the LORA). [link] [comments] |