As promised, here's something about the plug-ins for Stable Diffusion, the only AI image generator that's open-source, and makes for a great testbed for related research because of that.
The outcome of this research are various plug-ins and modifications that allow Stable Diffusion to do a lot of interesting things. And the authors? Hooboy, you would be surprised. It turns out that very serious companies, ones you would suspect of cooking their own tech of this kind on the side even if they haven't announced it publicly, build interesting things and share it for free.
Let's start with Semantic Segmentation. If an AI image generator makes images based on text input, Semantic Segmentation scans images, identifies elements and assigns text descriptions to them. A lot of serious companies have released open-source code of their implementation: you have Nvidia, Google, Meta and even Alibaba building that stuff. It might sound kinda underwhelming if fairly useful for helping visually impaired people (for example, Facebook uses it to generate alt texts for images posts automatically), but here's the kicker: Semantic Segmentation may be used in Stable Diffusion to automatically generate masks based on text description. Want to find a hand and redraw it to be more anatomically correct? Easy. How about changing the hair color without monkeying with it in Photoshop? Also easy. So easy that some basement-dwelling chud can script it to find the clothes on a woman and draw a nude body in their place (and did, and got himself in trouble when another teenage chud uploaded photos of girls from his class to the app).
Semantic Segmentation is the core of the popular Stable Diffusion extension called aDetailer: as I mentioned before, Semantic Segmentation can recognize what a hand is, even if it's distorted, and point the generator to inpaint a better version in its place. Same goes for the faces. And that's the two things aDetailer is built to fix.
Another thing are ControlNets: plugins that allow you to nudge the generation process a particular way, be it recreating the pose of a character down to hands and fingers (or just the face orientation and expression), following the outline of a sketch and filling in the details, even maintain perspective using depth maps. And then, based on that tech, you have PhotoMaker, created by Tencent's Applied Research Center, and its improved version, IP Adapter. The capabilities are impressive, particularly if you remember that a slightly outdated gaming PC can run Stable Diffusion at a decent pace with no need for an internet access, even with the plugins.
Also, with OpenAI's video generator Sora looming on the horizon, you should know that the first AI-generated (or at least redrawn) videos were created in Stable Diffusion as well. I don't intend to go down this rabbit hole for practical reasons (I have no need for using it for that particular purpose and my video card is a bit outdated), but it was on the sweaty basement-dwelling nerds to figure out how to fine-tune the whole thing to be consistent across a whole fuckton of frames, and they did it, the crazy sonsabitches.
So laugh all you want at the ornery, wobbly Stable Diffusion producing rounded, fractal blorps and fucky hands. Even basic capabilities like inpainting and outpainting still make Midjourney jealous, and if you look at the plugins, you can imagine a good few use cases you could never wring out of the competing algorithms - and run them on your own PC for free instead of relying on centralized black boxes with a monthly fee.
Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.
✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality
Anya is LIVE right now
FREE
Free to watch • No registration required • HD streaming
