The Solar Opposites Fandom!

Base image making stage:

Step 1: Write a prompt.

I’m not good at writing prompt but it does the job:

Positive prompt:

full color photo of RLDAN, (wearing_unbuttoned_black_long-sleeve_glitter_shirt), BREAK, RLDAN, (long light blond hair), BREAK, RLDAN, (holding a martini glass), in a bar, looking to the right, (flat lighting), sharp focus, raw photo, 8k uhd, high resolution, DSLR, high quality, Fujifilm XT3

Negative prompt:

((woman, 1girl, female)), ((tattoo)), cartoon, 3D, (teeth), (mouth opened), (deformed iris), (deformed pupils), ((poorly drawn)), (extra limbs), (bad anatomy), painting, drawing, weird colors, blurry, watermark, text, high contrast, high saturation, (muscular)

About the prompt, I didn’t use any negative embeddings because sometimes they affect my trigger word; I add “muscular” in negative prompt because if I don’t, it’ll give me super buff man and it’ll look weird; “BREAK” is for separating groups of words.

Step 2: Choose a decent “raw” image

Editing the prompt and adjusting the ControlNet weights according to the results, and cherry-picking the better ones. The style of the clothing is the most important feature to imitate the original animation, so it has to be accurate enough. After cherry-picking the first okay-ish image, I reused its seed, and put this image into Lineart-realistic preprocessor to use the Lineart ControlNet model. The second cherry picked image looked more natural on lighting and background, so I saved this one for future use. I then used ADetailer (an extension) to swap face roughly, not good enough but that’s far from final.

Step 3: Upscale this image

Since SD1.5 was based on 512 resolution, my initial image size was 682 x 512, but that’s too small to make a higher quality face. Hence I used img2img+Tiled Diffusion+ControlNet Tile Resample to upscale x 2. So I get a 1364 x 1024 image to work on. I furthur upscaled it with MTXX, just 3 clicks.

Step 4: Inpaint the hair and face

I wasn’t satisfied with the hairstyle in the previous image. It doesn’t look natural and there’s no tendril near ear, so I gotta fix it. I masked the hair area and edited my prompt, and used a 0.55 denoising to get a better hair, and then used inpaint sketch to draw a blond strand near his ear, and used a 0.75 denoising to add a tendril. There were some artifacts in the image so I used Photoshop to remove those. Finally I used Adetailer again to change the face, this time it looked much more similar to the real Dan.

Post-processing(?) stage:

The Stable Diffusion part was complete. Now the Photoshop time. I changed the background (remove the wall-like thing and replace it with night view), martini glass bottom (it was really messed up), the shirt buttons (there weren’t any before), and of course the hand (I normally use OpenPose hand map to control it but this pose is rather complex, and I don’t like using depth/canny models so…) with Generative Fill. Also plenty of times of remove tool (and patch tool, color replacement tool, burn tool, liquify…yada yada yada). Then I used Neural Filters-Harmonization to match the color tone of the torso to the tone of face. Finally, I used MTXX to make some small adjustments on face and body.

And the picture is done.