AI Workshop for Artists: Improving Your Art with AI

I said I would post this Wednesday but this is such a hot button issue it can't wait. If you are an artist, want to refine your drafting process, or are simply curious as to why you might want to use your own images in place of generated ones.

We're going to go step by step with your own sketches, starting from a simple background and slowly layering it up using an iterative process that uses your own sketches or compositions as a scaffolding. This guide will not cover every technique I use as anything omitted leading to the final product for this print falls far outside the scope of what I could fit here. If you are a furry artist I've met recently and would like to use it to make those things, I will be releasing a model mix once there are clean models with some sort of full opt-out or royalty based system for artists and may even train my own model in the future with artists in the community for a portion of proceeds from subscriptions.

To follow along, you will need:
The Automatic1111 interface which is open-source and freely available.
A model from Civit or the standard SD model.
Your own artwork.
Massive patience for trial and error.
Any drawing or raster editing program that works with layers and can be used for compositing.

Once you have A1111 up (there are countless guides already out there for this) you'll want to skip completely over the txt2img tab because we don't want to make commercial things from scratch except for reference or drafting. We're going to be focusing on using the Img2Img feature to translate your work into Latent AI space.

For best results, you want to use a standard aspect ratio and keep your image no larger than 1024 pixels in any direction. We'll be upscaling them later. Click the Img2Img tab in the interface and upload your sketch or whatever you're trying to modify with a prompt.

Now that we've got your image ready, we're going to start setting up the prompts. In negatives, put any stock photo, art, quality control tags, communities, or artists you want to make sure you aren't pulling from. I personally put all of the big stock photo websites in negatives, Artstation, cgsociety, deviantart, furaffinity (only necessary if you have a furry model mix).

For your actual prompt, this is something that is entirely specific to what result you want. I cannot tell you exactly what will and won't work for you because given we are actively trying to avoid similar "good" results. You can use extensions like PromptGen or Interrogator (not covered here) to extract prompts from images or doctor up a short prompt to add more detail, but these often take artist names specifically and will need to be pruned. If you're really struggling, there are built-in scripts that allow for batch generation with tweaked parameters to allow you to test things without sitting there and slightly changing the settings in between each time.

The most important setting on the img2img tab in general is the Denoise setting. AI generative models work by deconstructing noise into an image, and this is essentially telling the program how much of your original image you want to douse with noise to then be rebuilt by the model. If you simply want to touch your work up, you will be just fine with denoise of .05-.10, and will just try to smooth out any messy linework or add small detail. Set denoise too high, and you will begin to see your sketch quickly gets overwhelmed and replaced by an entirely different composition. You'll have to find the sweet spot to make the type of changes you want.

If you have a beefy card, you can take higher resolution images or sketches and composite them together, and use the AI model to clean up the seams or match color and lighting. Simply touch them up using the same process, plop them into the arrangement or composition you want, and run it through again. This method is extremely useful because you can make many small changes without masking out things, which takes lots of time.

You'll notice that the model will misplace things and mess up occasionally. This is entirely normal and to be expected. Editing programs are listed in requirements not to layer anything, but because you will be fixing things. A lot. You can also, again, take many generations with different seeds and use the results you like from each one, such as taking an image with bad eyes and good ears and vice versa to combine them.

For background elements, you can make noisemaps the same way, with white not being touched at all by the model and black being fully affected, with the ability to use greyscale color for a gradient effect.

All for now!

UPDATE: I mistakenly said 'Inpaint Mask' initially, but this is optional and is used after generating. Use Img2Img for the first step. You use 'Inpaint Upload' to guide your iterations after the first one. ControlNet is too much to cover here but is also a great resource for keeping gens consistent.