Three Ways to Generate AI Art Using Intel® Arc™ GPUs

Generative AI running in inside of GIMP on the left¹, within the A1111 webui in the middle², and SD.Next webui on the right³.

Generative AI art is gaining in popularity especially as tools now allow for higher control, iteration, and custom data sets. As well artists and creators seek local control, where they can fine tune their work, and make use of their own local hardware.  This leaves many people wondering, is my system up to the task: When it comes to the latest Intel hardware the answer for certain is YES. (see article on Stable Diffusion Optimize for Intel Silicon by Tom’s Hardware)

Below are three emerging solutions for doing Stable Diffusion Generative AI art using Intel Arc GPUs on a Windows laptop or PC. While these are not the only solutions, these are accessible and feature rich, able to support interests from the AI art-curious to AI code warriors. Follow the link below to learn more and get installation instructions.

• AI Art Inside an Image Editor via GIMP Plug-In: Ideal for artists looking to blend use of AI and traditional digital art techniques.
• AI Art using the A1111 WebUI on Windows: Power and ease of the A1111 WebUI with the performance OpenVINO provides.
• AI Art using SDXL running in SD.Next WebUI: Full support of the latest Stable Diffusion has to offer running in Windows or Linux

Note, it should be clear to anyone diving into the generative AI art space for the first time, that it is highly experimental, and moving very fast with new tools and methods continually evolving.  This space is far from settled with software often requiring users install Python and GIT, where installing and updating software may be done from a command line interface.

Inside an Image Editor

GIMP Image Editor with OpenVINO plug-in ⁴

Intel OpenVINO GIMP Experimental Plug-in:

Many of you may have seen fantastic videos of AI being used in image editors, allowing for a designer to lasso a section of an image, describe what should fill that area and nearly instantly get a result back. This capability is also available via the free GIMP image editor and can run on Intel CPU, integrated graphics, or Intel Arc discrete GPUs through an experimental Plug-In developed by the OpenVINO team at Intel.

Once installed users can select the OpenVINO options from the Layer menu giving them access to a variety of generative AI tools.  By adding AI capabilities directly inside of an image editor, artists are given access to AI generation alongside familiar tools and image editing techniques.

Features:

• Stable Diffusion 1.4 AND 1.5 text to image, and image to image Generation
• Supports Positive and Negative Prompts
• 512 Square, 512×768 Portrait, 768×512 landscape formats
• Stable Diffusion in-painting (lasso an area, use text to describe how to fill that area)
• Experimental Branch: Future support for ControlNet OpenPose

Benefits:

• Runs natively on Windows laptops or desktops
• Runs alongside a familiar toolset and workflow, more ideal for designers and artists
• Supports Intel CPUs, integrated graphics and GPUs including Intel Arc GPUs
• Simple install — all packages and models included in a single installer.
• Powered by OpenVINO, thus the code is optimized to deliver great performance on the target hardware.
• Runs locally, no cloud network or service needed
• Runs within free software, no subscription service needed

Instructions: https://github.com/intel/openvino-ai-plugins-gimp

Animated GIF demonstrating the in-painting feature from the the OpenVINO^TM GIMP Experimental Plug-in, allowing sections of an image to be generatively filled using text prompts. See citation at the end of the article for prompt sequence.

Animation of each phase of in-painting within the GIMP application⁵. Source: Bob Duffy, Intel employee

Using the A1111 WebUI in Windows:

A1111 webui running the ‘Accelerate with OpenVINO’ script, set to use the system’s discrete GPU, and running the custom Realistic Vision 5.1 model, generating the image of an Alchemist on the right 6. Source: Bob Duffy, Intel employee

OpenVINO WebUI Preview

The A1111 WebUI is potentially the most popular and widely lauded tool for running Stable Diffusion. The OpenVINO team has provided a fork of this popular tool, with support for using the OpenVINO framework, which is an open platform for optimizes AI inferencing to run across a variety of hardware include CPUs, GPUs and NPUs. This fork enables this widely popular tool and ecosystem to run on Intel Arc GPUs with all features and capabilities you’d expect from the A1111 WebUI.

Once installed the OpenVINO script can be selected in the WebUI, where users can choose a CPU or GPU on their system for generating the image. The first generated action takes longer than usual but once done subsequent iterations including batching multiple images will run as expected. This solution is well suited for users who want more control and options, thus do not mind having to adjust various settings, download specific models, and adjust configurations to get the result they are after, while also wanting the convenience of staying withing the Windows operating system.

Features:

• Runs in A1111, a very popular webui for Stable Diffusion
• Support for AI image generation using your CPU, iGPU, dGPU or NPU
• Stable Diffusion 1.5 and 2.1 models for text to image and image to image generation
• Support for popular custom models i.e. checkpoints from Civitai
• Stable Diffusion In-Painting (use text to describe how to fill an area of the image)
• Extras Tab for image upscaling
• Supports positive and negative prompts with weighting
• Supports for variable width and height settings
  (note performance issues sighted with some higher resolution images)
• Support for SDXL High Res image generation
• Support for ControlNet
• Support for custom LoRAs and VAEs

Benefits:

• Runs natively on Windows laptops or desktops
• Supports Intel Arc GPUs, Intel CPUs and integrated graphics
• Powered by OpenVINO, thus the code is optimized to deliver great performance on the target hardware.
• Runs locally, no cloud network or service needed
• Runs within a browser, no subscription service needed
• Tap into a larger ecosystem of custom models to better target desired output

Instructions: https://github.com/openvinotoolkit/stable-diffusion-webui/wiki/Installation-on-Intel-Silicon

Line art render of a kitchen designed in Revit* used to create generative AI Art renders using ControlNet Canny and Line Art features in A1111 to guide the output ⁷. Source: Bob Duffy, Intel employee

Using SDNEXT WebUI with SDXL support:

SD.Next WebUI running in Linux, via Intel Extensions for PyTorch (IPEX). Image source @Disty community contributor from the Intel Insiders Discord⁸

PyTorch on Windows or Linux

SDNEXT is a version of A1111 for the more advanced version of Stable Diffusion with SDXL support. This section describes a version with Intel Extensions for PyTorch (IPEX) you run on Windows or Linux.
This solution is suited for users who want the latest and greatest features without additional compile times when switching between models.

Features:

• Stable Diffusion SDXL Support for text to image and image to image generation
• Immediate support for custom models, LoRAs and extensions like ControlNet
• Supports positive and negative prompts with weighting
• Variable width and height settings beyond 1024 x 1024
• Image upscaling
• Stable Diffusion in-painting (lasso an area, use text to describe how to fill that area)
• Support for Fast LCM models
• Support for Windows and Linux

Benefits:

• Runs the most advanced version of Stable Diffusion
• Runs locally, no cloud network or service needed.
• Runs within a browser, no subscription service needed.
• Run SDXL refiners to increase the quality of output with high resolution images.
• Tap into a larger ecosystem of custom models, LoRAs and ControlNet features to better target the output of your images

Instructions: https://www.technopat.net/sosyal/konu/using-stable-diffusion-webui-with-intel-arc-gpus.2593077/

Discord Thread: https://discord.com/channels/554824368740630529/1127742927347666964/1127742927347666964

More to Come

The generative AI art space is certainly moving fast, allowing for creators and technical artist to realize a new workflow for editing and generating images.  These three methods are just scratching the surface of what is possible on Intel client hardware. We expect more exciting solutions ahead, as Intel is prioritizing AI workloads across client, edge and cloud. Stay tuned for more information as they emerge, and feel free to connect with our community of expert users on Insiders Discord or through social on @IntelGraphics.

Notices and Disclaimers

AI features may require software purchase, subscription or enablement by a software or platform provider, or may have specific configuration or compatibility requirements. Details at www.intel.com/PerformanceIndex. Results may vary.

Image Citations

1. OpenVINO AI Plug-ins for GIMP. (2023, 8, 11). Pos Prompt: “A 35 Year old cyberpunk male on a motorcycle, modern hair style, partially shaven beard. City in the background with neon lights. Fujifilm, photo realistic, detailed face, dramatic lighting 50mm lens“, Neg Prompt: “cartoon, illustration, render, cropped, monochromatic, nsfw“. Intel https://github.com/intel/openvino-ai-plugins-gimp
2. A1111 WebUI for Stable Diffusion. (2023, 7, 11). Pos Prompt: “A 35 Year old cyberpunk male on a motorcycle, modern hair style, partially shaven beard. City in the background with neon lights. Fujifilm, photo realistic, detailed face, dramatic lighting 50mm lens”, Neg Prompt: “ cartoon, illustration, render, cropped, monochromatic, nsfw“. Intel https://github.com/openvinotoolkit/stable-diffusion-webui/wiki/Installation-on-Intel-Silicon
3. Next WebUI for Stable Diffusion . (2023, 8, 17). Pos Prompt: “ A 35 Year old cyberpunk male on a motorcycle, modern hair style, partially shaven beard. City in the background with neon lights. Fujifilm, photo realistic, detailed face, dramatic lighting 50mm lens “, Neg Prompt: “ cartoon, illustration, render, cropped, monochromatic, nsfw  “ vladmatic: https://github.com/vladmandic/automatic
4. OpenVINO AI Plug-ins for GIMP. (2023, 6, 8 ). Pos Prompt: “concept art of female alchemist, in purple cloak, flowing hair, fantasy, magical orb, magical environment, DND, highly detailed” Neg Prompt: “cartoon, anime, disfigured, distorted, photograph“ . Intel https://github.com/intel/openvino-ai-plugins-gimp
5. OpenVINO AI Plug-ins for GIMP. (2023, 6, 8 ). Prompt: “close up photo of a man with beard and modern haircut, photo realistic, detailed skin, Fujifilm, 50mm”, In-painting: 1 ”city skyline”, 2 ”superhero suit”, 3 “clean shaven” 4 “skyscrapers”, 5 “skyscrapers”, 6 “superhero hair” Neg Prompt: “cartoon, anime, disfigured, distorted“ . Intel https://github.com/intel/openvino-ai-plugins-gimp
6. A1111 WebUI for Stable Diffusion. (2023, 6,). Pos Prompt: “photo of a beautiful magical female with flowing purple hair, dressed as an alchemist, wearing a decorative cloak, highly detailed, magical, night, fog, orb of light, detailed skin“, Neg Prompt: “cartoon, illustration, painting, nsfw, wine“ . Intel. https://github.com/openvinotoolkit/stable-diffusion-webui/wiki/Installation-on-Intel-Silicon
7. A1111 WebUI for Stable Diffusion. (2023, 6,). Pos Prompt: “Award winning photo of luxurious modern kitchen, high detailed, photo realism, Fujifilm, 50mm“, Neg Prompt: “ cartoon, illustration, painting “ with ControlNet – LineArt, Batch=4  https://github.com/openvinotoolkit/stable-diffusion-webui/wiki/Installation-on-Intel-Silicon
8. Next WebUI for Stable Diffusion . (2023, 6,). Pos Prompt: “photo of a beautiful magical female with flowing purple hair, dressed as an alchemist, wearing a decorative cloak, highly detailed, magical, night, fog, orb of light, detailed skin“, Neg Prompt: “NSFW, (worst quality), low quality: 1;4, lowres, blurry),(zombie, comic, sketch), (interlocking fingers, long neck, 2 girls)“ vladmatic: https://github.com/vladmandic/automatic