1用户1827
NoobAI-XL User Manual (WIP)
输入“/”快速插入内容
NoobAI-XL User Manual (WIP)
NoobAI-XL User Manual (WIP)
用户1827用户18272025年1月7日修改
2024年12月2日创建
22295
38170
4
2
1.
Introduction
This document provides a comprehensive, complete and up-to-date introduction to the NoobAI-XL model.
Please note that due to the dynamic updating and maintenance difficulties of a large amount of information, this document may have errors or be incomplete.
1.1
Basic Introduction
NoobAI-XL is a Text-to-Image diffusion model developed by Laxhar Dream Lab and sponsored by Lanyun. The model certificate is inherited from fair-ai-public-license-1. 0-sd and has additional restrictions (see NoobAI-XL model certificate). The model is based on SDXL model architecture. Its base model is Illustrious-xl-early-release-v0 model. It has been trained on the complete Danbooru and e621 datasets (about 13 million images) for a large number of rounds, with rich knowledge and excellent performance.
1.2
Summarize
NoobAI-XL has a huge amount of knowledge, which can restore the styles of tens of thousands of 2D characters and artists, recognize a large number of special concepts in 2D, and has rich furry knowledge.
NoobAI-XL provides both noise prediction (or say epsilon prediction) and V-prediction versions. In short, the noise prediction version generates more diverse and creative images, while the V-prediction version follows the prompts more and generates images with a wider color gamut and stronger lighting.
NoobAI-XL has an increasingly rich ecosystem community support, including various LoRA, ControlNet , IP-Adapter and so on.
NoobAI-XL includes a series of models, mainly noise prediction and V prediction, which will be described in detail later .
2.
Quick Start
2.1
Model Download
Site | Link |
CivitAI | |
LiblibAI | |
Huggingface |
2.2
Model Loading
NoobAI-XL models are divided into two categories: noise prediction ( epsilon prediction , or abbreviated as eps-pred ) models and V prediction ( v-prediction , or abbreviated as v-pred ) models. Models with the words "eps", "epsilon-pred", or "eps-pred" in their names are noise prediction models, which are not much different from other models. If you use them, you can skip this section directly. Models with the words "v" or "v-pred" in their names are V prediction models, which are different from most conventional models. Please read the installation guide in this section carefully! The principle of the V prediction model can be found in this article .
2.2.1
Loading of V-prediction Models
V prediction is a relatively rare Model Training technique. Models trained using this technique are called V prediction models. Compared with noise prediction, V prediction models are known for their higher obedience to hints, more comprehensive color gamut, and stronger light and shadow, represented by NovelAI Diffusion V3 and COSXL . Due to its late appearance and fewer models of this type, some mainstream graphic projects and UIs do not directly support it. Therefore, if you plan to use V prediction models, you need some additional operations. This section will introduce its specific usage. If you encounter any difficulties during use, you can also directly contact any model author for help.
A. Use in forge or reForge
Forge and reForge are two ai image generation UIs developed by lllyasviel and Panchovix respectively, both are extended versions of WebUI. Their main branch support the V prediction model, and the operation mode is almost the same as WebUI, so they are recommended. If you have installed one of them, you only need to run git pull update in its installation directory and restart it; if you have not installed it, you can refer to the online tutorial for installation and use.
B. Use in ComfyUI
ComfyUI is a image generation UI developed by comfyanonymous , allowing users to freely manipulate nodes, named for its flexibility and professionalism. Using the V prediction model only requires adding additional nodes.
C. Use in WebUI
WebUI is a project stable-diffusion-webui developed by AUTOMATIC1111 . Currently, the main branch of WebUI, which is the main branch, does not support the V prediction model and needs to switch to dev. Please note that this method is unstable and may have bugs. Improper use can even cause irreversible damage to WebUI. Therefore, please backup your WebUI in advance. The specific method is as follows:
1.
If you haven't installed WebUI yet, please refer to the online tutorial to install it.
2.
Open the Console or Terminal in the installation directory of your stable-diffusion-webui.
3.
Enter the command git checkout dev and press Enter.
4.
Restart WebUI.
D. Use in Diffusers
Diffusers is a Python diffusion model dedicated library. This usage requires users to have a certain code foundation, and is recommended for developers and researchers to use. Code example:
代码块
import torch
from diffusers import StableDiffusionXLPipeline, EulerDiscreteScheduler
ckpt_path = "/path/to/model.safetensors"
pipe = StableDiffusionXLPipeline.from_single_file(
ckpt_path,
use_safetensors=True,
torch_dtype=torch.float16,
)
scheduler_args = {"prediction_type": "v_prediction", "rescale_betas_zero_snr": True}
pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, **scheduler_args)
pipe.enable_xformers_memory_efficient_attention()
pipe = pipe.to("cuda")
prompt = """masterpiece, best quality, john_kafka, nixeu, quasarcake, chromatic aberration, film grain, horror \(theme\), limited palette, x-shaped pupils, high contrast, color contrast, cold colors, arlecchino \(genshin impact\), black theme, gritty, graphite \(medium\)"""
negative_prompt = "nsfw, worst quality, old, early, low quality, lowres, signature, username, logo, bad hands, mutated hands, mammal, anthro, furry, ambiguous form, feral, semi-anthro"
image = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
width=832,
height=1216,
num_inference_steps=28,
guidance_scale=5,
generator=torch.Generator().manual_seed(42),
).images[0]
image.save("output.png")