Instructions to use AiArtLab/sdxs-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use AiArtLab/sdxs-1b with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("AiArtLab/sdxs-1b", dtype=torch.bfloat16, device_map="cuda") prompt = "sdxs-1b" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
1.6b
Browse files- BASELINE_TEST.md +84 -0
- dataset.py +2 -2
- dataset_alchemist.log +40 -0
- dataset_test.py +126 -0
- generate.py +124 -0
- media/result_grid.jpg +2 -2
- migrate_test.py +95 -0
- prompts.txt +10 -0
- samples/unet_384x704_0.jpg +0 -3
- samples/unet_416x704_0.jpg +0 -3
- samples/unet_448x704_0.jpg +0 -3
- samples/unet_480x704_0.jpg +0 -3
- samples/unet_512x704_0.jpg +0 -3
- samples/unet_544x704_0.jpg +0 -3
- samples/unet_576x704_0.jpg +0 -3
- samples/unet_608x704_0.jpg +0 -3
- samples/unet_640x704_0.jpg +0 -3
- samples/unet_672x704_0.jpg +0 -3
- samples/unet_704x384_0.jpg +0 -3
- samples/unet_704x416_0.jpg +0 -3
- samples/unet_704x448_0.jpg +0 -3
- samples/unet_704x480_0.jpg +0 -3
- samples/unet_704x512_0.jpg +0 -3
- samples/unet_704x544_0.jpg +0 -3
- samples/unet_704x576_0.jpg +0 -3
- samples/unet_704x608_0.jpg +0 -3
- samples/unet_704x640_0.jpg +0 -3
- samples/unet_704x672_0.jpg +0 -3
- samples/unet_704x704_0.jpg +0 -3
- sdservice.py +187 -0
- test.ipynb +2 -2
- train.py +17 -3
- train_alchemist.log +0 -0
- train_test.log +17 -0
- train_test.py +223 -0
- unet/config.json +2 -2
- unet/diffusion_pytorch_model.safetensors +2 -2
BASELINE_TEST.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Baseline: тест обучаемости UNet на одной картинке
|
| 2 |
+
|
| 3 |
+
> Дата: 2026-08-04 · Скрипт: `train_test.py` · Гипотеза: может ли UNet выучить одну картинку
|
| 4 |
+
> Веса: `unet_test/` · Сэмплы: `samples_test/` · Лог: `train_test.log`
|
| 5 |
+
|
| 6 |
+
## Конфигурация
|
| 7 |
+
|
| 8 |
+
| Параметр | Значение |
|
| 9 |
+
|---|---|
|
| 10 |
+
| Модель | UNet 1.495B (копия `unet/` → `unet_test/`) |
|
| 11 |
+
| Датасет | `datasets/test` (1 сэмпл: фото 1086×1448 → кроп 448×640) |
|
| 12 |
+
| Латент GT | (1, 32, 80, 56) |
|
| 13 |
+
| Промпт | "candid photo of young woman sitting at cafe" |
|
| 14 |
+
| Шагов обучения | **1001** (1 шаг = 1 эпоха, батч 1) |
|
| 15 |
+
| lr | 5e-5 (AdamW8bit, betas 0.9/0.997, wd 0.001, clip 1.0) |
|
| 16 |
+
| Loss | Flow matching velocity, MSE |
|
| 17 |
+
| Сэмплинг проверки | 20 шагов, CFG 4.0, `gen` из чистого шума (seed 0), `rec` из 50% шума |
|
| 18 |
+
| Сэмплы на шагах | 0, 500, 1000 |
|
| 19 |
+
|
| 20 |
+
## Метрики
|
| 21 |
+
|
| 22 |
+
### PSNR (в латентном пространстве, vs GT)
|
| 23 |
+
|
| 24 |
+
| Шаг | PSNR rec, dB | PSNR gen, dB |
|
| 25 |
+
|---|---|---|
|
| 26 |
+
| 0 | 0.43 | −3.00 |
|
| 27 |
+
| 500 | 3.97 | **12.65** |
|
| 28 |
+
| 1000 | **5.95** | **15.07** |
|
| 29 |
+
|
| 30 |
+
**Тренд:** модель радикально учит картинку. `gen` (генерация из чистого шума) растёт с −3.00 до +15.07 dB — UNet запоминает изображение и воспроизводит его по промпту из ничего. `rec` растёт медленнее (3.97 → 5.95) — при 50%-шумном старте 20 шагов денойза не успевают сойтись к GT так близко, как прямая генерация.
|
| 31 |
+
|
| 32 |
+
### Loss (velocity MSE, каждый 50-й шаг)
|
| 33 |
+
|
| 34 |
+
| Шаг | Loss | Шаг | Loss |
|
| 35 |
+
|---|---|---|---|
|
| 36 |
+
| 1 | 0.9371 | 501 | 0.0473 |
|
| 37 |
+
| 26 | 0.8524 | 526 | 0.0381 |
|
| 38 |
+
| 51 | 1.1081 | 551 | 0.0567 |
|
| 39 |
+
| 76 | 1.0715 | 576 | 0.0305 |
|
| 40 |
+
| 101 | 0.8832 | 601 | 0.1005 |
|
| 41 |
+
| 126 | 0.4898 | 626 | 0.0297 |
|
| 42 |
+
| 151 | 0.7161 | 651 | 0.1560 |
|
| 43 |
+
| 176 | 0.3995 | 676 | 0.0338 |
|
| 44 |
+
| 201 | 0.4137 | 701 | 0.0215 |
|
| 45 |
+
| 226 | 0.1955 | 726 | 0.0757 |
|
| 46 |
+
| 251 | 0.1949 | 751 | 0.0334 |
|
| 47 |
+
| 276 | 0.1310 | 776 | 0.0215 |
|
| 48 |
+
| 301 | 0.1201 | 801 | 0.0156 |
|
| 49 |
+
| 326 | 0.7008 | 826 | 0.0163 |
|
| 50 |
+
| 351 | 0.2478 | 851 | 0.0245 |
|
| 51 |
+
| 376 | 0.3107 | 876 | 0.0281 |
|
| 52 |
+
| 401 | 0.0922 | 901 | 0.0187 |
|
| 53 |
+
| 426 | 0.0585 | 926 | 0.0260 |
|
| 54 |
+
| 451 | 1.0274 | 951 | 0.0256 |
|
| 55 |
+
| 476 | 0.1101 | 976 | 0.0264 |
|
| 56 |
+
| | | 1001 | 0.0511 |
|
| 57 |
+
|
| 58 |
+
min loss: **0.0132** (фаза активного заучивания ~шаги 700–900), loss снизился на 2 порядка: 1.27 → 0.01–0.05.
|
| 59 |
+
|
| 60 |
+
## Ресурсы
|
| 61 |
+
|
| 62 |
+
- ~1.3 с/шаг обучения · сэмплинг ~60 с на проверку · итого ~28 мин на 1001 шаг
|
| 63 |
+
- Память: ~15.1 / 16.4 GB (UNet fp32 + AdamW8bit + VAE fp16; text encoder выгружен после кодирования)
|
| 64 |
+
|
| 65 |
+
## Артефакты
|
| 66 |
+
|
| 67 |
+
- `unet_test/diffusion_pytorch_model.safetensors` — веса после 1001 шага (08:28)
|
| 68 |
+
- `samples_test/gt.png` — эталон (декодированный GT)
|
| 69 |
+
- `samples_test/rec_0000/0500/1000.png` — реконструкция из 50% шума
|
| 70 |
+
- `samples_test/gen_0000/0500/1000.png` — генерация из чистого шума (seed 0)
|
| 71 |
+
|
| 72 |
+
## Выводы
|
| 73 |
+
|
| 74 |
+
1. **Гипотеза подтверждена**: UNet способен выучить одну картинку. За 1001 шаг при lr 5e-5 loss упал на 2 порядка (до 0.013), генерация из чистого шума достигает **+15 dB PSNR** к оригиналу.
|
| 75 |
+
2. **Порог заучивания** — примерно между шагами 200–500: loss переходит в область <0.2, `gen` скачет с −3 до +12.65 dB.
|
| 76 |
+
3. **gen > rec** — при «заученной» картинке прямая генерация ближе к GT, чем 20-шаговая реконструкция из 50% шума (последняя просто не успевает сойтись).
|
| 77 |
+
4. Дальнейший рост (1000+ шагов, до +20+ dB) возможен, но с убывающей отдачей.
|
| 78 |
+
|
| 79 |
+
## Воспроизведение
|
| 80 |
+
|
| 81 |
+
```bash
|
| 82 |
+
cp -r unet/. unet_test/ # сброс к исходным весам
|
| 83 |
+
python3 train_test.py --epochs 1001 --sample-every-steps 500 --save-every-steps 500
|
| 84 |
+
```
|
dataset.py
CHANGED
|
@@ -33,8 +33,8 @@ step = 64 #32
|
|
| 33 |
empty_share = 0.0
|
| 34 |
limit = 0
|
| 35 |
|
| 36 |
-
folder_path = "
|
| 37 |
-
save_path = "datasets/
|
| 38 |
os.makedirs(save_path, exist_ok=True)
|
| 39 |
|
| 40 |
def clear_cuda_memory():
|
|
|
|
| 33 |
empty_share = 0.0
|
| 34 |
limit = 0
|
| 35 |
|
| 36 |
+
folder_path = "/home/recoilme/dataset/alchemist"
|
| 37 |
+
save_path = "datasets/alchemist"
|
| 38 |
os.makedirs(save_path, exist_ok=True)
|
| 39 |
|
| 40 |
def clear_cuda_memory():
|
dataset_alchemist.log
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
flash_attn is installed but failed to import: cannot import name '_wrapped_flash_attn_backward' from 'flash_attn.flash_attn_interface' (/home/recoilme/.local/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py). Falling back to native PyTorch attention.
|
| 2 |
+
[GPU 0] Загрузка моделей...
|
| 3 |
+
The config attributes {'latents_mean': [-0.03542253375053406, 0.20086465775966644, -0.016413161531090736, -0.0956302210688591, -0.2672063112258911, 0.2609933018684387, -0.07806991040706635, -0.48407721519470215, 0.21844269335269928, -0.1122383326292038, 0.27197545766830444, -0.18958772718906403, 0.18776826560497284, 0.0987580344080925, 0.2837068736553192, -0.4486690163612366, 0.4816776514053345, 0.02947971224784851, -0.1337375044822693, -0.39750921726226807, -0.08513020724058151, -0.054023586213588715, -0.3943594992160797, 0.23918119072914124, -0.12466679513454437, 0.09935147315263748, 0.31858691573143005, 0.48585832118988037, -0.6416525840759277, -0.15164820849895477, -0.4693508744239807, -0.13071806728839874], 'latents_std': [1.5792087316513062, 1.5769503116607666, 1.5864241123199463, 1.6454921960830688, 1.5336694717407227, 1.5587652921676636, 1.5838669538497925, 1.5659377574920654, 1.6860467195510864, 1.5192310810089111, 1.573639988899231, 1.5953549146652222, 1.5271092653274536, 1.6246271133422852, 1.7054023742675781, 1.607722282409668, 1.558642864227295, 1.5824549198150635, 1.6202995777130127, 1.6206320524215698, 1.6379750967025757, 1.6527063846588135, 1.498811960220337, 1.5706247091293335, 1.5854856967926025, 1.4828169345855713, 1.5693111419677734, 1.692481517791748, 1.6409776210784912, 1.6216280460357666, 1.6087706089019775, 1.5776633024215698]} were passed to AsymmetricAutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
|
| 4 |
+
Найдено 3350 изображений
|
| 5 |
+
[GPU 0] обрабатывает 3350 файлов
|
| 6 |
+
|
| 7 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_320x640_proc_0
|
| 8 |
+
[GPU 0] used_gb: 1.37 GB
|
| 9 |
+
|
| 10 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_384x640_proc_0
|
| 11 |
+
[GPU 0] used_gb: 1.60 GB
|
| 12 |
+
|
| 13 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_448x640_proc_0
|
| 14 |
+
[GPU 0] used_gb: 1.84 GB
|
| 15 |
+
|
| 16 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_512x640_proc_0
|
| 17 |
+
[GPU 0] used_gb: 2.07 GB
|
| 18 |
+
|
| 19 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_576x640_proc_0
|
| 20 |
+
[GPU 0] used_gb: 2.31 GB
|
| 21 |
+
|
| 22 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x320_proc_0
|
| 23 |
+
[GPU 0] used_gb: 2.31 GB
|
| 24 |
+
|
| 25 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x384_proc_0
|
| 26 |
+
[GPU 0] used_gb: 2.31 GB
|
| 27 |
+
|
| 28 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x448_proc_0
|
| 29 |
+
[GPU 0] used_gb: 2.31 GB
|
| 30 |
+
|
| 31 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x512_proc_0
|
| 32 |
+
[GPU 0] used_gb: 2.31 GB
|
| 33 |
+
|
| 34 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x576_proc_0
|
| 35 |
+
[GPU 0] used_gb: 2.31 GB
|
| 36 |
+
|
| 37 |
+
[GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x640_proc_0
|
| 38 |
+
[GPU 0] used_gb: 2.54 GB
|
| 39 |
+
|
| 40 |
+
✅ Сохранено
|
dataset_test.py
ADDED
|
@@ -0,0 +1,126 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# dataset_test.py — генерация датасета из одной картинки (тест UNet)
|
| 2 |
+
# Использование: python dataset_test.py
|
| 3 |
+
import torch
|
| 4 |
+
import os
|
| 5 |
+
import gc
|
| 6 |
+
import numpy as np
|
| 7 |
+
import shutil
|
| 8 |
+
|
| 9 |
+
from datasets import Dataset
|
| 10 |
+
from diffusers import AsymmetricAutoencoderKL
|
| 11 |
+
from torchvision.transforms import ToTensor, Normalize
|
| 12 |
+
from PIL import Image
|
| 13 |
+
|
| 14 |
+
# ---------------- 1️⃣ Настройки ----------------
|
| 15 |
+
dtype = torch.float16
|
| 16 |
+
min_size = 320
|
| 17 |
+
max_size = 640
|
| 18 |
+
step = 64
|
| 19 |
+
empty_share = 0.0
|
| 20 |
+
|
| 21 |
+
folder_path = "datasets/example"
|
| 22 |
+
save_path = "datasets/test"
|
| 23 |
+
|
| 24 |
+
# ---------------- 2️⃣ Модель ----------------
|
| 25 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 26 |
+
vae = AsymmetricAutoencoderKL.from_pretrained("vae", torch_dtype=dtype).to(device).eval()
|
| 27 |
+
|
| 28 |
+
shift_factor = getattr(vae.config, "shift_factor", 0.0) or 0.0
|
| 29 |
+
scaling_factor = getattr(vae.config, "scaling_factor", 1.0) or 1.0
|
| 30 |
+
|
| 31 |
+
mean = getattr(vae.config, "latents_mean", None)
|
| 32 |
+
std = getattr(vae.config, "latents_std", None)
|
| 33 |
+
has_mean_std = mean is not None and std is not None
|
| 34 |
+
if has_mean_std:
|
| 35 |
+
latents_std = torch.tensor(std, device=device, dtype=dtype).view(1, len(std), 1, 1)
|
| 36 |
+
latents_mean = torch.tensor(mean, device=device, dtype=dtype).view(1, len(mean), 1, 1)
|
| 37 |
+
else:
|
| 38 |
+
latents_std, latents_mean = 1.0, 0.0
|
| 39 |
+
|
| 40 |
+
# ---------------- 3️⃣ Трансформация (та же логика, что в dataset.py) ----------------
|
| 41 |
+
def transform(img):
|
| 42 |
+
original_width, original_height = img.size
|
| 43 |
+
|
| 44 |
+
if original_width >= original_height:
|
| 45 |
+
new_width = max_size
|
| 46 |
+
new_height = int(max_size * original_height / original_width)
|
| 47 |
+
else:
|
| 48 |
+
new_height = max_size
|
| 49 |
+
new_width = int(max_size * original_width / original_height)
|
| 50 |
+
|
| 51 |
+
if new_height < min_size or new_width < min_size:
|
| 52 |
+
if original_width <= original_height:
|
| 53 |
+
new_width = min_size
|
| 54 |
+
new_height = int(min_size * original_height / original_width)
|
| 55 |
+
else:
|
| 56 |
+
new_height = min_size
|
| 57 |
+
new_width = int(min_size * original_width / original_height)
|
| 58 |
+
|
| 59 |
+
crop_width = min(max_size, (new_width // step) * step)
|
| 60 |
+
crop_height = min(max_size, (new_height // step) * step)
|
| 61 |
+
crop_width = max(min_size, crop_width)
|
| 62 |
+
crop_height = max(min_size, crop_height)
|
| 63 |
+
|
| 64 |
+
img_resized = img.convert("RGB").resize((new_width, new_height), Image.LANCZOS)
|
| 65 |
+
top = (new_height - crop_height) // 3
|
| 66 |
+
left = 0
|
| 67 |
+
img_cropped = img_resized.crop((left, top, left + crop_width, top + crop_height))
|
| 68 |
+
|
| 69 |
+
img_tensor = ToTensor()(img_cropped)
|
| 70 |
+
img_tensor = Normalize(mean=[0.5] * 3, std=[0.5] * 3)(img_tensor)
|
| 71 |
+
return img_tensor, img_cropped, img_cropped.size[0], img_cropped.size[1]
|
| 72 |
+
|
| 73 |
+
# ---------------- 4️⃣ Обработка папки ----------------
|
| 74 |
+
image_paths, text_paths = [], []
|
| 75 |
+
for root, _, files in os.walk(folder_path):
|
| 76 |
+
for filename in sorted(files):
|
| 77 |
+
if filename.lower().endswith((".jpg", ".jpeg", ".png")):
|
| 78 |
+
image_path = os.path.join(root, filename)
|
| 79 |
+
text_path = os.path.splitext(image_path)[0] + ".txt"
|
| 80 |
+
if os.path.exists(text_path):
|
| 81 |
+
image_paths.append(image_path)
|
| 82 |
+
text_paths.append(text_path)
|
| 83 |
+
|
| 84 |
+
print(f"Найдено {len(image_paths)} изображений в {folder_path}")
|
| 85 |
+
|
| 86 |
+
# ---------------- 5️⃣ Энкодинг ----------------
|
| 87 |
+
latents_all, texts_all, widths_all, heights_all = [], [], [], []
|
| 88 |
+
|
| 89 |
+
for img_path, txt_path in zip(image_paths, text_paths):
|
| 90 |
+
img = Image.open(img_path).convert("RGB")
|
| 91 |
+
t_img, img_cropped, w, h = transform(img)
|
| 92 |
+
print(f" {os.path.basename(img_path)}: {img.size} -> crop {w}x{h}")
|
| 93 |
+
|
| 94 |
+
batch_tensor = t_img.unsqueeze(0).to(device, dtype)
|
| 95 |
+
with torch.no_grad():
|
| 96 |
+
posteriors = vae.encode(batch_tensor).latent_dist.mode()
|
| 97 |
+
if has_mean_std:
|
| 98 |
+
posteriors = (posteriors - latents_mean) / latents_std
|
| 99 |
+
posteriors = (posteriors - shift_factor) / scaling_factor
|
| 100 |
+
|
| 101 |
+
latents_all.append(posteriors.cpu().numpy()[0])
|
| 102 |
+
|
| 103 |
+
with open(txt_path, "r", encoding="utf-8") as f:
|
| 104 |
+
text = f.read().strip()
|
| 105 |
+
if empty_share > 0 and np.random.random() < empty_share:
|
| 106 |
+
text = ""
|
| 107 |
+
texts_all.append(text)
|
| 108 |
+
widths_all.append(w)
|
| 109 |
+
heights_all.append(h)
|
| 110 |
+
|
| 111 |
+
# ---------------- 6️⃣ Сохранение ----------------
|
| 112 |
+
if os.path.exists(save_path):
|
| 113 |
+
shutil.rmtree(save_path)
|
| 114 |
+
os.makedirs(save_path, exist_ok=True)
|
| 115 |
+
|
| 116 |
+
ds = Dataset.from_dict({
|
| 117 |
+
"vae": latents_all,
|
| 118 |
+
"text": texts_all,
|
| 119 |
+
"width": widths_all,
|
| 120 |
+
"height": heights_all,
|
| 121 |
+
})
|
| 122 |
+
ds.save_to_disk(save_path)
|
| 123 |
+
|
| 124 |
+
print(f"✅ Датасет сохранён: {save_path} | {len(ds)} сэмплов")
|
| 125 |
+
print(" Размеры:", ds["width"], ds["height"])
|
| 126 |
+
print(" Текст:", ds["text"])
|
generate.py
ADDED
|
@@ -0,0 +1,124 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
SDXS-1B image generation from prompts.txt.
|
| 3 |
+
|
| 4 |
+
Usage:
|
| 5 |
+
python generate.py # generates grid from prompts.txt
|
| 6 |
+
python generate.py --prompts my_list.txt # custom prompts file
|
| 7 |
+
python generate.py --prompt "your text" # single prompt (без файла)
|
| 8 |
+
python generate.py --index 0 # только промпт #0 из prompts.txt
|
| 9 |
+
"""
|
| 10 |
+
import argparse
|
| 11 |
+
import os
|
| 12 |
+
import math
|
| 13 |
+
import textwrap
|
| 14 |
+
|
| 15 |
+
import torch
|
| 16 |
+
import numpy as np
|
| 17 |
+
|
| 18 |
+
from pipeline_sdxs import SdxsPipeline
|
| 19 |
+
|
| 20 |
+
NEGATIVE_PROMPT = (
|
| 21 |
+
"bad quality grainy image with low details, incomplete text, "
|
| 22 |
+
"despite numerous technical flaws and distorted figures"
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
DEFAULT_PROMPTS_FILE = "prompts.txt"
|
| 26 |
+
GRID_OUTPUT = "media/result_grid.jpg"
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
def main():
|
| 30 |
+
parser = argparse.ArgumentParser(description="SDXS-1B generation")
|
| 31 |
+
parser.add_argument("--prompts", type=str, default=None, help="prompts file")
|
| 32 |
+
parser.add_argument("--prompt", type=str, default=None, help="single prompt")
|
| 33 |
+
parser.add_argument("--index", type=int, default=None, help="generate only prompt #N from file")
|
| 34 |
+
parser.add_argument("--steps", type=int, default=40)
|
| 35 |
+
parser.add_argument("--guidance", type=float, default=5.0)
|
| 36 |
+
parser.add_argument("--seed", type=int, default=0)
|
| 37 |
+
parser.add_argument("--width", type=int, default=1024)
|
| 38 |
+
parser.add_argument("--height", type=int, default=1408)
|
| 39 |
+
parser.add_argument("--cols", type=int, default=4, help="колонки в гриде")
|
| 40 |
+
parser.add_argument("--output", type=str, default=None,
|
| 41 |
+
help="папка для отдельных картинок (по умолчанию не сохраняются)")
|
| 42 |
+
args = parser.parse_args()
|
| 43 |
+
|
| 44 |
+
# ---------- сбор промптов ----------
|
| 45 |
+
prompts = None
|
| 46 |
+
if args.prompt:
|
| 47 |
+
prompts = [args.prompt]
|
| 48 |
+
else:
|
| 49 |
+
path = args.prompts or DEFAULT_PROMPTS_FILE
|
| 50 |
+
if not os.path.exists(path):
|
| 51 |
+
print(f"Файл промптов не найден: {path}")
|
| 52 |
+
return
|
| 53 |
+
with open(path, encoding="utf-8") as f:
|
| 54 |
+
prompts = [l.strip() for l in f if l.strip()]
|
| 55 |
+
|
| 56 |
+
if args.index is not None:
|
| 57 |
+
if args.index >= len(prompts):
|
| 58 |
+
print(f"Нет промпта #{args.index} (всего {len(prompts)})")
|
| 59 |
+
return
|
| 60 |
+
prompts = [prompts[args.index]]
|
| 61 |
+
|
| 62 |
+
if not prompts:
|
| 63 |
+
print("Нет промптов")
|
| 64 |
+
return
|
| 65 |
+
|
| 66 |
+
print(f"Промптов: {len(prompts)}")
|
| 67 |
+
|
| 68 |
+
# ---------- пайплайн ----------
|
| 69 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 70 |
+
pipe = SdxsPipeline.from_pretrained(
|
| 71 |
+
os.path.dirname(os.path.abspath(__file__)),
|
| 72 |
+
torch_dtype=torch.float16,
|
| 73 |
+
).to(device)
|
| 74 |
+
|
| 75 |
+
# ---------- генерация ----------
|
| 76 |
+
images = []
|
| 77 |
+
for i, prompt in enumerate(prompts):
|
| 78 |
+
print(f"[{i+1}/{len(prompts)}] {prompt[:80]}...")
|
| 79 |
+
image = pipe(
|
| 80 |
+
prompt=prompt,
|
| 81 |
+
negative_prompt=NEGATIVE_PROMPT,
|
| 82 |
+
guidance_scale=args.guidance,
|
| 83 |
+
width=args.width,
|
| 84 |
+
height=args.height,
|
| 85 |
+
seed=args.seed,
|
| 86 |
+
num_inference_steps=args.steps,
|
| 87 |
+
)[0][0]
|
| 88 |
+
images.append(image)
|
| 89 |
+
|
| 90 |
+
if args.output:
|
| 91 |
+
os.makedirs(args.output, exist_ok=True)
|
| 92 |
+
path = os.path.join(args.output, f"{i+1:03d}.png")
|
| 93 |
+
image.save(path)
|
| 94 |
+
print(f" saved {path}")
|
| 95 |
+
|
| 96 |
+
# ---------- грид ----------
|
| 97 |
+
os.makedirs(os.path.dirname(GRID_OUTPUT), exist_ok=True)
|
| 98 |
+
|
| 99 |
+
import matplotlib
|
| 100 |
+
matplotlib.use("Agg")
|
| 101 |
+
import matplotlib.pyplot as plt
|
| 102 |
+
|
| 103 |
+
cols = min(args.cols, len(images))
|
| 104 |
+
rows = math.ceil(len(images) / cols)
|
| 105 |
+
fig, axes = plt.subplots(rows, cols, figsize=(cols * 4, rows * 4.5), constrained_layout=True)
|
| 106 |
+
axes = list(np.array(axes).flatten())[:len(images)]
|
| 107 |
+
|
| 108 |
+
for i, (img, prompt) in enumerate(zip(images, prompts)):
|
| 109 |
+
ax = axes[i]
|
| 110 |
+
ax.imshow(img)
|
| 111 |
+
ax.axis("off")
|
| 112 |
+
ax.set_aspect("equal")
|
| 113 |
+
text = (prompt[:200] + "…") if len(prompt) > 200 else prompt
|
| 114 |
+
lines = textwrap.wrap(text, width=35)
|
| 115 |
+
while len(lines) < 4:
|
| 116 |
+
lines.append("")
|
| 117 |
+
ax.set_title("\n".join(lines), fontsize=9, pad=8)
|
| 118 |
+
|
| 119 |
+
plt.savefig(GRID_OUTPUT, bbox_inches="tight", dpi=150, format="jpeg")
|
| 120 |
+
print(f"grid -> {GRID_OUTPUT}")
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
if __name__ == "__main__":
|
| 124 |
+
main()
|
media/result_grid.jpg
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
migrate_test.py
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# migrate_test.py — миграция весов unet → unet_test с новым распределением трансформеров
|
| 2 |
+
# Эксперимент: T=[4,3,2,1] → T=[2,2,3,4] (внимание с верха в горлышко), L=[4,3,2,2] без изменений
|
| 3 |
+
import os, json, torch, re
|
| 4 |
+
from safetensors.torch import load_file, save_file
|
| 5 |
+
from diffusers import UNet2DConditionModel
|
| 6 |
+
|
| 7 |
+
SRC_DIR = "unet"
|
| 8 |
+
DST_DIR = "unet_test"
|
| 9 |
+
|
| 10 |
+
# Переопределения конфига (только то, что меняем; остальное — из исходного)
|
| 11 |
+
OVERRIDES = {
|
| 12 |
+
"transformer_layers_per_block": [2, 2, 3, 4],
|
| 13 |
+
}
|
| 14 |
+
|
| 15 |
+
os.makedirs(DST_DIR, exist_ok=True)
|
| 16 |
+
|
| 17 |
+
# 1. Конфиг: исходный + оверрайды
|
| 18 |
+
with open(os.path.join(SRC_DIR, "config.json"), "r", encoding="utf-8") as f:
|
| 19 |
+
cfg = json.load(f)
|
| 20 |
+
|
| 21 |
+
for k in ["_class_name", "_diffusers_version", "_name_or_path"]:
|
| 22 |
+
cfg.pop(k, None)
|
| 23 |
+
|
| 24 |
+
cfg.update(OVERRIDES)
|
| 25 |
+
|
| 26 |
+
with open(os.path.join(DST_DIR, "config.json"), "w", encoding="utf-8") as f:
|
| 27 |
+
json.dump({
|
| 28 |
+
"_class_name": "UNet2DConditionModel",
|
| 29 |
+
"_diffusers_version": "0.38.0",
|
| 30 |
+
"_name_or_path": "unet_test",
|
| 31 |
+
**cfg,
|
| 32 |
+
}, f, indent=2)
|
| 33 |
+
|
| 34 |
+
print(f"Конфиг: L={cfg['layers_per_block']} T={cfg['transformer_layers_per_block']}")
|
| 35 |
+
|
| 36 |
+
# 2. Загружаем старые веса
|
| 37 |
+
print("Загрузка старых весов...")
|
| 38 |
+
old_state = load_file(os.path.join(SRC_DIR, "diffusion_pytorch_model.safetensors"), device="cpu")
|
| 39 |
+
|
| 40 |
+
# 3. Новая модель
|
| 41 |
+
print("Создание новой модели...")
|
| 42 |
+
new_model = UNet2DConditionModel(**cfg)
|
| 43 |
+
new_state = new_model.state_dict()
|
| 44 |
+
|
| 45 |
+
old_params = sum(p.numel() for p in old_state.values()) / 1e9
|
| 46 |
+
new_params = sum(p.numel() for p in new_state.values()) / 1e9
|
| 47 |
+
print(f"Старый: {old_params:.3f}B → Новый: {new_params:.3f}B")
|
| 48 |
+
|
| 49 |
+
# 4. Копирование точных совпадений
|
| 50 |
+
copied = 0
|
| 51 |
+
for key, old_t in old_state.items():
|
| 52 |
+
if key in new_state and old_t.shape == new_state[key].shape:
|
| 53 |
+
new_state[key] = old_t.clone()
|
| 54 |
+
copied += 1
|
| 55 |
+
|
| 56 |
+
# 5. Warm-start новых слоёв от ближайших существующих (перебор индексов назад)
|
| 57 |
+
def find_source(old_state, key, new_state):
|
| 58 |
+
m = re.match(r"(.*\.)(\d+)\.(.+)", key)
|
| 59 |
+
if not m:
|
| 60 |
+
return None
|
| 61 |
+
stem, idx_str, suffix = m.group(1), m.group(2), m.group(3)
|
| 62 |
+
idx = int(idx_str)
|
| 63 |
+
for src_idx in range(idx - 1, -1, -1):
|
| 64 |
+
src_key = f"{stem}{src_idx}.{suffix}"
|
| 65 |
+
if src_key in old_state and old_state[src_key].shape == new_state[key].shape:
|
| 66 |
+
return src_key
|
| 67 |
+
return None
|
| 68 |
+
|
| 69 |
+
warm_started = []
|
| 70 |
+
for key in new_state:
|
| 71 |
+
if key not in old_state or old_state[key].shape != new_state[key].shape:
|
| 72 |
+
src_key = find_source(old_state, key, new_state)
|
| 73 |
+
if src_key:
|
| 74 |
+
new_state[key] = old_state[src_key].clone()
|
| 75 |
+
warm_started.append((key, src_key))
|
| 76 |
+
|
| 77 |
+
print(f"Прямых копий: {copied}")
|
| 78 |
+
print(f"Warm-start: {len(warm_started)} ключей")
|
| 79 |
+
for k, src in warm_started[:12]:
|
| 80 |
+
print(f" {k} ← {src}")
|
| 81 |
+
if len(warm_started) > 12:
|
| 82 |
+
print(f" ... и ещё {len(warm_started) - 12}")
|
| 83 |
+
|
| 84 |
+
# 6. Статистика
|
| 85 |
+
total_copied = sum(old_state[k].numel() for k in old_state if k in new_state and old_state[k].shape == new_state[k].shape)
|
| 86 |
+
total_ws = sum(new_state[k].numel() for k, _ in warm_started)
|
| 87 |
+
total_new = sum(v.numel() for v in new_state.values())
|
| 88 |
+
print(f"\nСкопировано: {total_copied/1e9:.3f}B ({total_copied/total_new*100:.1f}%)")
|
| 89 |
+
print(f"Warm-start: {total_ws/1e9:.3f}B ({total_ws/total_new*100:.1f}%)")
|
| 90 |
+
print(f"Random init: {(total_new - total_copied - total_ws)/1e9:.3f}B ({(total_new - total_copied - total_ws)/total_new*100:.1f}%)")
|
| 91 |
+
|
| 92 |
+
# 7. Сохранение
|
| 93 |
+
print("\nСохранение...")
|
| 94 |
+
save_file(new_state, os.path.join(DST_DIR, "diffusion_pytorch_model.safetensors"))
|
| 95 |
+
print(f"Готово → {DST_DIR}/")
|
prompts.txt
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
A digital illustration of a young male character with white hair, blue eyes, and a dark complexion. He is wearing a white jacket, brown pants, and a brown belt. His hands are in his pockets, and he is wearing a wristwatch. He has a pouch on his belt and is wearing glasses. The background is a simple gradient.
|
| 2 |
+
A young woman with striking blue eyes and pointed ears, adorned with a floral kimono and a tattoo. Her hair is styled in a braid, and she wears a pair of ears.
|
| 3 |
+
A fluffy domestic cat with piercing green eyes sits attentively in a sunlit room filled with natural light, its soft fur reflecting warm hues of orange through golden windows.
|
| 4 |
+
A fierce woman in black-and-white, wearing a spiked iron mask with sharp metallic spikes, sporting a high ponytail, her skin marked with battle-dirt, and eyes that reflect determination and strength.
|
| 5 |
+
A close-up of an astronaut's helmet with frosted, opaque visor, reflecting space's cold, frozen texture, resting on the visor a butterfly with vibrant, intricately patterned wings, and distant stars' faint glow.
|
| 6 |
+
A black BMW M3 sports car with black and yellow rims.
|
| 7 |
+
A young girl in a flowing, vibrant dress, her glowing eyes capturing the warmth of the day, sits on a grassy field, surrounded by anime-style elements.
|
| 8 |
+
A small bluebird with a white breast and black stripe perches on a branch, facing right with a blurred background of trees and bushes.
|
| 9 |
+
космонавт на мотоцикле
|
| 10 |
+
A muscular, topless male with tiger-like ears and a tail stands in a forest, holding a sword and wearing a blue outfit, gazing directly at the viewer.
|
samples/unet_384x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_416x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_448x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_480x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_512x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_544x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_576x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_608x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_640x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_672x704_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x384_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x416_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x448_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x480_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x512_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x544_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x576_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x608_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x640_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x672_0.jpg
DELETED
Git LFS Details
|
samples/unet_704x704_0.jpg
DELETED
Git LFS Details
|
sdservice.py
ADDED
|
@@ -0,0 +1,187 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
SDXS-1B HTTP-сервис для Go-телеграм-бота (адаптация sdservice.py из sdxs-2b).
|
| 3 |
+
|
| 4 |
+
POST / {"prompt": "..."} → {"images": [base64, base64], "refined_prompts": [...]}
|
| 5 |
+
|
| 6 |
+
Запуск:
|
| 7 |
+
python3 sdservice.py [--port 8989]
|
| 8 |
+
"""
|
| 9 |
+
from http.server import BaseHTTPRequestHandler, HTTPServer
|
| 10 |
+
import socketserver
|
| 11 |
+
import io
|
| 12 |
+
import torch
|
| 13 |
+
|
| 14 |
+
# Патч для torch.compile (обход бага accumulated_recompile_limit)
|
| 15 |
+
try:
|
| 16 |
+
import torch.utils._config_module as _cfg
|
| 17 |
+
if hasattr(_cfg, 'ConfigModule') and not hasattr(_cfg.ConfigModule, '_is_patched'):
|
| 18 |
+
original_setattr = _cfg.ConfigModule.__setattr__
|
| 19 |
+
|
| 20 |
+
def patched_setattr(self, name, value):
|
| 21 |
+
if name == 'accumulated_recompile_limit':
|
| 22 |
+
return
|
| 23 |
+
original_setattr(self, name, value)
|
| 24 |
+
|
| 25 |
+
_cfg.ConfigModule.__setattr__ = patched_setattr
|
| 26 |
+
_cfg.ConfigModule._is_patched = True
|
| 27 |
+
except Exception as e:
|
| 28 |
+
print(f"Патч torch config пропущен (некритично): {e}", flush=True)
|
| 29 |
+
|
| 30 |
+
import gc
|
| 31 |
+
import base64
|
| 32 |
+
import json
|
| 33 |
+
import threading
|
| 34 |
+
import time
|
| 35 |
+
import sys
|
| 36 |
+
import argparse
|
| 37 |
+
|
| 38 |
+
from diffusers import DiffusionPipeline
|
| 39 |
+
from PIL import Image
|
| 40 |
+
|
| 41 |
+
# ---------------- 1️⃣ Настройки и константы ----------------
|
| 42 |
+
PIPE1_ID = "/home/recoilme/sdxs-1b"
|
| 43 |
+
|
| 44 |
+
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
|
| 45 |
+
DTYPE = torch.float16 if torch.cuda.is_available() else torch.float32
|
| 46 |
+
|
| 47 |
+
gpu_lock = threading.Lock()
|
| 48 |
+
|
| 49 |
+
NEGATIVE_PROMPT_1 = (
|
| 50 |
+
"bad quality grainy image with low details, incomplete text, "
|
| 51 |
+
"despite numerous technical flaws and distorted figures"
|
| 52 |
+
)
|
| 53 |
+
|
| 54 |
+
print(f"Загрузка Модели 1 ({PIPE1_ID})...", flush=True)
|
| 55 |
+
pipe1 = DiffusionPipeline.from_pretrained(
|
| 56 |
+
PIPE1_ID,
|
| 57 |
+
torch_dtype=DTYPE,
|
| 58 |
+
trust_remote_code=True
|
| 59 |
+
).to(DEVICE)
|
| 60 |
+
|
| 61 |
+
pipe1.unet.to(memory_format=torch.channels_last)
|
| 62 |
+
|
| 63 |
+
# torch.compile: если не компилируется — работаем без него
|
| 64 |
+
try:
|
| 65 |
+
print("Компилирую UNet...", flush=True)
|
| 66 |
+
pipe1.unet = torch.compile(pipe1.unet, mode="reduce-overhead", fullgraph=False)
|
| 67 |
+
pipe1.vae = torch.compile(pipe1.vae, mode="reduce-overhead", fullgraph=False)
|
| 68 |
+
except Exception as e:
|
| 69 |
+
print(f"torch.compile не удался, работаем без него: {e}", flush=True)
|
| 70 |
+
|
| 71 |
+
# ---------------- 2️⃣ Вспомогательные функции ----------------
|
| 72 |
+
def encode_images_to_list(images):
|
| 73 |
+
"""Возвращает список строк base64 и агрессивно чистит оперативную память"""
|
| 74 |
+
encoded_list = []
|
| 75 |
+
for image in images:
|
| 76 |
+
with io.BytesIO() as buffer:
|
| 77 |
+
image.save(buffer, format='JPEG', quality=97)
|
| 78 |
+
encoded_image = base64.b64encode(buffer.getvalue()).decode('utf-8')
|
| 79 |
+
encoded_list.append(encoded_image)
|
| 80 |
+
|
| 81 |
+
gc.collect()
|
| 82 |
+
return encoded_list
|
| 83 |
+
|
| 84 |
+
def txt2img(prompt_text):
|
| 85 |
+
"""
|
| 86 |
+
Инференс на модели. Возвращает (2 картинки, 2 промпта).
|
| 87 |
+
Батч делаем списком промптов (num_images_per_prompt в пайплайне sdxs-1b нет).
|
| 88 |
+
"""
|
| 89 |
+
with torch.no_grad():
|
| 90 |
+
# refined = pipe1.refine_prompts(prompt_text)[0] # раскомментируй, чтобы улучшать промпт
|
| 91 |
+
refined = prompt_text
|
| 92 |
+
|
| 93 |
+
if refined == prompt_text:
|
| 94 |
+
# Промпт не изменился -> 2 картинки с одним промптом
|
| 95 |
+
prompts_list = [prompt_text, prompt_text]
|
| 96 |
+
else:
|
| 97 |
+
# Разные промпты -> батч с обоими
|
| 98 |
+
prompts_list = [prompt_text, refined]
|
| 99 |
+
|
| 100 |
+
images = pipe1(
|
| 101 |
+
prompt=prompts_list,
|
| 102 |
+
negative_prompt=[NEGATIVE_PROMPT_1, NEGATIVE_PROMPT_1],
|
| 103 |
+
width=768,
|
| 104 |
+
height=1152,
|
| 105 |
+
num_inference_steps=35,
|
| 106 |
+
).images
|
| 107 |
+
|
| 108 |
+
return [images[0], images[1]], prompts_list
|
| 109 |
+
|
| 110 |
+
# ---------------- 3️⃣ Серверная часть ----------------
|
| 111 |
+
class ThreadedHTTPServer(socketserver.ThreadingMixIn, HTTPServer):
|
| 112 |
+
daemon_threads = True
|
| 113 |
+
|
| 114 |
+
class RequestHandler(BaseHTTPRequestHandler):
|
| 115 |
+
|
| 116 |
+
def log_message(self, format, *args):
|
| 117 |
+
pass # отключаем стандартный вывод
|
| 118 |
+
|
| 119 |
+
def do_POST(self):
|
| 120 |
+
try:
|
| 121 |
+
content_length = int(self.headers['Content-Length'])
|
| 122 |
+
body = self.rfile.read(content_length)
|
| 123 |
+
|
| 124 |
+
data = json.loads(body.decode('utf-8'))
|
| 125 |
+
p1 = data.get('prompt', data.get('prompt1', ''))
|
| 126 |
+
p2 = data.get('prompt2', '')
|
| 127 |
+
full_prompt = f"{p1} {p2}".strip()
|
| 128 |
+
|
| 129 |
+
prompt_log = (full_prompt[:70] + '..') if len(full_prompt) > 70 else full_prompt
|
| 130 |
+
print(f"Prompt: {prompt_log}", file=sys.stderr, flush=True)
|
| 131 |
+
|
| 132 |
+
start_time = time.time()
|
| 133 |
+
|
| 134 |
+
with gpu_lock:
|
| 135 |
+
images, refined_prompts = txt2img(full_prompt)
|
| 136 |
+
|
| 137 |
+
generation_time = time.time() - start_time
|
| 138 |
+
|
| 139 |
+
base64_images = encode_images_to_list(images)
|
| 140 |
+
|
| 141 |
+
response_data = {
|
| 142 |
+
"images": base64_images,
|
| 143 |
+
"refined_prompts": refined_prompts
|
| 144 |
+
}
|
| 145 |
+
|
| 146 |
+
self.send_response(200)
|
| 147 |
+
self.send_header('Content-type', 'application/json')
|
| 148 |
+
self.end_headers()
|
| 149 |
+
self.wfile.write(json.dumps(response_data).encode('utf-8'))
|
| 150 |
+
|
| 151 |
+
print(f"[{generation_time:.2f}s] | Batch 2 | Prompt: {prompt_log}", file=sys.stderr, flush=True)
|
| 152 |
+
|
| 153 |
+
del images
|
| 154 |
+
del body
|
| 155 |
+
|
| 156 |
+
except Exception as e:
|
| 157 |
+
print(f"[ERROR] Ошибка при обработке запроса: {e}", file=sys.stderr, flush=True)
|
| 158 |
+
self.send_response(500)
|
| 159 |
+
self.send_header('Content-type', 'text/plain')
|
| 160 |
+
self.end_headers()
|
| 161 |
+
self.wfile.write(str(e).encode('utf-8'))
|
| 162 |
+
|
| 163 |
+
def run_server(port):
|
| 164 |
+
server_address = ('', port)
|
| 165 |
+
httpd = ThreadedHTTPServer(server_address, RequestHandler)
|
| 166 |
+
|
| 167 |
+
print("Прогрев модели (ожидайте завершения torch.compile)...", flush=True)
|
| 168 |
+
start_warmup = time.time()
|
| 169 |
+
with gpu_lock:
|
| 170 |
+
try:
|
| 171 |
+
txt2img("warmup prompt, simple test, 1girl")
|
| 172 |
+
print(f"Прогрев завершен за {time.time() - start_warmup:.2f} сек.", flush=True)
|
| 173 |
+
except Exception as e:
|
| 174 |
+
print(f"Ошибка во время прогрева (некритично): {e}", flush=True)
|
| 175 |
+
|
| 176 |
+
print(f'Сервер запущен на порту {port}...', flush=True)
|
| 177 |
+
try:
|
| 178 |
+
httpd.serve_forever()
|
| 179 |
+
except KeyboardInterrupt:
|
| 180 |
+
print("\nСервер остановлен.")
|
| 181 |
+
httpd.server_close()
|
| 182 |
+
|
| 183 |
+
if __name__ == '__main__':
|
| 184 |
+
parser = argparse.ArgumentParser(description="SDXS-1B HTTP сервис")
|
| 185 |
+
parser.add_argument("--port", type=int, default=8989)
|
| 186 |
+
args = parser.parse_args()
|
| 187 |
+
run_server(args.port)
|
test.ipynb
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:af07dcd7f492c3a229adc2742f3a094ccdcfb6c1f551c1ae51a06d809a108001
|
| 3 |
+
size 18066120
|
train.py
CHANGED
|
@@ -269,6 +269,15 @@ def generate_samples(
|
|
| 269 |
return images, captions
|
| 270 |
|
| 271 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 272 |
# ----------------------------- train ------------------------------
|
| 273 |
|
| 274 |
def main():
|
|
@@ -277,7 +286,7 @@ def main():
|
|
| 277 |
if args.seed is not None:
|
| 278 |
set_seed(args.seed)
|
| 279 |
|
| 280 |
-
accelerator = Accelerator(mixed_precision="
|
| 281 |
device = accelerator.device
|
| 282 |
is_main = accelerator.is_main_process
|
| 283 |
|
|
@@ -292,14 +301,14 @@ def main():
|
|
| 292 |
# UNet: загружаем из unet_path, если там есть веса — продолжаем
|
| 293 |
if is_main:
|
| 294 |
print(f"Loading UNet from {args.unet_path}...")
|
| 295 |
-
unet = UNet2DConditionModel.from_pretrained(args.unet_path, torch_dtype=torch.
|
| 296 |
if is_main:
|
| 297 |
print(f"UNet loaded: {sum(p.numel() for p in unet.parameters())/1e9:.3f}B params. Continuing training.")
|
| 298 |
unet.enable_gradient_checkpointing()
|
| 299 |
|
| 300 |
# text encoder всегда на GPU для обучения
|
| 301 |
text_encoder = Qwen3_5ForConditionalGeneration.from_pretrained(
|
| 302 |
-
"text_encoder", torch_dtype=torch.
|
| 303 |
).to(device).eval()
|
| 304 |
tokenizer = Qwen3_5Tokenizer.from_pretrained("tokenizer")
|
| 305 |
scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained("scheduler")
|
|
@@ -389,6 +398,7 @@ def main():
|
|
| 389 |
if args.wandb and images:
|
| 390 |
import wandb
|
| 391 |
wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=0)
|
|
|
|
| 392 |
|
| 393 |
for epoch in range(args.epochs):
|
| 394 |
batch_sampler.set_epoch(epoch)
|
|
@@ -445,6 +455,8 @@ def main():
|
|
| 445 |
if args.wandb:
|
| 446 |
import wandb
|
| 447 |
wandb.log(log_dict, step=global_step)
|
|
|
|
|
|
|
| 448 |
|
| 449 |
if is_main and vae is not None and (global_step == 5 or (now - last_sample_time) >= sample_interval):
|
| 450 |
images, captions = generate_samples(
|
|
@@ -462,6 +474,7 @@ def main():
|
|
| 462 |
if args.wandb and images:
|
| 463 |
import wandb
|
| 464 |
wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
|
|
|
|
| 465 |
last_sample_time = now
|
| 466 |
|
| 467 |
if not args.no_save and is_main and (now - last_save_time) >= save_interval:
|
|
@@ -490,6 +503,7 @@ def main():
|
|
| 490 |
if args.wandb and images:
|
| 491 |
import wandb
|
| 492 |
wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
|
|
|
|
| 493 |
|
| 494 |
if not args.no_save and is_main:
|
| 495 |
accelerator.unwrap_model(unet).save_pretrained(args.unet_path)
|
|
|
|
| 269 |
return images, captions
|
| 270 |
|
| 271 |
|
| 272 |
+
def save_samples_to_disk(images, captions, step, folder="samples"):
|
| 273 |
+
"""Сохраняет сгенерированные сэмплы в папку samples/ без номера шага в имени."""
|
| 274 |
+
if not images:
|
| 275 |
+
return
|
| 276 |
+
os.makedirs(folder, exist_ok=True)
|
| 277 |
+
for i, img in enumerate(images):
|
| 278 |
+
img.save(os.path.join(folder, f"{i}.png"))
|
| 279 |
+
|
| 280 |
+
|
| 281 |
# ----------------------------- train ------------------------------
|
| 282 |
|
| 283 |
def main():
|
|
|
|
| 286 |
if args.seed is not None:
|
| 287 |
set_seed(args.seed)
|
| 288 |
|
| 289 |
+
accelerator = Accelerator(mixed_precision="bf16")
|
| 290 |
device = accelerator.device
|
| 291 |
is_main = accelerator.is_main_process
|
| 292 |
|
|
|
|
| 301 |
# UNet: загружаем из unet_path, если там есть веса — продолжаем
|
| 302 |
if is_main:
|
| 303 |
print(f"Loading UNet from {args.unet_path}...")
|
| 304 |
+
unet = UNet2DConditionModel.from_pretrained(args.unet_path, torch_dtype=torch.bfloat16).to(device)
|
| 305 |
if is_main:
|
| 306 |
print(f"UNet loaded: {sum(p.numel() for p in unet.parameters())/1e9:.3f}B params. Continuing training.")
|
| 307 |
unet.enable_gradient_checkpointing()
|
| 308 |
|
| 309 |
# text encoder всегда на GPU для обучения
|
| 310 |
text_encoder = Qwen3_5ForConditionalGeneration.from_pretrained(
|
| 311 |
+
"text_encoder", torch_dtype=torch.bfloat16
|
| 312 |
).to(device).eval()
|
| 313 |
tokenizer = Qwen3_5Tokenizer.from_pretrained("tokenizer")
|
| 314 |
scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained("scheduler")
|
|
|
|
| 398 |
if args.wandb and images:
|
| 399 |
import wandb
|
| 400 |
wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=0)
|
| 401 |
+
save_samples_to_disk(images, captions, 0)
|
| 402 |
|
| 403 |
for epoch in range(args.epochs):
|
| 404 |
batch_sampler.set_epoch(epoch)
|
|
|
|
| 455 |
if args.wandb:
|
| 456 |
import wandb
|
| 457 |
wandb.log(log_dict, step=global_step)
|
| 458 |
+
if global_step % 10 == 1 or global_step <= 5:
|
| 459 |
+
print(f"step {global_step}: loss={loss.item():.4f} avg_1h={avg_loss:.4f}", flush=True)
|
| 460 |
|
| 461 |
if is_main and vae is not None and (global_step == 5 or (now - last_sample_time) >= sample_interval):
|
| 462 |
images, captions = generate_samples(
|
|
|
|
| 474 |
if args.wandb and images:
|
| 475 |
import wandb
|
| 476 |
wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
|
| 477 |
+
save_samples_to_disk(images, captions, global_step)
|
| 478 |
last_sample_time = now
|
| 479 |
|
| 480 |
if not args.no_save and is_main and (now - last_save_time) >= save_interval:
|
|
|
|
| 503 |
if args.wandb and images:
|
| 504 |
import wandb
|
| 505 |
wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
|
| 506 |
+
save_samples_to_disk(images, captions, global_step)
|
| 507 |
|
| 508 |
if not args.no_save and is_main:
|
| 509 |
accelerator.unwrap_model(unet).save_pretrained(args.unet_path)
|
train_alchemist.log
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
train_test.log
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
flash_attn is installed but failed to import: cannot import name '_wrapped_flash_attn_backward' from 'flash_attn.flash_attn_interface' (/home/recoilme/.local/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py). Falling back to native PyTorch attention.
|
| 2 |
+
Device: cuda
|
| 3 |
+
UNet: 1.495B params
|
| 4 |
+
The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
|
| 5 |
+
|
| 6 |
+
The config attributes {'latents_mean': [-0.03542253375053406, 0.20086465775966644, -0.016413161531090736, -0.0956302210688591, -0.2672063112258911, 0.2609933018684387, -0.07806991040706635, -0.48407721519470215, 0.21844269335269928, -0.1122383326292038, 0.27197545766830444, -0.18958772718906403, 0.18776826560497284, 0.0987580344080925, 0.2837068736553192, -0.4486690163612366, 0.4816776514053345, 0.02947971224784851, -0.1337375044822693, -0.39750921726226807, -0.08513020724058151, -0.054023586213588715, -0.3943594992160797, 0.23918119072914124, -0.12466679513454437, 0.09935147315263748, 0.31858691573143005, 0.48585832118988037, -0.6416525840759277, -0.15164820849895477, -0.4693508744239807, -0.13071806728839874], 'latents_std': [1.5792087316513062, 1.5769503116607666, 1.5864241123199463, 1.6454921960830688, 1.5336694717407227, 1.5587652921676636, 1.5838669538497925, 1.5659377574920654, 1.6860467195510864, 1.5192310810089111, 1.573639988899231, 1.5953549146652222, 1.5271092653274536, 1.6246271133422852, 1.7054023742675781, 1.607722282409668, 1.558642864227295, 1.5824549198150635, 1.6202995777130127, 1.6206320524215698, 1.6379750967025757, 1.6527063846588135, 1.498811960220337, 1.5706247091293335, 1.5854856967926025, 1.4828169345855713, 1.5693111419677734, 1.692481517791748, 1.6409776210784912, 1.6216280460357666, 1.6087706089019775, 1.5776633024215698]} were passed to AsymmetricAutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
|
| 7 |
+
Dataset: 1 samples
|
| 8 |
+
Caption: 'candid photo of young woman sitting at cafe'
|
| 9 |
+
text encoder выгружен из GPU
|
| 10 |
+
GT latent: (1, 32, 80, 56) (image 448x640)
|
| 11 |
+
Сохранён эталон: samples_test/gt.png
|
| 12 |
+
[step 0] PSNR rec=6.26 dB | gen=14.95 dB -> samples_test/gen_0000.png
|
| 13 |
+
|
| 14 |
+
[step 100] чекпоинт сохранён в unet_test
|
| 15 |
+
[step 100] PSNR rec=5.16 dB | gen=16.21 dB -> samples_test/gen_0100.png
|
| 16 |
+
|
| 17 |
+
Финальный чекпоинт: unet_test
|
train_test.py
ADDED
|
@@ -0,0 +1,223 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# train_test.py — тестовое обучение UNet на одной картинке (datasets/test)
|
| 2 |
+
# Гипотеза: может ли UNet выучить одну картинку (дымовой тест обучаемости).
|
| 3 |
+
# Батч 1, lr не трогаем (5e-5). Всё остальное — как в train.py, упрощено.
|
| 4 |
+
import os
|
| 5 |
+
import random
|
| 6 |
+
import gc
|
| 7 |
+
import argparse
|
| 8 |
+
|
| 9 |
+
import torch
|
| 10 |
+
import torch.nn.functional as F
|
| 11 |
+
import numpy as np
|
| 12 |
+
from tqdm import tqdm
|
| 13 |
+
from PIL import Image
|
| 14 |
+
|
| 15 |
+
from diffusers import UNet2DConditionModel, FlowMatchEulerDiscreteScheduler, AsymmetricAutoencoderKL
|
| 16 |
+
from transformers import Qwen3_5Tokenizer, Qwen3_5ForConditionalGeneration
|
| 17 |
+
from datasets import load_from_disk
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def set_seed(seed):
|
| 21 |
+
random.seed(seed)
|
| 22 |
+
np.random.seed(seed)
|
| 23 |
+
torch.manual_seed(seed)
|
| 24 |
+
torch.cuda.manual_seed_all(seed)
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def tensor_to_pil(tensor):
|
| 28 |
+
"""tensor: [C, H, W] в [0, 1]."""
|
| 29 |
+
arr = tensor.clamp(0, 1).cpu().permute(1, 2, 0).numpy()
|
| 30 |
+
return Image.fromarray((arr * 255).astype(np.uint8))
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def parse_args():
|
| 34 |
+
parser = argparse.ArgumentParser(description="SDXS-1B UNet test training (single image)")
|
| 35 |
+
parser.add_argument("--ds-path", type=str, default="datasets/test")
|
| 36 |
+
parser.add_argument("--unet-path", type=str, default="unet_test")
|
| 37 |
+
parser.add_argument("--epochs", type=int, default=100)
|
| 38 |
+
parser.add_argument("--batch-size", type=int, default=1)
|
| 39 |
+
parser.add_argument("--lr", type=float, default=5e-5)
|
| 40 |
+
parser.add_argument("--max-length", type=int, default=250)
|
| 41 |
+
parser.add_argument("--cfg-dropout", type=float, default=0.05, help="вероятность пустого промпта в обучении (для работы CFG)")
|
| 42 |
+
parser.add_argument("--save-every-steps", type=int, default=50)
|
| 43 |
+
parser.add_argument("--sample-every-steps", type=int, default=50)
|
| 44 |
+
parser.add_argument("--num-steps", type=int, default=20, help="шагов сэмплинга для проверки")
|
| 45 |
+
parser.add_argument("--seed", type=int, default=42)
|
| 46 |
+
return parser.parse_args()
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
@torch.no_grad()
|
| 50 |
+
def encode_text(text_encoder, tokenizer, text, device, max_length):
|
| 51 |
+
messages = [{"role": "user", "content": [{"type": "text", "text": text}]}]
|
| 52 |
+
formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)
|
| 53 |
+
toks = tokenizer([formatted], padding="max_length", max_length=max_length, truncation=True, return_tensors="pt").to(device)
|
| 54 |
+
out = text_encoder(input_ids=toks.input_ids, attention_mask=toks.attention_mask, output_hidden_states=True)
|
| 55 |
+
emb = out.hidden_states[-2].to(torch.float32)
|
| 56 |
+
return emb, toks.attention_mask.to(torch.int64)
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
@torch.no_grad()
|
| 60 |
+
def generate(unet, emb, mask, scheduler, latents, num_steps, device, cfg=4.0):
|
| 61 |
+
"""Денойзинг из заданных латентов с CFG (uncond_emb/mask — глобальные)."""
|
| 62 |
+
scheduler.set_timesteps(num_steps, device=device)
|
| 63 |
+
for t in scheduler.timesteps:
|
| 64 |
+
if cfg > 1.0:
|
| 65 |
+
lat_in = torch.cat([latents, latents], dim=0)
|
| 66 |
+
emb_in = torch.cat([UNCOND_EMB, emb], dim=0)
|
| 67 |
+
mask_in = torch.cat([UNCOND_MASK, mask], dim=0)
|
| 68 |
+
pred = unet(lat_in, t, encoder_hidden_states=emb_in, encoder_attention_mask=mask_in).sample
|
| 69 |
+
pred_u, pred_c = pred.chunk(2)
|
| 70 |
+
pred = pred_u + cfg * (pred_c - pred_u)
|
| 71 |
+
else:
|
| 72 |
+
pred = unet(latents, t, encoder_hidden_states=emb, encoder_attention_mask=mask).sample
|
| 73 |
+
latents = scheduler.step(pred, t, latents).prev_sample
|
| 74 |
+
return latents
|
| 75 |
+
|
| 76 |
+
UNCOND_EMB = None
|
| 77 |
+
UNCOND_MASK = None
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
@torch.no_grad()
|
| 81 |
+
def decode_latents(vae, latents, mean_t, std_t, device):
|
| 82 |
+
latents = latents * std_t.to(latents.device) + mean_t.to(latents.device)
|
| 83 |
+
decoded = vae.decode(latents.to(vae.dtype)).sample
|
| 84 |
+
return (decoded.float() / 2 + 0.5).clamp(0, 1)
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def sample_and_save(step, unet, emb, mask, scheduler, vae, latents_gt, mean_t, std_t, device, num_steps, gen_generator):
|
| 88 |
+
"""Генерирует gen (из шума, фиксированный seed) и rec (из 50% шума), сохраняет PNG + PSNR."""
|
| 89 |
+
unet.eval()
|
| 90 |
+
gen = generate(unet, emb, mask, scheduler, torch.randn(latents_gt.shape, device=latents_gt.device, dtype=latents_gt.dtype, generator=gen_generator), num_steps, device)
|
| 91 |
+
rec = generate(unet, emb, mask, scheduler, 0.5 * latents_gt + 0.5 * torch.randn_like(latents_gt), num_steps, device)
|
| 92 |
+
|
| 93 |
+
img_gen = tensor_to_pil(decode_latents(vae, gen, mean_t, std_t, device)[0])
|
| 94 |
+
img_rec = tensor_to_pil(decode_latents(vae, rec, mean_t, std_t, device)[0])
|
| 95 |
+
img_gen.save(f"samples_test/gen_{step:04d}.png")
|
| 96 |
+
img_rec.save(f"samples_test/rec_{step:04d}.png")
|
| 97 |
+
|
| 98 |
+
psnr_rec = 10 * torch.log10(1.0 / (F.mse_loss(rec, latents_gt) + 1e-8)).item()
|
| 99 |
+
psnr_gen = 10 * torch.log10(1.0 / (F.mse_loss(gen, latents_gt) + 1e-8)).item()
|
| 100 |
+
print(f"[step {step}] PSNR rec={psnr_rec:.2f} dB | gen={psnr_gen:.2f} dB -> samples_test/gen_{step:04d}.png")
|
| 101 |
+
unet.train()
|
| 102 |
+
return psnr_rec, psnr_gen
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
def main():
|
| 106 |
+
args = parse_args()
|
| 107 |
+
set_seed(args.seed)
|
| 108 |
+
|
| 109 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 110 |
+
print(f"Device: {device}")
|
| 111 |
+
|
| 112 |
+
# UNet (fp32, как в train.py)
|
| 113 |
+
unet = UNet2DConditionModel.from_pretrained(args.unet_path, torch_dtype=torch.float32).to(device)
|
| 114 |
+
print(f"UNet: {sum(p.numel() for p in unet.parameters())/1e9:.3f}B params")
|
| 115 |
+
unet.enable_gradient_checkpointing()
|
| 116 |
+
|
| 117 |
+
# text encoder — только чтобы посчитать эмбеддинг, потом выгружаем
|
| 118 |
+
text_encoder = Qwen3_5ForConditionalGeneration.from_pretrained("text_encoder", torch_dtype=torch.float16).to(device)
|
| 119 |
+
tokenizer = Qwen3_5Tokenizer.from_pretrained("tokenizer")
|
| 120 |
+
scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained("scheduler")
|
| 121 |
+
|
| 122 |
+
# VAE для декодирования при проверке
|
| 123 |
+
vae = AsymmetricAutoencoderKL.from_pretrained("vae", torch_dtype=torch.float16).to(device).eval()
|
| 124 |
+
|
| 125 |
+
mean = getattr(vae.config, "latents_mean", None)
|
| 126 |
+
std = getattr(vae.config, "latents_std", None)
|
| 127 |
+
if mean is not None and std is not None:
|
| 128 |
+
mean_t = torch.tensor(mean, device=device, dtype=torch.float32).view(1, -1, 1, 1)
|
| 129 |
+
std_t = torch.tensor(std, device=device, dtype=torch.float32).view(1, -1, 1, 1)
|
| 130 |
+
else:
|
| 131 |
+
mean_t, std_t = 0.0, 1.0
|
| 132 |
+
|
| 133 |
+
# ---------- датасет ----------
|
| 134 |
+
dataset = load_from_disk(args.ds_path)
|
| 135 |
+
print(f"Dataset: {len(dataset)} samples")
|
| 136 |
+
|
| 137 |
+
# эмбеддинги текста — один раз, до обучения
|
| 138 |
+
ds_text = dataset["text"][0]
|
| 139 |
+
print(f"Caption: {ds_text!r}")
|
| 140 |
+
emb, mask = encode_text(text_encoder, tokenizer, ds_text, device, args.max_length)
|
| 141 |
+
emb = emb.to(device)
|
| 142 |
+
mask = mask.to(device)
|
| 143 |
+
uncond_emb, uncond_mask = encode_text(text_encoder, tokenizer, "", device, args.max_length)
|
| 144 |
+
global UNCOND_EMB, UNCOND_MASK
|
| 145 |
+
UNCOND_EMB = uncond_emb.to(device)
|
| 146 |
+
UNCOND_MASK = uncond_mask.to(device)
|
| 147 |
+
|
| 148 |
+
# выгружаем text encoder — он больше не нужен
|
| 149 |
+
del text_encoder, tokenizer
|
| 150 |
+
gc.collect()
|
| 151 |
+
torch.cuda.empty_cache()
|
| 152 |
+
print("text encoder выгружен из GPU")
|
| 153 |
+
|
| 154 |
+
latents_gt = torch.from_numpy(np.array(dataset[0]["vae"], dtype=np.float32)).unsqueeze(0).to(device)
|
| 155 |
+
h, w = dataset["height"][0], dataset["width"][0]
|
| 156 |
+
print(f"GT latent: {tuple(latents_gt.shape)} (image {w}x{h})")
|
| 157 |
+
|
| 158 |
+
# ---------- оптимизатор ----------
|
| 159 |
+
import bitsandbytes as bnb
|
| 160 |
+
optimizer = bnb.optim.AdamW8bit(
|
| 161 |
+
unet.parameters(), lr=args.lr, betas=(0.9, 0.997), eps=1e-7, weight_decay=0.001
|
| 162 |
+
)
|
| 163 |
+
|
| 164 |
+
os.makedirs("samples_test", exist_ok=True)
|
| 165 |
+
gen_generator = torch.Generator(device=device).manual_seed(0) # фиксированный шум для gen
|
| 166 |
+
|
| 167 |
+
# ---------- эталон + сэмплы до обучения (step 0) ----------
|
| 168 |
+
gt_pil = tensor_to_pil(decode_latents(vae, latents_gt, mean_t, std_t, device)[0])
|
| 169 |
+
gt_pil.save("samples_test/gt.png")
|
| 170 |
+
print("Сохранён эталон: samples_test/gt.png")
|
| 171 |
+
sample_and_save(0, unet, emb, mask, scheduler, vae, latents_gt, mean_t, std_t, device, args.num_steps, gen_generator)
|
| 172 |
+
|
| 173 |
+
unet.train()
|
| 174 |
+
global_step = 0
|
| 175 |
+
pbar = tqdm(total=args.epochs, desc="train_test")
|
| 176 |
+
|
| 177 |
+
for epoch in range(args.epochs):
|
| 178 |
+
for _ in range(1): # одна картинка -> один шаг на эпоху
|
| 179 |
+
latents = latents_gt
|
| 180 |
+
|
| 181 |
+
noise = torch.randn_like(latents)
|
| 182 |
+
t = torch.rand(latents.shape[0], device=device, dtype=torch.float32)
|
| 183 |
+
t_bc = t.view(-1, 1, 1, 1)
|
| 184 |
+
|
| 185 |
+
# CFG dropout: с вероятностью cfg_dropout учимся на пустом промпте
|
| 186 |
+
if random.random() < args.cfg_dropout:
|
| 187 |
+
emb_use, mask_use = UNCOND_EMB, UNCOND_MASK
|
| 188 |
+
else:
|
| 189 |
+
emb_use, mask_use = emb, mask
|
| 190 |
+
|
| 191 |
+
noisy = (1.0 - t_bc) * latents + t_bc * noise
|
| 192 |
+
timesteps = (t * 1000).to(torch.float32)
|
| 193 |
+
|
| 194 |
+
pred = unet(noisy, timesteps, encoder_hidden_states=emb_use, encoder_attention_mask=mask_use).sample
|
| 195 |
+
target = noise - latents
|
| 196 |
+
loss = F.mse_loss(pred.float(), target.float())
|
| 197 |
+
|
| 198 |
+
loss.backward()
|
| 199 |
+
torch.nn.utils.clip_grad_norm_(unet.parameters(), 1.0)
|
| 200 |
+
optimizer.step()
|
| 201 |
+
optimizer.zero_grad(set_to_none=True)
|
| 202 |
+
|
| 203 |
+
global_step += 1
|
| 204 |
+
pbar.update(1)
|
| 205 |
+
pbar.set_postfix(loss=f"{loss.item():.4f}")
|
| 206 |
+
|
| 207 |
+
# ---------- проверка ----------
|
| 208 |
+
if global_step % args.save_every_steps == 0:
|
| 209 |
+
unet.save_pretrained(args.unet_path)
|
| 210 |
+
print(f"\n[step {global_step}] чекпоинт сохранён в {args.unet_path}")
|
| 211 |
+
|
| 212 |
+
if global_step % args.sample_every_steps == 0:
|
| 213 |
+
sample_and_save(global_step, unet, emb, mask, scheduler, vae, latents_gt, mean_t, std_t, device, args.num_steps, gen_generator)
|
| 214 |
+
|
| 215 |
+
pbar.close()
|
| 216 |
+
|
| 217 |
+
unet.save_pretrained(args.unet_path)
|
| 218 |
+
print(f"Финальный чекпоинт: {args.unet_path}")
|
| 219 |
+
torch.cuda.synchronize()
|
| 220 |
+
|
| 221 |
+
|
| 222 |
+
if __name__ == "__main__":
|
| 223 |
+
main()
|
unet/config.json
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a222ca0472aa440909a7ebb4187ae2f2094cac543abd8072e82c981a0fcc93ea
|
| 3 |
+
size 1915
|
unet/diffusion_pytorch_model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f2991a2970f1dfed537fae22edeb55a3596fd37a0ab100dc18683896440ae8ab
|
| 3 |
+
size 3376130168
|