Text-to-Image
Diffusers
Safetensors
recoilme commited on
Commit
5ae8af8
·
1 Parent(s): b58ae02
BASELINE_TEST.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Baseline: тест обучаемости UNet на одной картинке
2
+
3
+ > Дата: 2026-08-04 · Скрипт: `train_test.py` · Гипотеза: может ли UNet выучить одну картинку
4
+ > Веса: `unet_test/` · Сэмплы: `samples_test/` · Лог: `train_test.log`
5
+
6
+ ## Конфигурация
7
+
8
+ | Параметр | Значение |
9
+ |---|---|
10
+ | Модель | UNet 1.495B (копия `unet/` → `unet_test/`) |
11
+ | Датасет | `datasets/test` (1 сэмпл: фото 1086×1448 → кроп 448×640) |
12
+ | Латент GT | (1, 32, 80, 56) |
13
+ | Промпт | "candid photo of young woman sitting at cafe" |
14
+ | Шагов обучения | **1001** (1 шаг = 1 эпоха, батч 1) |
15
+ | lr | 5e-5 (AdamW8bit, betas 0.9/0.997, wd 0.001, clip 1.0) |
16
+ | Loss | Flow matching velocity, MSE |
17
+ | Сэмплинг проверки | 20 шагов, CFG 4.0, `gen` из чистого шума (seed 0), `rec` из 50% шума |
18
+ | Сэмплы на шагах | 0, 500, 1000 |
19
+
20
+ ## Метрики
21
+
22
+ ### PSNR (в латентном пространстве, vs GT)
23
+
24
+ | Шаг | PSNR rec, dB | PSNR gen, dB |
25
+ |---|---|---|
26
+ | 0 | 0.43 | −3.00 |
27
+ | 500 | 3.97 | **12.65** |
28
+ | 1000 | **5.95** | **15.07** |
29
+
30
+ **Тренд:** модель радикально учит картинку. `gen` (генерация из чистого шума) растёт с −3.00 до +15.07 dB — UNet запоминает изображение и воспроизводит его по промпту из ничего. `rec` растёт медленнее (3.97 → 5.95) — при 50%-шумном старте 20 шагов денойза не успевают сойтись к GT так близко, как прямая генерация.
31
+
32
+ ### Loss (velocity MSE, каждый 50-й шаг)
33
+
34
+ | Шаг | Loss | Шаг | Loss |
35
+ |---|---|---|---|
36
+ | 1 | 0.9371 | 501 | 0.0473 |
37
+ | 26 | 0.8524 | 526 | 0.0381 |
38
+ | 51 | 1.1081 | 551 | 0.0567 |
39
+ | 76 | 1.0715 | 576 | 0.0305 |
40
+ | 101 | 0.8832 | 601 | 0.1005 |
41
+ | 126 | 0.4898 | 626 | 0.0297 |
42
+ | 151 | 0.7161 | 651 | 0.1560 |
43
+ | 176 | 0.3995 | 676 | 0.0338 |
44
+ | 201 | 0.4137 | 701 | 0.0215 |
45
+ | 226 | 0.1955 | 726 | 0.0757 |
46
+ | 251 | 0.1949 | 751 | 0.0334 |
47
+ | 276 | 0.1310 | 776 | 0.0215 |
48
+ | 301 | 0.1201 | 801 | 0.0156 |
49
+ | 326 | 0.7008 | 826 | 0.0163 |
50
+ | 351 | 0.2478 | 851 | 0.0245 |
51
+ | 376 | 0.3107 | 876 | 0.0281 |
52
+ | 401 | 0.0922 | 901 | 0.0187 |
53
+ | 426 | 0.0585 | 926 | 0.0260 |
54
+ | 451 | 1.0274 | 951 | 0.0256 |
55
+ | 476 | 0.1101 | 976 | 0.0264 |
56
+ | | | 1001 | 0.0511 |
57
+
58
+ min loss: **0.0132** (фаза активного заучивания ~шаги 700–900), loss снизился на 2 порядка: 1.27 → 0.01–0.05.
59
+
60
+ ## Ресурсы
61
+
62
+ - ~1.3 с/шаг обучения · сэмплинг ~60 с на проверку · итого ~28 мин на 1001 шаг
63
+ - Память: ~15.1 / 16.4 GB (UNet fp32 + AdamW8bit + VAE fp16; text encoder выгружен после кодирования)
64
+
65
+ ## Артефакты
66
+
67
+ - `unet_test/diffusion_pytorch_model.safetensors` — веса после 1001 шага (08:28)
68
+ - `samples_test/gt.png` — эталон (декодированный GT)
69
+ - `samples_test/rec_0000/0500/1000.png` — реконструкция из 50% шума
70
+ - `samples_test/gen_0000/0500/1000.png` — генерация из чистого шума (seed 0)
71
+
72
+ ## Выводы
73
+
74
+ 1. **Гипотеза подтверждена**: UNet способен выучить одну картинку. За 1001 шаг при lr 5e-5 loss упал на 2 порядка (до 0.013), генерация из чистого шума достигает **+15 dB PSNR** к оригиналу.
75
+ 2. **Порог заучивания** — примерно между шагами 200–500: loss переходит в область <0.2, `gen` скачет с −3 до +12.65 dB.
76
+ 3. **gen > rec** — при «заученной» картинке прямая генерация ближе к GT, чем 20-шаговая реконструкция из 50% шума (последняя просто не успевает сойтись).
77
+ 4. Дальнейший рост (1000+ шагов, до +20+ dB) возможен, но с убывающей отдачей.
78
+
79
+ ## Воспроизведение
80
+
81
+ ```bash
82
+ cp -r unet/. unet_test/ # сброс к исходным весам
83
+ python3 train_test.py --epochs 1001 --sample-every-steps 500 --save-every-steps 500
84
+ ```
dataset.py CHANGED
@@ -33,8 +33,8 @@ step = 64 #32
33
  empty_share = 0.0
34
  limit = 0
35
 
36
- folder_path = "datasets/butterfly"
37
- save_path = "datasets/butterfly_704_vae8x16x"
38
  os.makedirs(save_path, exist_ok=True)
39
 
40
  def clear_cuda_memory():
 
33
  empty_share = 0.0
34
  limit = 0
35
 
36
+ folder_path = "/home/recoilme/dataset/alchemist"
37
+ save_path = "datasets/alchemist"
38
  os.makedirs(save_path, exist_ok=True)
39
 
40
  def clear_cuda_memory():
dataset_alchemist.log ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ flash_attn is installed but failed to import: cannot import name '_wrapped_flash_attn_backward' from 'flash_attn.flash_attn_interface' (/home/recoilme/.local/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py). Falling back to native PyTorch attention.
2
+ [GPU 0] Загрузка моделей...
3
+ The config attributes {'latents_mean': [-0.03542253375053406, 0.20086465775966644, -0.016413161531090736, -0.0956302210688591, -0.2672063112258911, 0.2609933018684387, -0.07806991040706635, -0.48407721519470215, 0.21844269335269928, -0.1122383326292038, 0.27197545766830444, -0.18958772718906403, 0.18776826560497284, 0.0987580344080925, 0.2837068736553192, -0.4486690163612366, 0.4816776514053345, 0.02947971224784851, -0.1337375044822693, -0.39750921726226807, -0.08513020724058151, -0.054023586213588715, -0.3943594992160797, 0.23918119072914124, -0.12466679513454437, 0.09935147315263748, 0.31858691573143005, 0.48585832118988037, -0.6416525840759277, -0.15164820849895477, -0.4693508744239807, -0.13071806728839874], 'latents_std': [1.5792087316513062, 1.5769503116607666, 1.5864241123199463, 1.6454921960830688, 1.5336694717407227, 1.5587652921676636, 1.5838669538497925, 1.5659377574920654, 1.6860467195510864, 1.5192310810089111, 1.573639988899231, 1.5953549146652222, 1.5271092653274536, 1.6246271133422852, 1.7054023742675781, 1.607722282409668, 1.558642864227295, 1.5824549198150635, 1.6202995777130127, 1.6206320524215698, 1.6379750967025757, 1.6527063846588135, 1.498811960220337, 1.5706247091293335, 1.5854856967926025, 1.4828169345855713, 1.5693111419677734, 1.692481517791748, 1.6409776210784912, 1.6216280460357666, 1.6087706089019775, 1.5776633024215698]} were passed to AsymmetricAutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
4
+ Найдено 3350 изображений
5
+ [GPU 0] обрабатывает 3350 файлов
6
+
7
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_320x640_proc_0
8
+ [GPU 0] used_gb: 1.37 GB
9
+
10
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_384x640_proc_0
11
+ [GPU 0] used_gb: 1.60 GB
12
+
13
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_448x640_proc_0
14
+ [GPU 0] used_gb: 1.84 GB
15
+
16
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_512x640_proc_0
17
+ [GPU 0] used_gb: 2.07 GB
18
+
19
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_576x640_proc_0
20
+ [GPU 0] used_gb: 2.31 GB
21
+
22
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x320_proc_0
23
+ [GPU 0] used_gb: 2.31 GB
24
+
25
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x384_proc_0
26
+ [GPU 0] used_gb: 2.31 GB
27
+
28
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x448_proc_0
29
+ [GPU 0] used_gb: 2.31 GB
30
+
31
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x512_proc_0
32
+ [GPU 0] used_gb: 2.31 GB
33
+
34
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x576_proc_0
35
+ [GPU 0] used_gb: 2.31 GB
36
+
37
+ [GPU 0] Saved: datasets/alchemist_temp/chunk_1_640x640_proc_0
38
+ [GPU 0] used_gb: 2.54 GB
39
+
40
+ ✅ Сохранено
dataset_test.py ADDED
@@ -0,0 +1,126 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # dataset_test.py — генерация датасета из одной картинки (тест UNet)
2
+ # Использование: python dataset_test.py
3
+ import torch
4
+ import os
5
+ import gc
6
+ import numpy as np
7
+ import shutil
8
+
9
+ from datasets import Dataset
10
+ from diffusers import AsymmetricAutoencoderKL
11
+ from torchvision.transforms import ToTensor, Normalize
12
+ from PIL import Image
13
+
14
+ # ---------------- 1️⃣ Настройки ----------------
15
+ dtype = torch.float16
16
+ min_size = 320
17
+ max_size = 640
18
+ step = 64
19
+ empty_share = 0.0
20
+
21
+ folder_path = "datasets/example"
22
+ save_path = "datasets/test"
23
+
24
+ # ---------------- 2️⃣ Модель ----------------
25
+ device = "cuda" if torch.cuda.is_available() else "cpu"
26
+ vae = AsymmetricAutoencoderKL.from_pretrained("vae", torch_dtype=dtype).to(device).eval()
27
+
28
+ shift_factor = getattr(vae.config, "shift_factor", 0.0) or 0.0
29
+ scaling_factor = getattr(vae.config, "scaling_factor", 1.0) or 1.0
30
+
31
+ mean = getattr(vae.config, "latents_mean", None)
32
+ std = getattr(vae.config, "latents_std", None)
33
+ has_mean_std = mean is not None and std is not None
34
+ if has_mean_std:
35
+ latents_std = torch.tensor(std, device=device, dtype=dtype).view(1, len(std), 1, 1)
36
+ latents_mean = torch.tensor(mean, device=device, dtype=dtype).view(1, len(mean), 1, 1)
37
+ else:
38
+ latents_std, latents_mean = 1.0, 0.0
39
+
40
+ # ---------------- 3️⃣ Трансформация (та же логика, что в dataset.py) ----------------
41
+ def transform(img):
42
+ original_width, original_height = img.size
43
+
44
+ if original_width >= original_height:
45
+ new_width = max_size
46
+ new_height = int(max_size * original_height / original_width)
47
+ else:
48
+ new_height = max_size
49
+ new_width = int(max_size * original_width / original_height)
50
+
51
+ if new_height < min_size or new_width < min_size:
52
+ if original_width <= original_height:
53
+ new_width = min_size
54
+ new_height = int(min_size * original_height / original_width)
55
+ else:
56
+ new_height = min_size
57
+ new_width = int(min_size * original_width / original_height)
58
+
59
+ crop_width = min(max_size, (new_width // step) * step)
60
+ crop_height = min(max_size, (new_height // step) * step)
61
+ crop_width = max(min_size, crop_width)
62
+ crop_height = max(min_size, crop_height)
63
+
64
+ img_resized = img.convert("RGB").resize((new_width, new_height), Image.LANCZOS)
65
+ top = (new_height - crop_height) // 3
66
+ left = 0
67
+ img_cropped = img_resized.crop((left, top, left + crop_width, top + crop_height))
68
+
69
+ img_tensor = ToTensor()(img_cropped)
70
+ img_tensor = Normalize(mean=[0.5] * 3, std=[0.5] * 3)(img_tensor)
71
+ return img_tensor, img_cropped, img_cropped.size[0], img_cropped.size[1]
72
+
73
+ # ---------------- 4️⃣ Обработка папки ----------------
74
+ image_paths, text_paths = [], []
75
+ for root, _, files in os.walk(folder_path):
76
+ for filename in sorted(files):
77
+ if filename.lower().endswith((".jpg", ".jpeg", ".png")):
78
+ image_path = os.path.join(root, filename)
79
+ text_path = os.path.splitext(image_path)[0] + ".txt"
80
+ if os.path.exists(text_path):
81
+ image_paths.append(image_path)
82
+ text_paths.append(text_path)
83
+
84
+ print(f"Найдено {len(image_paths)} изображений в {folder_path}")
85
+
86
+ # ---------------- 5️⃣ Энкодинг ----------------
87
+ latents_all, texts_all, widths_all, heights_all = [], [], [], []
88
+
89
+ for img_path, txt_path in zip(image_paths, text_paths):
90
+ img = Image.open(img_path).convert("RGB")
91
+ t_img, img_cropped, w, h = transform(img)
92
+ print(f" {os.path.basename(img_path)}: {img.size} -> crop {w}x{h}")
93
+
94
+ batch_tensor = t_img.unsqueeze(0).to(device, dtype)
95
+ with torch.no_grad():
96
+ posteriors = vae.encode(batch_tensor).latent_dist.mode()
97
+ if has_mean_std:
98
+ posteriors = (posteriors - latents_mean) / latents_std
99
+ posteriors = (posteriors - shift_factor) / scaling_factor
100
+
101
+ latents_all.append(posteriors.cpu().numpy()[0])
102
+
103
+ with open(txt_path, "r", encoding="utf-8") as f:
104
+ text = f.read().strip()
105
+ if empty_share > 0 and np.random.random() < empty_share:
106
+ text = ""
107
+ texts_all.append(text)
108
+ widths_all.append(w)
109
+ heights_all.append(h)
110
+
111
+ # ---------------- 6️⃣ Сохранение ----------------
112
+ if os.path.exists(save_path):
113
+ shutil.rmtree(save_path)
114
+ os.makedirs(save_path, exist_ok=True)
115
+
116
+ ds = Dataset.from_dict({
117
+ "vae": latents_all,
118
+ "text": texts_all,
119
+ "width": widths_all,
120
+ "height": heights_all,
121
+ })
122
+ ds.save_to_disk(save_path)
123
+
124
+ print(f"✅ Датасет сохранён: {save_path} | {len(ds)} сэмплов")
125
+ print(" Размеры:", ds["width"], ds["height"])
126
+ print(" Текст:", ds["text"])
generate.py ADDED
@@ -0,0 +1,124 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ SDXS-1B image generation from prompts.txt.
3
+
4
+ Usage:
5
+ python generate.py # generates grid from prompts.txt
6
+ python generate.py --prompts my_list.txt # custom prompts file
7
+ python generate.py --prompt "your text" # single prompt (без файла)
8
+ python generate.py --index 0 # только промпт #0 из prompts.txt
9
+ """
10
+ import argparse
11
+ import os
12
+ import math
13
+ import textwrap
14
+
15
+ import torch
16
+ import numpy as np
17
+
18
+ from pipeline_sdxs import SdxsPipeline
19
+
20
+ NEGATIVE_PROMPT = (
21
+ "bad quality grainy image with low details, incomplete text, "
22
+ "despite numerous technical flaws and distorted figures"
23
+ )
24
+
25
+ DEFAULT_PROMPTS_FILE = "prompts.txt"
26
+ GRID_OUTPUT = "media/result_grid.jpg"
27
+
28
+
29
+ def main():
30
+ parser = argparse.ArgumentParser(description="SDXS-1B generation")
31
+ parser.add_argument("--prompts", type=str, default=None, help="prompts file")
32
+ parser.add_argument("--prompt", type=str, default=None, help="single prompt")
33
+ parser.add_argument("--index", type=int, default=None, help="generate only prompt #N from file")
34
+ parser.add_argument("--steps", type=int, default=40)
35
+ parser.add_argument("--guidance", type=float, default=5.0)
36
+ parser.add_argument("--seed", type=int, default=0)
37
+ parser.add_argument("--width", type=int, default=1024)
38
+ parser.add_argument("--height", type=int, default=1408)
39
+ parser.add_argument("--cols", type=int, default=4, help="колонки в гриде")
40
+ parser.add_argument("--output", type=str, default=None,
41
+ help="папка для отдельных картинок (по умолчанию не сохраняются)")
42
+ args = parser.parse_args()
43
+
44
+ # ---------- сбор промптов ----------
45
+ prompts = None
46
+ if args.prompt:
47
+ prompts = [args.prompt]
48
+ else:
49
+ path = args.prompts or DEFAULT_PROMPTS_FILE
50
+ if not os.path.exists(path):
51
+ print(f"Файл промптов не найден: {path}")
52
+ return
53
+ with open(path, encoding="utf-8") as f:
54
+ prompts = [l.strip() for l in f if l.strip()]
55
+
56
+ if args.index is not None:
57
+ if args.index >= len(prompts):
58
+ print(f"Нет промпта #{args.index} (всего {len(prompts)})")
59
+ return
60
+ prompts = [prompts[args.index]]
61
+
62
+ if not prompts:
63
+ print("Нет промптов")
64
+ return
65
+
66
+ print(f"Промптов: {len(prompts)}")
67
+
68
+ # ---------- пайплайн ----------
69
+ device = "cuda" if torch.cuda.is_available() else "cpu"
70
+ pipe = SdxsPipeline.from_pretrained(
71
+ os.path.dirname(os.path.abspath(__file__)),
72
+ torch_dtype=torch.float16,
73
+ ).to(device)
74
+
75
+ # ---------- генерация ----------
76
+ images = []
77
+ for i, prompt in enumerate(prompts):
78
+ print(f"[{i+1}/{len(prompts)}] {prompt[:80]}...")
79
+ image = pipe(
80
+ prompt=prompt,
81
+ negative_prompt=NEGATIVE_PROMPT,
82
+ guidance_scale=args.guidance,
83
+ width=args.width,
84
+ height=args.height,
85
+ seed=args.seed,
86
+ num_inference_steps=args.steps,
87
+ )[0][0]
88
+ images.append(image)
89
+
90
+ if args.output:
91
+ os.makedirs(args.output, exist_ok=True)
92
+ path = os.path.join(args.output, f"{i+1:03d}.png")
93
+ image.save(path)
94
+ print(f" saved {path}")
95
+
96
+ # ---------- грид ----------
97
+ os.makedirs(os.path.dirname(GRID_OUTPUT), exist_ok=True)
98
+
99
+ import matplotlib
100
+ matplotlib.use("Agg")
101
+ import matplotlib.pyplot as plt
102
+
103
+ cols = min(args.cols, len(images))
104
+ rows = math.ceil(len(images) / cols)
105
+ fig, axes = plt.subplots(rows, cols, figsize=(cols * 4, rows * 4.5), constrained_layout=True)
106
+ axes = list(np.array(axes).flatten())[:len(images)]
107
+
108
+ for i, (img, prompt) in enumerate(zip(images, prompts)):
109
+ ax = axes[i]
110
+ ax.imshow(img)
111
+ ax.axis("off")
112
+ ax.set_aspect("equal")
113
+ text = (prompt[:200] + "…") if len(prompt) > 200 else prompt
114
+ lines = textwrap.wrap(text, width=35)
115
+ while len(lines) < 4:
116
+ lines.append("")
117
+ ax.set_title("\n".join(lines), fontsize=9, pad=8)
118
+
119
+ plt.savefig(GRID_OUTPUT, bbox_inches="tight", dpi=150, format="jpeg")
120
+ print(f"grid -> {GRID_OUTPUT}")
121
+
122
+
123
+ if __name__ == "__main__":
124
+ main()
media/result_grid.jpg CHANGED

Git LFS Details

  • SHA256: bceac77a5e7cccdf2c15eeed5dc26b60ecf2035f6fb54aa56b2c1d8949711756
  • Pointer size: 132 Bytes
  • Size of remote file: 7.99 MB

Git LFS Details

  • SHA256: 2dd1b34fa5dd7b3aedd2dc4d7618c19c94281a2e42b041d206baf6419a3982ea
  • Pointer size: 131 Bytes
  • Size of remote file: 466 kB
migrate_test.py ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # migrate_test.py — миграция весов unet → unet_test с новым распределением трансформеров
2
+ # Эксперимент: T=[4,3,2,1] → T=[2,2,3,4] (внимание с верха в горлышко), L=[4,3,2,2] без изменений
3
+ import os, json, torch, re
4
+ from safetensors.torch import load_file, save_file
5
+ from diffusers import UNet2DConditionModel
6
+
7
+ SRC_DIR = "unet"
8
+ DST_DIR = "unet_test"
9
+
10
+ # Переопределения конфига (только то, что меняем; остальное — из исходного)
11
+ OVERRIDES = {
12
+ "transformer_layers_per_block": [2, 2, 3, 4],
13
+ }
14
+
15
+ os.makedirs(DST_DIR, exist_ok=True)
16
+
17
+ # 1. Конфиг: исходный + оверрайды
18
+ with open(os.path.join(SRC_DIR, "config.json"), "r", encoding="utf-8") as f:
19
+ cfg = json.load(f)
20
+
21
+ for k in ["_class_name", "_diffusers_version", "_name_or_path"]:
22
+ cfg.pop(k, None)
23
+
24
+ cfg.update(OVERRIDES)
25
+
26
+ with open(os.path.join(DST_DIR, "config.json"), "w", encoding="utf-8") as f:
27
+ json.dump({
28
+ "_class_name": "UNet2DConditionModel",
29
+ "_diffusers_version": "0.38.0",
30
+ "_name_or_path": "unet_test",
31
+ **cfg,
32
+ }, f, indent=2)
33
+
34
+ print(f"Конфиг: L={cfg['layers_per_block']} T={cfg['transformer_layers_per_block']}")
35
+
36
+ # 2. Загружаем старые веса
37
+ print("Загрузка старых весов...")
38
+ old_state = load_file(os.path.join(SRC_DIR, "diffusion_pytorch_model.safetensors"), device="cpu")
39
+
40
+ # 3. Новая модель
41
+ print("Создание новой модели...")
42
+ new_model = UNet2DConditionModel(**cfg)
43
+ new_state = new_model.state_dict()
44
+
45
+ old_params = sum(p.numel() for p in old_state.values()) / 1e9
46
+ new_params = sum(p.numel() for p in new_state.values()) / 1e9
47
+ print(f"Старый: {old_params:.3f}B → Новый: {new_params:.3f}B")
48
+
49
+ # 4. Копирование точных совпадений
50
+ copied = 0
51
+ for key, old_t in old_state.items():
52
+ if key in new_state and old_t.shape == new_state[key].shape:
53
+ new_state[key] = old_t.clone()
54
+ copied += 1
55
+
56
+ # 5. Warm-start новых слоёв от ближайших существующих (перебор индексов назад)
57
+ def find_source(old_state, key, new_state):
58
+ m = re.match(r"(.*\.)(\d+)\.(.+)", key)
59
+ if not m:
60
+ return None
61
+ stem, idx_str, suffix = m.group(1), m.group(2), m.group(3)
62
+ idx = int(idx_str)
63
+ for src_idx in range(idx - 1, -1, -1):
64
+ src_key = f"{stem}{src_idx}.{suffix}"
65
+ if src_key in old_state and old_state[src_key].shape == new_state[key].shape:
66
+ return src_key
67
+ return None
68
+
69
+ warm_started = []
70
+ for key in new_state:
71
+ if key not in old_state or old_state[key].shape != new_state[key].shape:
72
+ src_key = find_source(old_state, key, new_state)
73
+ if src_key:
74
+ new_state[key] = old_state[src_key].clone()
75
+ warm_started.append((key, src_key))
76
+
77
+ print(f"Прямых копий: {copied}")
78
+ print(f"Warm-start: {len(warm_started)} ключей")
79
+ for k, src in warm_started[:12]:
80
+ print(f" {k} ← {src}")
81
+ if len(warm_started) > 12:
82
+ print(f" ... и ещё {len(warm_started) - 12}")
83
+
84
+ # 6. Статистика
85
+ total_copied = sum(old_state[k].numel() for k in old_state if k in new_state and old_state[k].shape == new_state[k].shape)
86
+ total_ws = sum(new_state[k].numel() for k, _ in warm_started)
87
+ total_new = sum(v.numel() for v in new_state.values())
88
+ print(f"\nСкопировано: {total_copied/1e9:.3f}B ({total_copied/total_new*100:.1f}%)")
89
+ print(f"Warm-start: {total_ws/1e9:.3f}B ({total_ws/total_new*100:.1f}%)")
90
+ print(f"Random init: {(total_new - total_copied - total_ws)/1e9:.3f}B ({(total_new - total_copied - total_ws)/total_new*100:.1f}%)")
91
+
92
+ # 7. Сохранение
93
+ print("\nСохранение...")
94
+ save_file(new_state, os.path.join(DST_DIR, "diffusion_pytorch_model.safetensors"))
95
+ print(f"Готово → {DST_DIR}/")
prompts.txt ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ A digital illustration of a young male character with white hair, blue eyes, and a dark complexion. He is wearing a white jacket, brown pants, and a brown belt. His hands are in his pockets, and he is wearing a wristwatch. He has a pouch on his belt and is wearing glasses. The background is a simple gradient.
2
+ A young woman with striking blue eyes and pointed ears, adorned with a floral kimono and a tattoo. Her hair is styled in a braid, and she wears a pair of ears.
3
+ A fluffy domestic cat with piercing green eyes sits attentively in a sunlit room filled with natural light, its soft fur reflecting warm hues of orange through golden windows.
4
+ A fierce woman in black-and-white, wearing a spiked iron mask with sharp metallic spikes, sporting a high ponytail, her skin marked with battle-dirt, and eyes that reflect determination and strength.
5
+ A close-up of an astronaut's helmet with frosted, opaque visor, reflecting space's cold, frozen texture, resting on the visor a butterfly with vibrant, intricately patterned wings, and distant stars' faint glow.
6
+ A black BMW M3 sports car with black and yellow rims.
7
+ A young girl in a flowing, vibrant dress, her glowing eyes capturing the warmth of the day, sits on a grassy field, surrounded by anime-style elements.
8
+ A small bluebird with a white breast and black stripe perches on a branch, facing right with a blurred background of trees and bushes.
9
+ космонавт на мотоцикле
10
+ A muscular, topless male with tiger-like ears and a tail stands in a forest, holding a sword and wearing a blue outfit, gazing directly at the viewer.
samples/unet_384x704_0.jpg DELETED

Git LFS Details

  • SHA256: 324f09db2e93c1f43a3a7047a968461b7b771ea9111a767f7969acfda5c85816
  • Pointer size: 131 Bytes
  • Size of remote file: 234 kB
samples/unet_416x704_0.jpg DELETED

Git LFS Details

  • SHA256: 4fded5aae0f06df1fa0312972851b7d6c928147aab396a39731485acb86988ac
  • Pointer size: 131 Bytes
  • Size of remote file: 390 kB
samples/unet_448x704_0.jpg DELETED

Git LFS Details

  • SHA256: 7210354931331bba5ead35a70b78a3c546383b4b93e949c1d1992992607605ce
  • Pointer size: 131 Bytes
  • Size of remote file: 682 kB
samples/unet_480x704_0.jpg DELETED

Git LFS Details

  • SHA256: 35960da2a7017cd3195063a31122ec0132833de4e59ca9ed0067e32365f2c6c5
  • Pointer size: 131 Bytes
  • Size of remote file: 281 kB
samples/unet_512x704_0.jpg DELETED

Git LFS Details

  • SHA256: 73a7adad96d2452137bed0ae7de9c80fc6193b0d6541d43fb963189340ab65b8
  • Pointer size: 131 Bytes
  • Size of remote file: 425 kB
samples/unet_544x704_0.jpg DELETED

Git LFS Details

  • SHA256: 2bf71c763a8a780203ca13bc6fbc588ced0c8cc2f0c6b09454c91746eeba71d6
  • Pointer size: 131 Bytes
  • Size of remote file: 491 kB
samples/unet_576x704_0.jpg DELETED

Git LFS Details

  • SHA256: d78d24a1bbbbb4c3f8f60b520510840045da21ee6712bf63e901bda849eca98c
  • Pointer size: 131 Bytes
  • Size of remote file: 192 kB
samples/unet_608x704_0.jpg DELETED

Git LFS Details

  • SHA256: ee87f4770941074538c7ba791cb486066023a6d17ae6e81ca11aaca1e44f1997
  • Pointer size: 131 Bytes
  • Size of remote file: 356 kB
samples/unet_640x704_0.jpg DELETED

Git LFS Details

  • SHA256: d84607b446d86beb62f93763add980b08e05f412b95dced2f358e5cf032f9228
  • Pointer size: 131 Bytes
  • Size of remote file: 234 kB
samples/unet_672x704_0.jpg DELETED

Git LFS Details

  • SHA256: bbd03c6cc3bee42c9354f797a04fdc6800e7f8f838379225f6f1194b5117e33c
  • Pointer size: 131 Bytes
  • Size of remote file: 660 kB
samples/unet_704x384_0.jpg DELETED

Git LFS Details

  • SHA256: 765135c115f321584f57d13e6602d5e97176ef947dc6207fb60e05f58ec57aac
  • Pointer size: 131 Bytes
  • Size of remote file: 324 kB
samples/unet_704x416_0.jpg DELETED

Git LFS Details

  • SHA256: c0051df464e8bca1eae50ec4b52ad61989975d76869bf687935869ef37067696
  • Pointer size: 130 Bytes
  • Size of remote file: 62.9 kB
samples/unet_704x448_0.jpg DELETED

Git LFS Details

  • SHA256: f6d0714d3b00b78ecd4634c604b2e297793a9d64abcee006fca8f97b524ba815
  • Pointer size: 131 Bytes
  • Size of remote file: 358 kB
samples/unet_704x480_0.jpg DELETED

Git LFS Details

  • SHA256: a8211f9a5f5b27010c6d0fa08ee4eae528bd4a2dac9c2e871bf6ebfd8b2d3040
  • Pointer size: 131 Bytes
  • Size of remote file: 530 kB
samples/unet_704x512_0.jpg DELETED

Git LFS Details

  • SHA256: aad3fc2a31839f3a43e92ab68db0e8a8c858ece9e1d9462b136c95422658a18e
  • Pointer size: 131 Bytes
  • Size of remote file: 529 kB
samples/unet_704x544_0.jpg DELETED

Git LFS Details

  • SHA256: 1675273c431a1cdfba5f9baab9d147d9eb4dbb1eaf08092dc42ee1fc5d197d4c
  • Pointer size: 131 Bytes
  • Size of remote file: 804 kB
samples/unet_704x576_0.jpg DELETED

Git LFS Details

  • SHA256: 9b69aa06edf5b8f4644c4cdcaa0929b95cc64be3bcc9f46d2d28aa2ad477ae77
  • Pointer size: 131 Bytes
  • Size of remote file: 534 kB
samples/unet_704x608_0.jpg DELETED

Git LFS Details

  • SHA256: b0cef69a53de216757794ee36e3bd5a551058b979cc05dd1bd72bd7a687478e4
  • Pointer size: 131 Bytes
  • Size of remote file: 508 kB
samples/unet_704x640_0.jpg DELETED

Git LFS Details

  • SHA256: 7cbdee42345c428c853f2d9680846eba5671119af27041e9929f037b0c33fa86
  • Pointer size: 131 Bytes
  • Size of remote file: 395 kB
samples/unet_704x672_0.jpg DELETED

Git LFS Details

  • SHA256: 48b79d6be7313f5e5da0fd2e3c53c5b3fc24fe39f8d113b5b0db6fdeb68938a3
  • Pointer size: 131 Bytes
  • Size of remote file: 730 kB
samples/unet_704x704_0.jpg DELETED

Git LFS Details

  • SHA256: 5f44128177fe0e201e2f043b704d2b8c6880bc3196e0f03909a8bc32773feda6
  • Pointer size: 131 Bytes
  • Size of remote file: 193 kB
sdservice.py ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ SDXS-1B HTTP-сервис для Go-телеграм-бота (адаптация sdservice.py из sdxs-2b).
3
+
4
+ POST / {"prompt": "..."} → {"images": [base64, base64], "refined_prompts": [...]}
5
+
6
+ Запуск:
7
+ python3 sdservice.py [--port 8989]
8
+ """
9
+ from http.server import BaseHTTPRequestHandler, HTTPServer
10
+ import socketserver
11
+ import io
12
+ import torch
13
+
14
+ # Патч для torch.compile (обход бага accumulated_recompile_limit)
15
+ try:
16
+ import torch.utils._config_module as _cfg
17
+ if hasattr(_cfg, 'ConfigModule') and not hasattr(_cfg.ConfigModule, '_is_patched'):
18
+ original_setattr = _cfg.ConfigModule.__setattr__
19
+
20
+ def patched_setattr(self, name, value):
21
+ if name == 'accumulated_recompile_limit':
22
+ return
23
+ original_setattr(self, name, value)
24
+
25
+ _cfg.ConfigModule.__setattr__ = patched_setattr
26
+ _cfg.ConfigModule._is_patched = True
27
+ except Exception as e:
28
+ print(f"Патч torch config пропущен (некритично): {e}", flush=True)
29
+
30
+ import gc
31
+ import base64
32
+ import json
33
+ import threading
34
+ import time
35
+ import sys
36
+ import argparse
37
+
38
+ from diffusers import DiffusionPipeline
39
+ from PIL import Image
40
+
41
+ # ---------------- 1️⃣ Настройки и константы ----------------
42
+ PIPE1_ID = "/home/recoilme/sdxs-1b"
43
+
44
+ DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
45
+ DTYPE = torch.float16 if torch.cuda.is_available() else torch.float32
46
+
47
+ gpu_lock = threading.Lock()
48
+
49
+ NEGATIVE_PROMPT_1 = (
50
+ "bad quality grainy image with low details, incomplete text, "
51
+ "despite numerous technical flaws and distorted figures"
52
+ )
53
+
54
+ print(f"Загрузка Модели 1 ({PIPE1_ID})...", flush=True)
55
+ pipe1 = DiffusionPipeline.from_pretrained(
56
+ PIPE1_ID,
57
+ torch_dtype=DTYPE,
58
+ trust_remote_code=True
59
+ ).to(DEVICE)
60
+
61
+ pipe1.unet.to(memory_format=torch.channels_last)
62
+
63
+ # torch.compile: если не компилируется — работаем без него
64
+ try:
65
+ print("Компилирую UNet...", flush=True)
66
+ pipe1.unet = torch.compile(pipe1.unet, mode="reduce-overhead", fullgraph=False)
67
+ pipe1.vae = torch.compile(pipe1.vae, mode="reduce-overhead", fullgraph=False)
68
+ except Exception as e:
69
+ print(f"torch.compile не удался, работаем без него: {e}", flush=True)
70
+
71
+ # ---------------- 2️⃣ Вспомогательные функции ----------------
72
+ def encode_images_to_list(images):
73
+ """Возвращает список строк base64 и агрессивно чистит оперативную память"""
74
+ encoded_list = []
75
+ for image in images:
76
+ with io.BytesIO() as buffer:
77
+ image.save(buffer, format='JPEG', quality=97)
78
+ encoded_image = base64.b64encode(buffer.getvalue()).decode('utf-8')
79
+ encoded_list.append(encoded_image)
80
+
81
+ gc.collect()
82
+ return encoded_list
83
+
84
+ def txt2img(prompt_text):
85
+ """
86
+ Инференс на модели. Возвращает (2 картинки, 2 промпта).
87
+ Батч делаем списком промптов (num_images_per_prompt в пайплайне sdxs-1b нет).
88
+ """
89
+ with torch.no_grad():
90
+ # refined = pipe1.refine_prompts(prompt_text)[0] # раскомментируй, чтобы улучшать промпт
91
+ refined = prompt_text
92
+
93
+ if refined == prompt_text:
94
+ # Промпт не изменился -> 2 картинки с одним промптом
95
+ prompts_list = [prompt_text, prompt_text]
96
+ else:
97
+ # Разные промпты -> батч с обоими
98
+ prompts_list = [prompt_text, refined]
99
+
100
+ images = pipe1(
101
+ prompt=prompts_list,
102
+ negative_prompt=[NEGATIVE_PROMPT_1, NEGATIVE_PROMPT_1],
103
+ width=768,
104
+ height=1152,
105
+ num_inference_steps=35,
106
+ ).images
107
+
108
+ return [images[0], images[1]], prompts_list
109
+
110
+ # ---------------- 3️⃣ Серверная часть ----------------
111
+ class ThreadedHTTPServer(socketserver.ThreadingMixIn, HTTPServer):
112
+ daemon_threads = True
113
+
114
+ class RequestHandler(BaseHTTPRequestHandler):
115
+
116
+ def log_message(self, format, *args):
117
+ pass # отключаем стандартный вывод
118
+
119
+ def do_POST(self):
120
+ try:
121
+ content_length = int(self.headers['Content-Length'])
122
+ body = self.rfile.read(content_length)
123
+
124
+ data = json.loads(body.decode('utf-8'))
125
+ p1 = data.get('prompt', data.get('prompt1', ''))
126
+ p2 = data.get('prompt2', '')
127
+ full_prompt = f"{p1} {p2}".strip()
128
+
129
+ prompt_log = (full_prompt[:70] + '..') if len(full_prompt) > 70 else full_prompt
130
+ print(f"Prompt: {prompt_log}", file=sys.stderr, flush=True)
131
+
132
+ start_time = time.time()
133
+
134
+ with gpu_lock:
135
+ images, refined_prompts = txt2img(full_prompt)
136
+
137
+ generation_time = time.time() - start_time
138
+
139
+ base64_images = encode_images_to_list(images)
140
+
141
+ response_data = {
142
+ "images": base64_images,
143
+ "refined_prompts": refined_prompts
144
+ }
145
+
146
+ self.send_response(200)
147
+ self.send_header('Content-type', 'application/json')
148
+ self.end_headers()
149
+ self.wfile.write(json.dumps(response_data).encode('utf-8'))
150
+
151
+ print(f"[{generation_time:.2f}s] | Batch 2 | Prompt: {prompt_log}", file=sys.stderr, flush=True)
152
+
153
+ del images
154
+ del body
155
+
156
+ except Exception as e:
157
+ print(f"[ERROR] Ошибка при обработке запроса: {e}", file=sys.stderr, flush=True)
158
+ self.send_response(500)
159
+ self.send_header('Content-type', 'text/plain')
160
+ self.end_headers()
161
+ self.wfile.write(str(e).encode('utf-8'))
162
+
163
+ def run_server(port):
164
+ server_address = ('', port)
165
+ httpd = ThreadedHTTPServer(server_address, RequestHandler)
166
+
167
+ print("Прогрев модели (ожидайте завершения torch.compile)...", flush=True)
168
+ start_warmup = time.time()
169
+ with gpu_lock:
170
+ try:
171
+ txt2img("warmup prompt, simple test, 1girl")
172
+ print(f"Прогрев завершен за {time.time() - start_warmup:.2f} сек.", flush=True)
173
+ except Exception as e:
174
+ print(f"Ошибка во время прогрева (некритично): {e}", flush=True)
175
+
176
+ print(f'Сервер запущен на порту {port}...', flush=True)
177
+ try:
178
+ httpd.serve_forever()
179
+ except KeyboardInterrupt:
180
+ print("\nСервер остановлен.")
181
+ httpd.server_close()
182
+
183
+ if __name__ == '__main__':
184
+ parser = argparse.ArgumentParser(description="SDXS-1B HTTP сервис")
185
+ parser.add_argument("--port", type=int, default=8989)
186
+ args = parser.parse_args()
187
+ run_server(args.port)
test.ipynb CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b0a4b80dd4edf35a451706cd02f1c02a91f8ca1d3539719f6fc591fd86089945
3
- size 17906175
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af07dcd7f492c3a229adc2742f3a094ccdcfb6c1f551c1ae51a06d809a108001
3
+ size 18066120
train.py CHANGED
@@ -269,6 +269,15 @@ def generate_samples(
269
  return images, captions
270
 
271
 
 
 
 
 
 
 
 
 
 
272
  # ----------------------------- train ------------------------------
273
 
274
  def main():
@@ -277,7 +286,7 @@ def main():
277
  if args.seed is not None:
278
  set_seed(args.seed)
279
 
280
- accelerator = Accelerator(mixed_precision="no")
281
  device = accelerator.device
282
  is_main = accelerator.is_main_process
283
 
@@ -292,14 +301,14 @@ def main():
292
  # UNet: загружаем из unet_path, если там есть веса — продолжаем
293
  if is_main:
294
  print(f"Loading UNet from {args.unet_path}...")
295
- unet = UNet2DConditionModel.from_pretrained(args.unet_path, torch_dtype=torch.float32).to(device)
296
  if is_main:
297
  print(f"UNet loaded: {sum(p.numel() for p in unet.parameters())/1e9:.3f}B params. Continuing training.")
298
  unet.enable_gradient_checkpointing()
299
 
300
  # text encoder всегда на GPU для обучения
301
  text_encoder = Qwen3_5ForConditionalGeneration.from_pretrained(
302
- "text_encoder", torch_dtype=torch.float32
303
  ).to(device).eval()
304
  tokenizer = Qwen3_5Tokenizer.from_pretrained("tokenizer")
305
  scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained("scheduler")
@@ -389,6 +398,7 @@ def main():
389
  if args.wandb and images:
390
  import wandb
391
  wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=0)
 
392
 
393
  for epoch in range(args.epochs):
394
  batch_sampler.set_epoch(epoch)
@@ -445,6 +455,8 @@ def main():
445
  if args.wandb:
446
  import wandb
447
  wandb.log(log_dict, step=global_step)
 
 
448
 
449
  if is_main and vae is not None and (global_step == 5 or (now - last_sample_time) >= sample_interval):
450
  images, captions = generate_samples(
@@ -462,6 +474,7 @@ def main():
462
  if args.wandb and images:
463
  import wandb
464
  wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
 
465
  last_sample_time = now
466
 
467
  if not args.no_save and is_main and (now - last_save_time) >= save_interval:
@@ -490,6 +503,7 @@ def main():
490
  if args.wandb and images:
491
  import wandb
492
  wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
 
493
 
494
  if not args.no_save and is_main:
495
  accelerator.unwrap_model(unet).save_pretrained(args.unet_path)
 
269
  return images, captions
270
 
271
 
272
+ def save_samples_to_disk(images, captions, step, folder="samples"):
273
+ """Сохраняет сгенерированные сэмплы в папку samples/ без номера шага в имени."""
274
+ if not images:
275
+ return
276
+ os.makedirs(folder, exist_ok=True)
277
+ for i, img in enumerate(images):
278
+ img.save(os.path.join(folder, f"{i}.png"))
279
+
280
+
281
  # ----------------------------- train ------------------------------
282
 
283
  def main():
 
286
  if args.seed is not None:
287
  set_seed(args.seed)
288
 
289
+ accelerator = Accelerator(mixed_precision="bf16")
290
  device = accelerator.device
291
  is_main = accelerator.is_main_process
292
 
 
301
  # UNet: загружаем из unet_path, если там есть веса — продолжаем
302
  if is_main:
303
  print(f"Loading UNet from {args.unet_path}...")
304
+ unet = UNet2DConditionModel.from_pretrained(args.unet_path, torch_dtype=torch.bfloat16).to(device)
305
  if is_main:
306
  print(f"UNet loaded: {sum(p.numel() for p in unet.parameters())/1e9:.3f}B params. Continuing training.")
307
  unet.enable_gradient_checkpointing()
308
 
309
  # text encoder всегда на GPU для обучения
310
  text_encoder = Qwen3_5ForConditionalGeneration.from_pretrained(
311
+ "text_encoder", torch_dtype=torch.bfloat16
312
  ).to(device).eval()
313
  tokenizer = Qwen3_5Tokenizer.from_pretrained("tokenizer")
314
  scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained("scheduler")
 
398
  if args.wandb and images:
399
  import wandb
400
  wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=0)
401
+ save_samples_to_disk(images, captions, 0)
402
 
403
  for epoch in range(args.epochs):
404
  batch_sampler.set_epoch(epoch)
 
455
  if args.wandb:
456
  import wandb
457
  wandb.log(log_dict, step=global_step)
458
+ if global_step % 10 == 1 or global_step <= 5:
459
+ print(f"step {global_step}: loss={loss.item():.4f} avg_1h={avg_loss:.4f}", flush=True)
460
 
461
  if is_main and vae is not None and (global_step == 5 or (now - last_sample_time) >= sample_interval):
462
  images, captions = generate_samples(
 
474
  if args.wandb and images:
475
  import wandb
476
  wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
477
+ save_samples_to_disk(images, captions, global_step)
478
  last_sample_time = now
479
 
480
  if not args.no_save and is_main and (now - last_save_time) >= save_interval:
 
503
  if args.wandb and images:
504
  import wandb
505
  wandb.log({"samples": [wandb.Image(img, caption=cap) for img, cap in zip(images, captions)]}, step=global_step)
506
+ save_samples_to_disk(images, captions, global_step)
507
 
508
  if not args.no_save and is_main:
509
  accelerator.unwrap_model(unet).save_pretrained(args.unet_path)
train_alchemist.log ADDED
The diff for this file is too large to render. See raw diff
 
train_test.log ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ flash_attn is installed but failed to import: cannot import name '_wrapped_flash_attn_backward' from 'flash_attn.flash_attn_interface' (/home/recoilme/.local/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py). Falling back to native PyTorch attention.
2
+ Device: cuda
3
+ UNet: 1.495B params
4
+ The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
5
+
6
+ The config attributes {'latents_mean': [-0.03542253375053406, 0.20086465775966644, -0.016413161531090736, -0.0956302210688591, -0.2672063112258911, 0.2609933018684387, -0.07806991040706635, -0.48407721519470215, 0.21844269335269928, -0.1122383326292038, 0.27197545766830444, -0.18958772718906403, 0.18776826560497284, 0.0987580344080925, 0.2837068736553192, -0.4486690163612366, 0.4816776514053345, 0.02947971224784851, -0.1337375044822693, -0.39750921726226807, -0.08513020724058151, -0.054023586213588715, -0.3943594992160797, 0.23918119072914124, -0.12466679513454437, 0.09935147315263748, 0.31858691573143005, 0.48585832118988037, -0.6416525840759277, -0.15164820849895477, -0.4693508744239807, -0.13071806728839874], 'latents_std': [1.5792087316513062, 1.5769503116607666, 1.5864241123199463, 1.6454921960830688, 1.5336694717407227, 1.5587652921676636, 1.5838669538497925, 1.5659377574920654, 1.6860467195510864, 1.5192310810089111, 1.573639988899231, 1.5953549146652222, 1.5271092653274536, 1.6246271133422852, 1.7054023742675781, 1.607722282409668, 1.558642864227295, 1.5824549198150635, 1.6202995777130127, 1.6206320524215698, 1.6379750967025757, 1.6527063846588135, 1.498811960220337, 1.5706247091293335, 1.5854856967926025, 1.4828169345855713, 1.5693111419677734, 1.692481517791748, 1.6409776210784912, 1.6216280460357666, 1.6087706089019775, 1.5776633024215698]} were passed to AsymmetricAutoencoderKL, but are not expected and will be ignored. Please verify your config.json configuration file.
7
+ Dataset: 1 samples
8
+ Caption: 'candid photo of young woman sitting at cafe'
9
+ text encoder выгружен из GPU
10
+ GT latent: (1, 32, 80, 56) (image 448x640)
11
+ Сохранён эталон: samples_test/gt.png
12
+ [step 0] PSNR rec=6.26 dB | gen=14.95 dB -> samples_test/gen_0000.png
13
+
14
+ [step 100] чекпоинт сохранён в unet_test
15
+ [step 100] PSNR rec=5.16 dB | gen=16.21 dB -> samples_test/gen_0100.png
16
+
17
+ Финальный чекпоинт: unet_test
train_test.py ADDED
@@ -0,0 +1,223 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # train_test.py — тестовое обучение UNet на одной картинке (datasets/test)
2
+ # Гипотеза: может ли UNet выучить одну картинку (дымовой тест обучаемости).
3
+ # Батч 1, lr не трогаем (5e-5). Всё остальное — как в train.py, упрощено.
4
+ import os
5
+ import random
6
+ import gc
7
+ import argparse
8
+
9
+ import torch
10
+ import torch.nn.functional as F
11
+ import numpy as np
12
+ from tqdm import tqdm
13
+ from PIL import Image
14
+
15
+ from diffusers import UNet2DConditionModel, FlowMatchEulerDiscreteScheduler, AsymmetricAutoencoderKL
16
+ from transformers import Qwen3_5Tokenizer, Qwen3_5ForConditionalGeneration
17
+ from datasets import load_from_disk
18
+
19
+
20
+ def set_seed(seed):
21
+ random.seed(seed)
22
+ np.random.seed(seed)
23
+ torch.manual_seed(seed)
24
+ torch.cuda.manual_seed_all(seed)
25
+
26
+
27
+ def tensor_to_pil(tensor):
28
+ """tensor: [C, H, W] в [0, 1]."""
29
+ arr = tensor.clamp(0, 1).cpu().permute(1, 2, 0).numpy()
30
+ return Image.fromarray((arr * 255).astype(np.uint8))
31
+
32
+
33
+ def parse_args():
34
+ parser = argparse.ArgumentParser(description="SDXS-1B UNet test training (single image)")
35
+ parser.add_argument("--ds-path", type=str, default="datasets/test")
36
+ parser.add_argument("--unet-path", type=str, default="unet_test")
37
+ parser.add_argument("--epochs", type=int, default=100)
38
+ parser.add_argument("--batch-size", type=int, default=1)
39
+ parser.add_argument("--lr", type=float, default=5e-5)
40
+ parser.add_argument("--max-length", type=int, default=250)
41
+ parser.add_argument("--cfg-dropout", type=float, default=0.05, help="вероятность пустого промпта в обучении (для работы CFG)")
42
+ parser.add_argument("--save-every-steps", type=int, default=50)
43
+ parser.add_argument("--sample-every-steps", type=int, default=50)
44
+ parser.add_argument("--num-steps", type=int, default=20, help="шагов сэмплинга для проверки")
45
+ parser.add_argument("--seed", type=int, default=42)
46
+ return parser.parse_args()
47
+
48
+
49
+ @torch.no_grad()
50
+ def encode_text(text_encoder, tokenizer, text, device, max_length):
51
+ messages = [{"role": "user", "content": [{"type": "text", "text": text}]}]
52
+ formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)
53
+ toks = tokenizer([formatted], padding="max_length", max_length=max_length, truncation=True, return_tensors="pt").to(device)
54
+ out = text_encoder(input_ids=toks.input_ids, attention_mask=toks.attention_mask, output_hidden_states=True)
55
+ emb = out.hidden_states[-2].to(torch.float32)
56
+ return emb, toks.attention_mask.to(torch.int64)
57
+
58
+
59
+ @torch.no_grad()
60
+ def generate(unet, emb, mask, scheduler, latents, num_steps, device, cfg=4.0):
61
+ """Денойзинг из заданных латентов с CFG (uncond_emb/mask — глобальные)."""
62
+ scheduler.set_timesteps(num_steps, device=device)
63
+ for t in scheduler.timesteps:
64
+ if cfg > 1.0:
65
+ lat_in = torch.cat([latents, latents], dim=0)
66
+ emb_in = torch.cat([UNCOND_EMB, emb], dim=0)
67
+ mask_in = torch.cat([UNCOND_MASK, mask], dim=0)
68
+ pred = unet(lat_in, t, encoder_hidden_states=emb_in, encoder_attention_mask=mask_in).sample
69
+ pred_u, pred_c = pred.chunk(2)
70
+ pred = pred_u + cfg * (pred_c - pred_u)
71
+ else:
72
+ pred = unet(latents, t, encoder_hidden_states=emb, encoder_attention_mask=mask).sample
73
+ latents = scheduler.step(pred, t, latents).prev_sample
74
+ return latents
75
+
76
+ UNCOND_EMB = None
77
+ UNCOND_MASK = None
78
+
79
+
80
+ @torch.no_grad()
81
+ def decode_latents(vae, latents, mean_t, std_t, device):
82
+ latents = latents * std_t.to(latents.device) + mean_t.to(latents.device)
83
+ decoded = vae.decode(latents.to(vae.dtype)).sample
84
+ return (decoded.float() / 2 + 0.5).clamp(0, 1)
85
+
86
+
87
+ def sample_and_save(step, unet, emb, mask, scheduler, vae, latents_gt, mean_t, std_t, device, num_steps, gen_generator):
88
+ """Генерирует gen (из шума, фиксированный seed) и rec (из 50% шума), сохраняет PNG + PSNR."""
89
+ unet.eval()
90
+ gen = generate(unet, emb, mask, scheduler, torch.randn(latents_gt.shape, device=latents_gt.device, dtype=latents_gt.dtype, generator=gen_generator), num_steps, device)
91
+ rec = generate(unet, emb, mask, scheduler, 0.5 * latents_gt + 0.5 * torch.randn_like(latents_gt), num_steps, device)
92
+
93
+ img_gen = tensor_to_pil(decode_latents(vae, gen, mean_t, std_t, device)[0])
94
+ img_rec = tensor_to_pil(decode_latents(vae, rec, mean_t, std_t, device)[0])
95
+ img_gen.save(f"samples_test/gen_{step:04d}.png")
96
+ img_rec.save(f"samples_test/rec_{step:04d}.png")
97
+
98
+ psnr_rec = 10 * torch.log10(1.0 / (F.mse_loss(rec, latents_gt) + 1e-8)).item()
99
+ psnr_gen = 10 * torch.log10(1.0 / (F.mse_loss(gen, latents_gt) + 1e-8)).item()
100
+ print(f"[step {step}] PSNR rec={psnr_rec:.2f} dB | gen={psnr_gen:.2f} dB -> samples_test/gen_{step:04d}.png")
101
+ unet.train()
102
+ return psnr_rec, psnr_gen
103
+
104
+
105
+ def main():
106
+ args = parse_args()
107
+ set_seed(args.seed)
108
+
109
+ device = "cuda" if torch.cuda.is_available() else "cpu"
110
+ print(f"Device: {device}")
111
+
112
+ # UNet (fp32, как в train.py)
113
+ unet = UNet2DConditionModel.from_pretrained(args.unet_path, torch_dtype=torch.float32).to(device)
114
+ print(f"UNet: {sum(p.numel() for p in unet.parameters())/1e9:.3f}B params")
115
+ unet.enable_gradient_checkpointing()
116
+
117
+ # text encoder — только чтобы посчитать эмбеддинг, потом выгружаем
118
+ text_encoder = Qwen3_5ForConditionalGeneration.from_pretrained("text_encoder", torch_dtype=torch.float16).to(device)
119
+ tokenizer = Qwen3_5Tokenizer.from_pretrained("tokenizer")
120
+ scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained("scheduler")
121
+
122
+ # VAE для декодирования при проверке
123
+ vae = AsymmetricAutoencoderKL.from_pretrained("vae", torch_dtype=torch.float16).to(device).eval()
124
+
125
+ mean = getattr(vae.config, "latents_mean", None)
126
+ std = getattr(vae.config, "latents_std", None)
127
+ if mean is not None and std is not None:
128
+ mean_t = torch.tensor(mean, device=device, dtype=torch.float32).view(1, -1, 1, 1)
129
+ std_t = torch.tensor(std, device=device, dtype=torch.float32).view(1, -1, 1, 1)
130
+ else:
131
+ mean_t, std_t = 0.0, 1.0
132
+
133
+ # ---------- датасет ----------
134
+ dataset = load_from_disk(args.ds_path)
135
+ print(f"Dataset: {len(dataset)} samples")
136
+
137
+ # эмбеддинги текста — один раз, до обучения
138
+ ds_text = dataset["text"][0]
139
+ print(f"Caption: {ds_text!r}")
140
+ emb, mask = encode_text(text_encoder, tokenizer, ds_text, device, args.max_length)
141
+ emb = emb.to(device)
142
+ mask = mask.to(device)
143
+ uncond_emb, uncond_mask = encode_text(text_encoder, tokenizer, "", device, args.max_length)
144
+ global UNCOND_EMB, UNCOND_MASK
145
+ UNCOND_EMB = uncond_emb.to(device)
146
+ UNCOND_MASK = uncond_mask.to(device)
147
+
148
+ # выгружаем text encoder — он больше не нужен
149
+ del text_encoder, tokenizer
150
+ gc.collect()
151
+ torch.cuda.empty_cache()
152
+ print("text encoder выгружен из GPU")
153
+
154
+ latents_gt = torch.from_numpy(np.array(dataset[0]["vae"], dtype=np.float32)).unsqueeze(0).to(device)
155
+ h, w = dataset["height"][0], dataset["width"][0]
156
+ print(f"GT latent: {tuple(latents_gt.shape)} (image {w}x{h})")
157
+
158
+ # ---------- оптимизатор ----------
159
+ import bitsandbytes as bnb
160
+ optimizer = bnb.optim.AdamW8bit(
161
+ unet.parameters(), lr=args.lr, betas=(0.9, 0.997), eps=1e-7, weight_decay=0.001
162
+ )
163
+
164
+ os.makedirs("samples_test", exist_ok=True)
165
+ gen_generator = torch.Generator(device=device).manual_seed(0) # фиксированный шум для gen
166
+
167
+ # ---------- эталон + сэмплы до обучения (step 0) ----------
168
+ gt_pil = tensor_to_pil(decode_latents(vae, latents_gt, mean_t, std_t, device)[0])
169
+ gt_pil.save("samples_test/gt.png")
170
+ print("Сохранён эталон: samples_test/gt.png")
171
+ sample_and_save(0, unet, emb, mask, scheduler, vae, latents_gt, mean_t, std_t, device, args.num_steps, gen_generator)
172
+
173
+ unet.train()
174
+ global_step = 0
175
+ pbar = tqdm(total=args.epochs, desc="train_test")
176
+
177
+ for epoch in range(args.epochs):
178
+ for _ in range(1): # одна картинка -> один шаг на эпоху
179
+ latents = latents_gt
180
+
181
+ noise = torch.randn_like(latents)
182
+ t = torch.rand(latents.shape[0], device=device, dtype=torch.float32)
183
+ t_bc = t.view(-1, 1, 1, 1)
184
+
185
+ # CFG dropout: с вероятностью cfg_dropout учимся на пустом промпте
186
+ if random.random() < args.cfg_dropout:
187
+ emb_use, mask_use = UNCOND_EMB, UNCOND_MASK
188
+ else:
189
+ emb_use, mask_use = emb, mask
190
+
191
+ noisy = (1.0 - t_bc) * latents + t_bc * noise
192
+ timesteps = (t * 1000).to(torch.float32)
193
+
194
+ pred = unet(noisy, timesteps, encoder_hidden_states=emb_use, encoder_attention_mask=mask_use).sample
195
+ target = noise - latents
196
+ loss = F.mse_loss(pred.float(), target.float())
197
+
198
+ loss.backward()
199
+ torch.nn.utils.clip_grad_norm_(unet.parameters(), 1.0)
200
+ optimizer.step()
201
+ optimizer.zero_grad(set_to_none=True)
202
+
203
+ global_step += 1
204
+ pbar.update(1)
205
+ pbar.set_postfix(loss=f"{loss.item():.4f}")
206
+
207
+ # ---------- проверка ----------
208
+ if global_step % args.save_every_steps == 0:
209
+ unet.save_pretrained(args.unet_path)
210
+ print(f"\n[step {global_step}] чекпоинт сохранён в {args.unet_path}")
211
+
212
+ if global_step % args.sample_every_steps == 0:
213
+ sample_and_save(global_step, unet, emb, mask, scheduler, vae, latents_gt, mean_t, std_t, device, args.num_steps, gen_generator)
214
+
215
+ pbar.close()
216
+
217
+ unet.save_pretrained(args.unet_path)
218
+ print(f"Финальный чекпоинт: {args.unet_path}")
219
+ torch.cuda.synchronize()
220
+
221
+
222
+ if __name__ == "__main__":
223
+ main()
unet/config.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ddf243ef7260c568c09fd28ca97cb8af215b857f764a309b2833cbf858cc1787
3
- size 1910
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a222ca0472aa440909a7ebb4187ae2f2094cac543abd8072e82c981a0fcc93ea
3
+ size 1915
unet/diffusion_pytorch_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c305871fc514cc77c51d171191c3aae5744ac221741bf241daf4e78007c9b8ba
3
- size 5979195336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f2991a2970f1dfed537fae22edeb55a3596fd37a0ab100dc18683896440ae8ab
3
+ size 3376130168