Sygil-Dev/sygil-webui
1
1
Fork 0
mirror of https://github.com/Sygil-Dev/sygil-webui.git synced 2024-12-17 00:29:06 +03:00
Code Issues Projects Releases Wiki Activity
a1763c3c9e
sygil-webui/data/scn2img_examples/landscape_3d.scn2img.md

38 lines
877 B
Markdown
Raw Normal View History

Scene-to-Image Prompt Layering System (#1179) # Summary of the change - new Scene-to-Image tab - new scn2img function - functions for loading and running monocular_depth_estimation with tensorflow # Description (relevant motivation, which issue is fixed) Related to discussion #925 > Would it be possible to have a layers system where we could do have foreground, mid, and background objects which relate to one another and share the style? So we could say generate a landscape, one another layer generate a castle, and on another layer generate a crowd of people. To make this work I made a prompt-based layering system in a new "Scene-to-Image" tab. You write a a multi-line prompt that looks like markdown, where each section declares one layer. It is hierarchical, so each layer can have their own child layers. Examples: https://imgur.com/a/eUxd5qn ![](https://i.imgur.com/L61w00Q.png) In the frontend you can find a brief documentation for the syntax, examples and reference for the various arguments. Here a short summary: Sections with "prompt" and child layers are img2img, without child layers they are txt2img. Without "prompt" they are just images, useful for mask selection, image composition, etc. Images can be initialized with "color", resized with "resize" and their position specified with "pos". Rotation and rotation center are "rotation" and "center". Mask can automatically be selected by color or by estimated depth based on https://huggingface.co/spaces/atsantiago/Monocular_Depth_Filter. ![](https://i.imgur.com/8rMHWmZ.png) # Additional dependencies that are required for this change For mask selection by monocular depth estimation tensorflow is required and the model must be cloned to ./src/monocular_depth_estimation/ Changes in environment.yaml: - einops>=0.3.0 - tensorflow>=2.10.0 Einops must be allowed to be newer for tensorflow to work. # Checklist: - [x] I have changed the base branch to `dev` - [x] I have performed a self-review of my own code - [x] I have commented my code in hard-to-understand areas - [x] I have made corresponding changes to the documentation Co-authored-by: hlky <106811348+hlky@users.noreply.github.com>
2022-10-02 20:23:37 +03:00
size: 512,512
mask_blur: 6
prompt: fantasy landscape with castle and forest and waterfalls, trending on artstation
denoising_strength: 0.6
seed: 1
image_editor_mode: Mask
mask_mode: 0
mask_restore: True
# mask the left which contains artifacts
color: 255,255,255,0
blend:multiply
size: 100,512
pos: 50,256
# mask the top-left which contains lots of artifacts
color: 255,255,255,0
blend:multiply
size: 280,128
pos: 128,64
# go forward and turn head left to look at the left waterfalls
transform3d: True
transform3d_depth_scale: 10000
transform3d_from_hfov: 60
transform3d_to_hfov: 60
transform3d_from_pose: 0,0,0, 0,0,0
transform3d_to_pose: 4000,0,2000, 0,-50,0
transform3d_min_mask: 0
transform3d_max_mask: 255
transform3d_inpaint_radius: 5
transform3d_inpaint_method: 1
##
prompt: fantasy landscape with castle and forest and waterfalls, trending on artstation
seed: 1
Reference in New Issue Copy Permalink