Branched-off projects

[semjon00]

There are a couple of projects which had branched from the code in this repo. These add some quite cool features. I think it would be a good idea to collaborate with the contributors of these repositories to eventually merge these features (back) into a single project - this way any new supported model, bugfix, generation technique, etc. will be immediately accessible to be used for these features. And in total it will be less work to maintain. This repository already provides some of the features, but not all of them.

• https://github.com/graemeniedermayer/stable-diffusion-webui-normalmap-script
• https://github.com/Extraltodeus/depthmap2mask

I think however, that this repo should be refactored first.

[graemeniedermayer]

Yes I definitely agree! I have more free time in about 2 weeks so I should be able to help out! I also wanted to make sure that the normal maps are use-able with controlnet (or at least there's a toggle for compatibility).

With how many projects use depth video I think it would be good to integrate some sort of depth flickering (16 bit deflickering). Some of the common algorithms fail at 16 bit, so it might be a little require a little elbow grease. I haven't found a solution i'm overly happy with.

This would allow workflows like https://www.youtube.com/watch?v=t-8I7EkIL8c to be much easier.

[semjon00]

@graemeniedermayer Hello! I have gotten to move the NormalMap functionality to this repository. The changes are currently in the dev branch and they will be mainlined in some time (we have a massive refactor on the way). I have added you as a coauthor. Could you please test these changes? Do they work correctly?

[semjon00]

Some discussion here: #282

[graemeniedermayer]

I updated the top of the readme to point here.

There's quite a few experiments I've wanted to get to, so I'm not sure I'll archive yet. I've been trying to get tangent normals and reduced depthmap with baked in normals working. As well as some video experiments.

[semjon00]

@graemeniedermayer Thank you! For experiments, you could force-push depthmap repository main into normalmap repository main, so merging in future would be easier. By the way, could you please test-drive the new standalone mode that just landed? I personally find it more convenient to develop using it. #290

[graemeniedermayer]

By the way, could you test-drive the new standalone mode that just landed?

@semjon00 Sure! I realised I was using some of auto1111 for the api so I'll probably need to add a few things to keep that working.

you could force-push depthmap repository main into normalmap repository main

I might be misunderstand, but I'm not sure this a good idea. If people update their extensions and they have both (depth+normal) installed they'd end up with two depth tabs and it causes some problems. I think a slower transition would probably be better. In like a week or two if everything is stable here then it might make sense to add a [depreciated] to the title.

[semjon00]

Riiight. And, also, I forgot that the normalmap repo has an excellent Readme with examples that should be moved here (I think mainly to the wiki: #297). Sorry, I was still sleepy when I responded 😬

[graemeniedermayer]

@semjon00 So I can do 1024 by 1024 zoe_depth with standalone. Whereas I'm basically capped on gpu at 768 by 768 with the webui. This is with 8gb ram.

Basically this is amazing!!

[graemeniedermayer]

One small issue I'm experiencing is it saying "Generated!" when it runs out of memory. This the error handling might cause some confusion.

[semjon00]

Glad it helps ☺
Lol, just realized the "Generated!" is inside the finally block. Added fixing it into the backlog.

[graemeniedermayer]

There does seem to be some form of memory leaks as it can only do a couple of images in a row. Still a killer feature.