Convert into 3D photo/video like on Facebook?

[Jonseed]

Does anyone know of a way to locally convert an image and depth map into a 3D photo/video like those you see on Facebook? I've seen some GitHub repos that will do this (like this one), but they are quite complex to set up, and require quite a lot of dependencies. I haven't been able to get any of them working on Windows. It would be cool if there was a tool that easily took an image and depth map like those this script produces, and produced a 3D photo/videos.

[kalkal11]

if you want the easy mode of using this, have a look at a program called visions of chaos but there are some prerequisites to get it running.

Funny you ask because I also asked if thygate might consider making an extension for that too earlier :)

[thygate]

Depthy / 3D-image

Those are definitely nice options for viewing !

I will be adding a section on Viewing to the readme later.

[Jonseed]

@thygate I bet you could integrate the depthy repo with your script, or expand it into an extension, turn it into a whole tab in the UI.
https://github.com/panrafal/depthy

[thygate]

@thygate I bet you could integrate the depthy repo with your script, or expand it into an extension, turn it into a whole tab in the UI. https://github.com/panrafal/depthy

I was thinking in the same direction..

[Jonseed]

@thygate also, you might include an option in the script to invert the depth map, as Depthy expects the reverse as you have it, black is near, white is far away.

[thygate]

@thygate also, you might include an option in the script to invert the depth map, as Depthy expects the reverse as you have it, black is near, white is far away.

now included in v0.1.6

[thygate]

Layered Depth Inpainting
We propose a method for converting a single RGB-D input image into a 3D photo, i.e., a multi-layer representation for novel view synthesis that contains hallucinated color and depth structures in regions occluded in the original view.

That's very interesting, I'll have to look into that.

I have threejs and Unity3D projects that display on lookingglass and SteamVR devices. (wip)
It's always the same technique of rendering a texture on a (subdivided) plane with a displacement modfier. You could do the same in blender for instance. This is already good when viewed straight-on on a 3D display, but as soon as you start moving the camera things fall apart quickly.

Layered Depth Inpainting looks like a good upgrade .. I will definitely look into it when i find some time.
Thanks for sharing and good tip !

[dangerweenie]

this is a popular technique in Nuke and After Effects as well for doing background with paralax (i.e. storm clouds moving in FG and BG, etc)