DeoVR Volumetric/6DOF marathon

DeoAesthetics

We are further looking into embedding 6DOF volumetric experiences into DeoVR.

We would have to figure out:

Multicamera video recording
Representation generation
Encoding
Data streaming
Decoding

I will keep track cool of stuff we are looking at like https://renderpeople.com/free-3d-people/

DeoAesthetics

Some interesting discussion we have got in our Slack. Wanted to share it here
As a small footnote on 6DoF volumetric video cost, capture stage companies like Metastage, 4DViews, Dimension, Arcturus, 8i, Mod Tech Labs, etc are all working at "loss leader pricing".
When you see them quote a crazy high cost per minute for 6DoF footage captured, do realize they are bleeding red losses of huge amounts on their accounting ledgers since they are burning cash at LA/SF hollywood levels.
These well funded operations have done near zero effort to try to manage costs, or efficiencies. They are doing "get big fast, and hope for a buyout".
That's about all I can say when you see a $5K to $10K per finished minute of capture as a pricing quote from companies like the now defunct and closed Intel Studios capture stage.....
example: https://www.protocol.com/intel-studios-ar-vr-capture-closure
Those huge operations have little hope of fixing their cost structures. Small startups that do portable on-site capture with very lean support teams are DeoVRs best hope for volumetric video partners that production clients could afford and rely on.
Its nice to see the indie budget compatible technologies and firms showing up now.
The leanest, most efficient volumetric capture tech I am aware of from the last 5 years that was supper effective came from the Israeli company Mantis Vision. They had Samsung's sensor partner helping them on structured light bespoke sensors for ultra fast turn around capture.

https://mantis-vision.com
A lot of other companies are DIYing their way towards the same end approaches now.

DeoAesthetics

https://volucap.com/

Something from Meta https://vr-nerf.github.io/

Here is a summary of what the open-source Kartaverse v6 project has been researching for OpenXR driven real-time rendered lightfield workflows tuned for stereo 3D compatible on-set virtual production use cases.
https://docs.google.com/document/d/13JhhssoyfDg-jWWySAJMB7P6GnkiUVNSXNwZhQ9MdVM/edit

upcoming Nov 17 2023 - Feb 2024 efforts to make distributed pathtraced 6DoF virtual production tech a thing that indie filmmakers and on-set VP people can explore in an accessible fashion.
Google Docs | Kartaverse x LexhagVFX | Cloud to Camera VP Workflows
https://docs.google.com/document/d/1KOqFczk6PR5vBndWEQt9VP5qnkNtPmzLXTu0fETUWuo/edit#heading=h.hx61ypjm6lqm

DeoAesthetics

https://zju3dv.github.io/4k4d/
the speed for 4kdk seems most impressive so far. In general i think that streaming meshes + textures is not very flexible - so glad about splatting/pointcloud progress

DeoAesthetics

here's another approach mixed with NeRF and classic 3d mesh
https://huggingface.co/papers/2311.08581
it creates less data but has a lot of artifacts, in my opinion the best way is pure gaussian splats. Now, for example, a pure single frame with a person takes 40Mb, and this volume is certainly large to make a video, but researchers write that they will be able to reduce it in times.
Lots of goPro )

DeoAesthetics

Shot with 64 cameras, each frame rendering 20-40 minutes. (quite far from watching live sports this way )

DeoAesthetics

philpw99

Is this really the direction DeoVR is going ?

It's pretty cool, but it seems quite far away from practical use. Like how do you expect consumers to bring volumetric scanners for a trip? And why a consumer needs volumetric experiences?

VR180 is all I need. I don't even want VR360. I am too lazy to stitch videos, and too lazy to turn my head back to see what's behind me.

If no consumers can make this kind of volumetric videos, then who will make them and post them on DeoVR.com ?

Admit. This kind of things are great for corporations who need good representation of true 3D, but will they really pay much more money for it instead of the good old Zoom meetings ?

I know my words are quite harsh here. Please don't take it the wrong way. I only want DeoVR to be more popular and grow rapidly. I am just very confused about this volumetric tech and what kind of benefit it can bring to DeoVR.

DeoAesthetics

philpw99 there are some cool 3D scanning concepts for DeoVR partner sites. If we can make it work it will boost the whole 3D scans economy the same like we are boosting VR180 content creation.

Perfecting VR180 is definitely a focus. There are some few perspective technologies we are investigating.

Francin

@philpw99 "#p5360 VR180 is all I need. I don't even want VR360

Headsets with 140+ degrees FOV are round the corner and turning the head to center your eyes on part of the scene of a VR180 video will cause a black margin to appear at the side of your vision... not nice.

Manufacturers are already demonstrating at trade shows headsets with FOV 210 (hypervision). Don't think you'll be so satisfied with VR180 then.

coldbricks

Hey NSFW (hope it's OK) , but realgirlsnow.com works flawlessly in a Pico 4 passthrough. The 6dof is nearly flawless, but it seems that it is sort of a proof of concept, there's not a ton of content, but it definitely works and it works really cool. I had to sign up for a month just to see what it was like and I was immediately convinced this is the future of vr.