Share the easy way to convert GoPro videos to VR180

philpw99

You don't need Mistika or DaVinci Resolve, you just need ffmpeg.
First you need to know the horizontal and vertical FOV. For my GoPro 11 black, the FOV specs are here. Different shooting mode has different H_FOV and V_FOV. I usually shoot with 8:7 wide 60fps mode, so the numbers are 122 and 108.

Now run the ffmpeg with the following parameter:

ffmpeg -i input.mp4 -filter_complex "v360=input=sg:ih_fov=122:iv_fov=108:output=e:h_fov=180:v_fov=180:w=3840:h=3840:alpha_mask=1[out];color=c=black:s=3840x3840:r=59.94[bg];[bg][out]overlay=shortest=1" output.mp4

The command above basically tell a "v360" filter to convert "sg" format, which is the real format of GoPro footages, into EquiRectangular format. The input resolution is 3840x3360, and output is 3840x3840 for left eye or right eye. Of course you can increase it to 7680x7680 to retain more details, but I think majority of VR headsets can only handle 7680x3840 videos, so increasing the resolution beyond 3840x3840 is not practical.

Here's an example:
Original GoPro video image
Converted video image

If you download the converted picture and view it in your player as VR180 format, you will see that all the lines are straight. It's a close-to-perfect conversion, and it doesn't require you to learn a new video editor like DaVinci Resolve.

More over, because I have Nvidia video card, I can accelerate the conversion with the following command:

ffmpeg -hwaccel cuda -i input.mp4 -filter_complex "v360=input=sg:ih_fov=122:iv_fov=108:output=e:h_fov=180:v_fov=180:w=3840:h=3840:alpha_mask=1[out];color=c=black:s=3840x3840:r=59.94[bg];[bg][out]overlay=shortest=1" -c:v hevc_nvenc -cq 16 output.mp4

The conversion speed can be up to 20fps. If you use a batch file, you can easily convert all your footage in just one command.

It took me quite some time to figure this out. I hope by sharing it other people will also find it useful, and no need to do so many trial-and-errors like I did.

terryfi3d

Hi Phil, thanks for sharing your guide. I watched the flat3D videos, how does the output look in vr180?
I do something similar with 180Kino for DJIAction2 but it is not opensource and lacks profiles for gopro.
Cheers.

philpw99

There is a "Converted video image" in the post that you can download and see the result yourself. Since the file is hosted by google, if you are from China, you'll need some extra vpn help to download the picture.

The output in VR players looks like a curve screen that has extrusions on all four corners. It's exactly how gopro videos should have mapped to a hemisphere, but of course if you don't like the four corner extrusion, you can always use a mask in your favorite video editing program to change its shape, and/or blur its edges.

The interesting thing is, though the converted video are using roughly half of the total pixels, in VR players it seems like occupying 80% of the 180 degree fov.

This video https://deovr.com/4gib86 has exactly the same result in the beginning. You can view it as an example.

180Kino2 is a very cool program for DJI Action users. It will combine left and right eye videos into a VRout.mp4.
However, I prefer using a batch file to convert all footage from a shooting, then match the left and right clips in the video editor later. 180Kino doesn't have features like batch processing, clip matching, or codec settings. So the ffmpeg route is my preferred way to do the conversion.

philpw99

Sorry guys, I was partly wrong in the above script.
Recently I was editing a full clip from old GoPro videos and I realize a fatal error:
While the above ffmpeg script works for single eye picture/video, it doesn't work well with stereo ones. The center part is OK, but the left/right edges are difficult to look.

At first I thought it was due to the "iv_fov=108" parameter causing a vertical mis-alignment, then after measuring output files, there was no mis-alignment. The objects near edges were exact where they were supposed to be.

Now I think it's due to our own eyes and the way VR player handle objects near left/right edges. Simply put, when converting GoPro videos to VR180, we just cannot make them too close to the left/right edges. It will make your eyes difficult to handle.

So the solution is changing to "iv_fov=100" and "ih_fov=113", basically shrink down the video, make it further away from the edges. You will get smaller images, but in exchange you will get much more comfortable 3D videos to watch.

I saw some videos from DeoVR having the same kind of problem, and they were also GoPro clips originally. I guess they probably using the same technique here to convert the video to 3D VR180. For the horrible problem that my script had caused. I have to apologize to you!

The number iv_fov=100 and ih_fov=113 are just my best result so far. There might be better numbers around that will yield better result. It's up to you to experiment.

If you find better numbers, please post here and let me know. Thank you!

PS1:
this method also applies to single GoPro photos. If you have those 8:7, 5568x4872 photos, you can convert them to VR180 format with this command:
ffmpeg -i %1 -filter_complex "v360=input=sg:ih_fov=113:iv_fov=100:output=e:h_fov=180:v_fov=180:w=11520:h=11520:alpha_mask=1[out];color=c=black:s=11520x11520[bg];[bg][out]overlay" -f image2 -frames:v 1 -qmin 1 -q:v 1 -pix_fmt yuvj422p "%~dpn1-out.jpg"

Here it will convert a 5568x4872 GoPro photo to 11520x11520 VR photo. The VR player can handle this kind of extra large photos, but when you combine 2 together to form a 3D photo, It's better to shrink them down to 5760x5760 then combine them into a 11520x5760 photo so the VR players can handle it.

PS2:
If you need to use your favorite video editor to edit the video, HEVC/x265 is a terrible choice. So it will be much better to convert your clips to the lightest Apple ProRes and increase the output size to 7860x7860 in one process:
ffmpeg -i input.mp4 -r 59.94 -filter_complex "v360=input=sg:ih_fov=113:iv_fov=100:output=e:h_fov=180:v_fov=180:w=7860:h=7860:alpha_mask=1[out];color=c=black:s=7860x7860[bg];[bg][out]overlay=repeatlast=0:shortest=1" -c:v prores -profile:v 0 -pix_fmt yuv422p10le output.mov
( I know it's not recommended to use "-profile:v 0", but it will only increase the size 8x, profile 1 will increase the size to 16x, and I honestly don't see the difference. Of course you can specify the profile you want, the best one is 3. )
With ProRes files it will be much easier and faster to edit.