Video lags when playing anything 5K and above

dizzyMongoose

I have an issue where when I attempt to play a 5K video, the audio runs okay but the video is slow, so the playback is out of sync. Is this something I can fix through settings somewhere on the app, headset or PC, or is my hardware too weak?

Skybox does seem to handle 5K videos better for me, in that they don't desync, though there's some artifacting still when I navigate around the video that clears up after a second. It's entirely possible it's compensating by instead dropping frames rather than desyncing, though.

My setup:

VR Headset: Oculus Quest 2 w/ Oculus Link (USB 3.0 cable)
CPU: AMD Ryzen 5 3600
Motherboard: Gigabyte X570 Aorus Elite
RAM: 16 GB DDR4-3200
Video Card: EVGA GeForce RTX 2070 Super 8 GB Black
PSU: Corsair RMx 650W
OS: Windows 10 Pro

Yar

dizzyMongoose I agree with philpw99, your CPU looks weak for 4k + video

philpw99

No, your hardware is more than enough to handle the video above 5k. It's just DeoVR has some difficulties handling the 10bit video output. If you transcode the video from 10bit to 8bit, even 8k videos will play smoothly. I think this is a pipeline issue, because the decoder is only working at 60% for 10bit videos, not 100%. Something in the pipeline has cause the traffic to slow down and the decoder didn't get the full data to decode in time.

Oh, there is a little fix too. If you are using LAV video decoder, uncheck the "P010" in "Output Formats" will help a little bit. This way, it will force LAV decoder to decode the video in 8bit format, but still, probably won't help enough.
I only have a Geforce 1080, so the result might be different.

dizzyMongoose

I don't believe my videos are in 10-bit, though I don't really know much about encoding, so I could be reading the stats wrong. Here's the MediaInfo for one of the 5K videos that is lagging:

Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 11.0 GiB
Duration : 52 min 15 s
Overall bit rate : 30.0 Mb/s
Writing application : Lavf58.76.100

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6@Main
Codec ID : hev1
Codec ID/Info : High Efficiency Video Coding
Duration : 52 min 15 s
Bit rate : 29.9 Mb/s
Width : 5 400 pixels
Height : 2 700 pixels
Display aspect ratio : 2.000
Frame rate mode : Constant
Frame rate : 59.940 (60000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.034
Stream size : 10.9 GiB (100%)
Writing library : x265 3.5+9-bf91444e0:[Windows][GCC 10.2.0][64 bit] 8bit+10bit+12bit
Encoding settings : cpuid=1111039 / frame-threads=6 / numa-pools=48,0 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=5400x2700 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=360 / keyint=360 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=0 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=20.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=30000 / vbv-bufsize=60000 / vbv-init=0.9 / min-vbv-fullness=50.0 / max-vbv-fullness=80.0 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=1 / colorprim=1 / transfer=1 / colormatrix=1 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=255 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Language : English
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 52 min 15 s
Source duration : 52 min 15 s
Source_Duration_LastFrame : -13 ms
Bit rate mode : Constant
Bit rate : 129 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 48.0 MiB (0%)
Source stream size : 48.0 MiB (0%)
Language : English
Default : Yes
Alternate group : 1
mdhd_Duration : 3135966

Other
ID : 3
Type : Time code
Format : QuickTime TC
Duration : 52 min 15 s
Frame rate : 59.940 (60000/1001) FPS
Time code of first frame : 00:00:00;00
Time code, striped : Yes
Language : English
Default : No

philpw99

The video you provided is 8bit color depth. If you cannot even play such a file smoothly, then obviously you are not using a hardware decoder to handle the HEVC decoding. I recommend using K-Lite Codec pack.

After you install the Codec pack, change the "LAV Video" property to use hardware decoder, some said "DXVA 2 native" is the best, my personal good setting is "Nvidia Cuvid (old)". It's old but it works fine for me.

Now, when you play the video above with DeoVR, you should see LAV icons in the notification area (where the speaker and clock icons are). If you don't see the LAV icons, run the "Codec Tweak Tool" that came with K-Lite. Make sure the "Preferred Splitters" are all LAV Splitter, and the "preferred decoders" should also be "LAV video". You probably need to restart the computer to make it effective( my own experience is that it works right away).

With LAV decoder, you should play all 8 bit video, even the 8k ones, smoothly. Now 10 bit is another beast to slay.

dizzyMongoose

I do have K-Lite Codec installed with LAV Video on, but after playing around with the hardware decoder setting, the Nvidia Cuvid decoder (and only that decoder) got rid of the video lag. Wild. Thanks a bunch!

boboweb

Yar

Hahahaha such a bullshit ... 🙂 Ryzen 5 3600 is too weak for 4K+ videos on RTX2070 Super !?!? Are you crazy ?? 🙂🙂🙂
dizzyMongoose my friend just dont listen these bullshits - your hardware is more then enough for smooth play of 8K vids. The problem is in deovr versions over 9.xx . But here you wont get any support from the developers.
Just try to downgrade to 8.xx version or try different video player . Dont mess with that LAV codecs bullshits too .

@philpw99 its time to understand that there is much bigger problem then 10bit vids so better dont mislead the people . thanks.

philpw99

boboweb
I am confused why Ryzen 5 3600 cannot handle 2700p 8bit video myself.
I have Ryzen 5 2600, and using pure software decode, the CPU using only 26% power to handle a 2700p video.
Let alone the 3600 is much better than 2600.
That's why I thought in the beginning that it was the 10bit problem. It turned out not to be true. It was an 8bit video. That's why I posted the second one to provide a different fix. I didn't try to mislead people in anyway. God knows how many tries you have to do before you find the real solution. Each try is just an attempt of fix, not a misleading.

boboweb

Guys common are you serious !?? For what CPU handling you are talking about at all ??? The man said he has RTX 2070 Super and it has hardware decoding everything up to 8K 10bit . My CPU is i5 2500K - 10 years old just 4 core CPU and i can play all 8K vids with 120fps on my 1080Ti ... So please be more serious .. and dont mislead the people they must to invest in new CPUs ! Just fix the deovr software...

philpw99

boboweb
You confused yourself with CPU and GPU decoding.
While I said that the CPU Ryzen 5 3600 itself should be able to handle a 5k video, you kept referring the issue to GPU decoding.
CPU decoding is the "software only" decoding. It uses only CPU, which is Ryzen 5, to decode the video. It doesn't involve the video card. In LAV Video setting, it's when "hardware acceleration" set to "none".

Today's GPUs are beasts. GeForce 1080 can handle 8kx8k, and GeForce 2070 should be able to handle 16k x 16k HEVC decoding. Yet our friend @dizzyMongoose still have a problem of playing a mere 5k video smoothly.

So from my point of view, even just with the software decoding, his machine should be fine playing the video, so I suggested it was 10 bit problem. Then from his reply post, it turned out it's still the "hardware acceleration" issue.

This means, even the CPU is fast enough to decode, DeoVR still have some kind of pipeline problem, which makes the video lagging or jumping.

@Yar Well, while I am glad that using good old "cuvid" fixed his problem, I start wondering whether DeoVR's decision to use new runtime really worth the effort? Rewriting code from the ground is quite painful, yet the result is actually not as good as the 8.xx ones. What kind of benefit does this new runtime brings?

boboweb

philpw99

I am not confusing myself you can bet on this ... I have more then 30 years of experience with hardware so please dont try to argue with me on topics that you are not prepared so well ...
The man said he has RTX2070 Super - so for what CPUs you are talking about at all ???
And the 10 bits are nothing to do with the new faulty deovr versions ... Otherwise why all works smooth on version 8.55 but not on 9.5 ??

And no RTX 2070 cant handle 16k x 16k - it can handle just 8K same as RTX 3090 ...

https://docs.nvidia.com/video-technologies/video-codec-sdk/nvdec-video-decoder-api-prog-guide/

philpw99

boboweb
After reading your reply, I don't know what to say any more... So I figure I'll just shut up...

notworking

I seem to suffer from the same issue. My rig is considerably weaker than dizzyMongoose's, but I was able to play up to 8k video perfectly before the most recent update. Everything had worked perfectly ever since I installed the HEVC codecs through Microsoft Store.
The newest update to DeoVR has rendered it pretty much unusable, most likely for many people.

philpw99

notworking
About the HEVC codec from Microsoft, I have an interesting revelation:

When I play a 3630p 8bit video in Windows Media Player, it's definitely using the HEVC codec. There is no LAV icons in the notification area, and the CPU usage of my poor Ryzen 5 2600 jumps to 100%. Meanwhile the GPU decoder usage remains at 0%.
Therefore, the HEVC codec in my PC (Win10 1809, build 17763) is obviously using pure software decoding. DXVA2 obviously not working.

Now when I play a 3630p 10 bit video in Windows Media Player, the playback is insufferably slow and jumpy. The CPU usage drops to 65% to 80% !! Meanwhile the GPU decoder usage is still 0%.
This means the Microsoft Media Player (with HEVC codec) also suffers from the same issue like DeoVR player ! It's in fact worse, because in DeoVR at least I can make the GPU do the decoding, but in Windows Media Player, there is nothing you can do.

If I use the PotPlayer, MPC or VLC, they all play the 10bit video smoothly, and they all use 80% to 90% GPU decoder power, while the CPU is basically idling.

In short, I think it's something broke in the Windows Media Player, its 10 bit handling is terrible ! Unfortunately DeoVR is using some part of WMP (maybe the renderer?) , so it suffers the same pipeline issue.

notworking

philpw99
Interesting. Other VR video players seem to work just fine. It's just DeoVR that works terribly now. I tried Windows Media PIayer, and it pretty much crashes instantly if I try to play some videos.
MPC doesn't even seem to play them properly for me anymore, so I have no idea why the other VR players work.

I'm sure they're using the HEVC codecs though, because otherwise they would run really poorly on my rig.
One VR player says that it's using "Media Foundation" to play them, I'm assuming that's basically without DirectShow and it works fine.
So basically, I have no idea where the problem lies at this point, but at least other players seem to work.
I got a Ryzen 5 2600x and a GTX 1650 Super.

philpw99

boboweb
Very interesting to know the details !

boboweb notworking @dizzyMongoose

OMG !! I finally got 7k 10bit video to play smoothly ! What a freaking journey !!
I think the HEVC codec provided by Microsoft does work with DXVA and DeoVR magically !

To play all the other 8bit videos (7k, 8k), you set LAV Decoder to "Nvidia Cuvid (old)" and enable the "Direct Show" in DeoVR player. They should play just fine.

To play a 7k or 8k 10bit video, you need to open the file in DeoVR first, it will play lagging and jumping, then you click on "DirectShow" to turn it off. Once the direct show in DeoVR is turned off. Magically it will play smoothly and nicely. Next time when DeoVR open it, it will remember the setting and automatically turn the direct show off for this video.

My guess why this fix works:
Once you turn off "DirectShow" in DeoVR, the player will use HEVC codec to play the video, and HEVC supports DXVA2, which fully supports 10bit decoding. So you will finally see smoothly playing 10bit video in DeoVR.

However, if you play a regular 8bit video with HEVC codec, somehow the DXVA2 failed, and the codec will use CPU to do the decoding, which will result in poor frame rate.

In short, for now, enable "DirectShow" for all regular videos, and disable "DirectShow" for 10 bit videos. Don't the "Alternative Method" at all.

boboweb

https://imgur.com/RbEDfhK

On this screenshot you can see 8K vid played with PotPlayer using its buld-in FFMpeg Decoder.
The VPE is loaded to 90% and you can get smooth playback - 60fps and above ...

https://imgur.com/pWyral2
On this screenshot you can see 8K vid played with PotPlayer using the FFMpeg CUDA Decoder (LAV Filters).
The VPE is loaded to 70% and you can NOT get smooth playback - only about 45fps (no mather LAV filters settings) .

Guess the problem of DeoVR versions after 9.xx use exactly that LAV Filters decoder by default and this is the problem with stuttering playback .

Other my observation from today:

If I run this vid via DeoVR 8.55 trough VirtualDesktop => Ultra Quality 2688x2784@120Hz and SteamVR i can get around 100fps

If I run this vid via DeoVR 8.55 trough VirtualDesktop => Ultra Quality 2688x2784@120Hz and Oculus engine i can get around 120fps (probably more if the display supports over 120Hz)

With DeoVR 9.7 the best i can get is around 75fps in both ways with Alternative decode enabled in DeoVR options and about 35fps if it is not enabled .

Hope this is userful for someone and this my post wont be deleted as my previous ones...

boboweb

philpw99

Hey Phil good for you but do you really believe most of us havent already tried Microsoft HEVC codecs and Direct Show option long time ago !?? 🙂🙂🙂 Unfortunately this doesnt solve the problem for most of us .
Can you try to understand once that on very same system (hardware , drivers codecs , filters and all)
everything works smooth on DeoVR v8.55 but not on v 9.xx ?

And something else for someone smooth play can be 60fps . For me really smooth play means at least 90fps ... better 120 ... You also never share at what render resolution you get that smooth play , and this is important too ... Do you start deovr via SteamVR or Oculus , do you use VD or cable and so on ...So its not so simple ... 😉

philpw99

boboweb
I know that this solution probably works only for me, but damm it I spent quite a long time to find it. And this solution is actually not as simple as you thought.
First if you only have HEVC codec not LAV, it won't work because most 8bit videos will play poorly.
Second if you have LAV Decoder and set the hardware acceleration, it still won't work because 10bit files get really low frame rate. Even after you install HEVC, the same issue remains.
Third if you only switching the "direct show" on and off, it won't fix the problem either. The 8bit videos will give you trouble if you switch it off. So you will not think about turning it off.

So I disagree with you saying we had tried these things already. No, no , this is a fix that's very hard to find. Man o' Man. Thank God now I don't have to re-encode all those 10bit videos !

My gig always shows DeoVR running at 90fps. It's the video decoding that gave me the trouble. It doesn't matter if DeoVR runs at 1000 fps, if the video can only shows 20 frames/second, what good is the VR frame rate?
I like the 8.xx, that's the one hooked me up with DeoVR. but I don't want to stick to that version forever. I prefer using new version for their new improvement. Right now the video playback is no longer a problem. I can really enjoy using the new ones !

philpw99

Sorry, it seems I am wrong again. 🙂
I just found out DeoVR have ffmpeg, avformat and avcodec in the plugin folder.
So it's obvious that when not using "direct show", DeoVR will use ffmpeg to play the video, which is supposed to be great. I guess HEVC codec doesn't really matter.
Still I don't know why ffmpeg handles 8bit videos so poorly in this case.

boboweb

philpw99

By the way what are these 10bit vids ? Which VR studio ?
I wonder can you really see difference with the 8bit vids on these VR set screens am not sure for example Quest 2 IPS screen is 10 bit or its rather 8bit... ? Maybe on some OLED screens you can notice a difference ...
What is your VR set ?

Yar

philpw99
You are wrong, the HEVC codec matters.
The correct setting of the video codecs installed in the system also matters if we are talking about Windows platforms.