Ok I seem to have found the fix. Probably there is a more optimal way but at least it worked for me.
I just took ffmpeg and converted the original file into mjpeg without any other changes and then took the output and converted it back to x264 using command like
-i 1.mp4 -q:v 0 -c:v libx264 2.mp4
here's the ffprobe output for original file that won't play:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Duration: 00:31:19.84, start: 0.000000, bitrate: 6180 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x960, 5915 kb/s, 30 fps, 30 tbr, 90k tbn, 2000k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 262 kb/s (default)
Metadata:
handler_name : SoundHandler
And here's one for the final one that plays nicely:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '2.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.42.100
Duration: 00:31:19.87, start: 0.000000, bitrate: 2343 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc), 1920x960, 2206 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
Metadata:
handler_name : SoundHandler