Hdr to sdr bt2020 conversion

Discussion in 'ZIDOO X9S' started by asharma, Jan 24, 2018.

  1. asharma

    asharma New Member

    Hi folks...on the x9s, is it possible to convert hdr bt2020 to sdr bt2020. In other words if the display is 4K, no hdr, will the Zidoo still output bt2020 for wide Color gamut or will it only output rec 709? Thanks
     
  2. bob

    bob Active Member Zidoo TECH Supporter

    The current output 709. if you tv not support hdr.
    thanks
     
  3. Stefano C

    Stefano C Member

    I have a similar problem with my tv, a full hd lg le8800; when I start watching a hdr file I can't see well like a bluray , I havent's reached the same contrast and colour (hdr to sd effect on)
     
  4. n_p

    n_p Active Member

    Just to give you the theoretical perspective - a "correct" downconversion from HDR into SDR is impossible -
    regardless if your TV is bt2020 (but not HDR) "capable", or not.

    HDR was not designed to be backwards compatible - whatever conversion the Zidoo box does is probably not a 3D matrix remapping - and there is no "standard" anyone could apply to do the remapping (3D matrix, or not ;) ). They probably do a gamma transformation, and compress the colorspace - god knows how.. ;)

    Also bt2020 "capable" but not HDR capable - doesnt mean -anything-, because the only bt2020 standard out there includes HDR.

    Lets do a visualisation example. :)

    Lets think of the bt2020 color space (actually DCI-P3 more likely) as a cube with HDR defining the height of the cube (Y = luminance = "brightness").

    If you now insist, that you have a "bt2020 capable device" that cant reproduce HDR, the color space now becomes a cube that isnt even remotely that high, but more problematic - that follows no standard whatsoever (no one could even theoretically know how the colors on your TV look like). Even where your "most saturated colors lie" follows no standard (should be calibrated for bt2020 primaries, but in 50% of the cases are wrongly chosen to be DCI-P3 primaries - so the marketing could put a 95% of DCI-P3 label on it).. :)

    So have mercy with @bob if you demand the impossible - because the only other color standard available that TVs would follow is SDR and bt 709 (just because im curious @bob: at what gamma? ;) ).

    How do you do a conversion into a non standard colorspace, that is different on every tv/projector? Answer: You dont.

    You could do one into DCI-P3 @100 nits, that would be double "non-standard", but a rough approximation, what most "non HDR capable devices, that claim to do bt2020" out there would aim for. ;)

    At some point, someone has to tell you guys though - that you cant retrofit, what you havent bought. ;)

    Also - color accuracy in any of those "downconversion" modes is fundamentally impossible.

    (But the same goes for HDR 10, on all HDR 10 capable devices - just in another way.. ;) (Because HDR10, as a standard is broken - and 40% of the studios provide inaccurate metadata in the Blurays they release into the market. :) ) )

    edit: To explain their (HDR10 capable device owners) problem, they all have "HDR height" int he cube, but its different for every one of them. (Fun!) To "lessen the problem" a system was deviced, where every "HDR 10 capable TV" would use the correct gamma curve, but again - would need a color transformation to depending on its "capabilities". This is the point where the "metadata" concept comes in. The mastering studio would define what screen the content was mastered on (most often 2000-4000 nits, with most current TVs sitting at 1200 max.), and the TV would then downconvert the color space, based on this value and its own capabilities. The issue being, that if you decide that ONCE for every movie, even even colors that could be displayed "correctly" (because a scene f.e. didnt include HDR elemens) would now be converted to different (wrong) colors, because the entire colorspace is being compressed. So far - so "normal" for the studios, that wanted to sell you content, but then they started to put in incorrect mastering brightness values (0), to "make their films look brighter" on existing displays out there - because customers complained - that it looked not bright enough. At which point they ruined the HDR10 concept as well as the releases they sold into the market. Fun. :)

    The thing LG started to do on its OLEDs to "compensate" for the entire ecosystem being ruined at this point - was to do a real time, scene based, frame analysis - and then apply heuristics to "what kind of scene this might be, based on content and element brightness", and then remap the entire scene dynamically based on their best guess. :) This needs knowledge of the capabilities of the TV, and dedicated hardware - none of which Android TV box sellers have. :)

    At the moment all other TV manufacturers are starting to copy that approach. :) In short - the whole HDR thing, atm is a clusterf*ck.

    The way out of this is - either reaching 10k nits quickly - which will be impossible for the end consumer in the next 10 years, or popularizing either Dolby Vision, oder HDR10+. The issue there is, that this is on the verge to becoming the next format war, with manufacturers and content owners neatly split between camps - and marketing still proposing, that the normal customer is too darft to even realize, that HDR 10 is broken - so why not produce with that in mind still...
     
    Last edited: Jan 26, 2018
  5. asharma

    asharma New Member

    You are faaaaaaaar more technical than me Sir...all I know is Panasonic and Oppo “somehow” do it...a handful of us who want more brightness from our projectors are seeking “sdr bt2020”...not even sure if that is the right term...perhaps I should ask if the Zidoo “has the feature that the Panasonic and Oppo” do...
     
  6. n_p

    n_p Active Member

    Panasonic and Oppo do it:

    They probably do a downconversion to either

    bt709 @100nits (probably 2.4 gamma (plg))
    or
    DCI-P3@100nits (probably 2.4 gamma (plg)) (the "bt2020 space" thing you would like Zidoo to do in this thread - the only difference being - "more saturated colors, in some cases" :) )


    The problem here becomes that how you do that, is by "making it up". :) And by badly approximating. ;)

    Both "targets" above loose "a significant amount of color information" - lets say 50%, based on them being smaller. You cant just "say" that everyone of those colors becomes the same color - just less bright - because you would introduce severe clipping, where entire content would vanish from the film (leaving just a bright white image, blue sky, green effect bubble, ...). So you will have to do some sort of transformation. And that transformation is non standard.

    Meaning - there is no one, who has defined, what it should look like. Everyone can do "whatever".

    Panasonics "whatever" seemingly is more liked than Oppos "whatever", than Zidoos "whatever". ;) Based on popular opinion looking at "some content" (whatever that is, because what you'll loose will be different depending on every scene in every movie.. ;) ).

    There is another issue. Non HDR10 capable devices, use a different gamma curve - so there has to be a gamma transformation going on at the same time as well.

    Now - how do you calculate a gamma transformation, and color space compression at the same time, in real time - on an Android box for 8,2 million pixels 24 times a second? (Towards a transformation target you've just invented (good luck in the non standard devices out there supporting it) ). Answer - you dont. :) You cheat. :)

    I'd be interested in the different forms of cheating that are going on - but "demanding Panasonics form of cheating" (again - no standardization anywhere.. ;) ) will be tricky.

    I guess (just a shot in the dark), that Panasonic is using DCI-P3 @100nits, 2.4 gamma (plg) as a target, and opts to show "detail" rather than brightness - but, at some point, applies some form of clipping as well.

    If Zidoo can implement this "as well as Panasonic" - I dont know, but its a hard/insane task at pretty much every level.. ;) Because "how good it is" depends on what you are looking at - and HDR explicitly not being backwards compatible - so what ever you discard (and you will discard color information left and right.. ;) ), is up to you and your team of color processing engineers.. ;)

    Maybe start an open source project and see who bites.. ;)

    But then - the HDR 10 issue (incorrect metadata), still applies - and a new issue is introduced, with displaying the image at a different gamma, and you have to make significant shortcuts to do the "transformation" realtime - on a chipset thats not "made to be able to do that". :)

    In short - "I really dont know what Panasonic does, or how they do it... ;) " I just know, that a group of 4 customers on the internet say its "rad", and much better than Oppos implementation . ;)
     
    Last edited: Jan 26, 2018
  7. asharma

    asharma New Member

    Wholly crap...u know your stuff! All I know is whatever my Panni is doing is converting to sdr for added brightness BUT maintaining the Wide Color Gamut.
     
  8. n_p

    n_p Active Member

    "For added brightness" compared to what? ;) Because you dont change the max brightness on the TV. And the TV doesnt change the brightness depending on the input signal.

    Even that (the "added brightness" thing), is achieved by a "non standard gamma conversion" (one gamma mapped to another gamma curve (one ends at 100 nits the other one at 10.000 nits ;) )) - ie. made up - especially in regard to max "brightness" ;)

    That it "maintains wide color gamut", most likely means that it uses the DCI-P3 @100 nits color space as a target and not bt709. Yes - maybe Zidoo can do that. Maybe you'll be more happy with that -

    but what "you loose" depends on each and every scene, and the exact transformation that is applied, is non standard - non color accurate in every case - and they can't do it the right way (3D color Lut transformation) - because the chip cant do it in real time. :)

    Also - by optimizing for your "needs" they would exclude people who want to watch UHD rips on non "wide color gamut devices" (marketing feature - never has been a standard). :) Or at least have to add another toggle.
    -

    Here is another approach. If your TV has "wide color gamut", it probably has a mode that "displays normal bluray content in WCG", right? Just activate that. The colors then are displayed more saturated, but more or less just as made up... ;)

    If you have both a Panasonic player, and the Zidoo I'd be interested in comparison pictures on the same content in that "mode" though - make sure that you use a fixed aperture if you can. ;)

    I'd actually be interested to find out what it could mean, that the Zidoo image is "less bright" f.e. .. :)
     
    Last edited: Jan 26, 2018
  9. n_p

    n_p Active Member

    Not necessary anymore. :)

    I just read this:

    And I laughed... ;)

    If you wan't to read the explanation - just on gamma mapping (excluding all color issues.. ;) ) again from another person:

    (That better highlight detail == near black detail loss, btw. At the cost of color innaccuracy accross the board. Salesmen really should stop selling you people UHD discs.. :) )

    src:
    http://forum.blu-ray.com/showthread.php?t=291020

    So knowing this, here is the new assessment.

    Zidoo either does no gamma transformation, or just one you don't like.
    No one does any form of color remapping - because otherwise they would not provide a "freeform slider" for gamma. ;)

    Short of introducing a 25pt slider you could switch for every scene in a movie, and that you'd have to move depending on the studio that mastered the UHD release - and excluding severe color errors that come with compressing the source color space "on the fly" by more than 50% on hardware thats not capable to do so accurately - there is no solution.

    We are still not sure, if Zidoo does a gamma transformation at all, in case they dont - btw the entire image would look severely washed out (compared to even a blueray of the "same" content). So it would be easy to spot.

    But it could just as well be, that you like one of the 25 "steps" on Panasonics slider better, than the one Zidoo chose to go with.. ;)

    And you are worried, that you cant get more saturated WCG colors... :)
    (Explaination - not chosing the "correct gamma" (with correct 3D LUT color remapping) between 2.2 and 2.4 alone - can result in color differences at around the same scale as "WCG or not" (in medium saturated colors) - introducing a 25 point slider for gamma remapping means, that Panasonic has probably given up on color remapping altogether, because no one in their right mind would create 25 different color maps for 25 different gamma transformations. The range betwen -12 and +12 most likely causes saturation differences larger than between bt709 and "WCG" (DCI-P3 @100nits) - on medium saturated colors. :) )
     
    Last edited: Jan 26, 2018
  10. asharma

    asharma New Member

    You’re waaaay above my head...the Panni does provide a slider...
     
  11. asharma

    asharma New Member

    And the latest from Oppo...4 modes for sdr bt2020 (WCG)

    • Mode 1: Optimized to preserve the most highlight details
    • Mode 2: Optimized to preserve a balance between highlight and shadow details
    • Mode 3: Optimized to preserve a balance between mid-tone and highlight details
    • Mode 4: Optimized to preserve the most shadow details.
     
  12. n_p

    n_p Active Member

    In the video thats embedded on the forum page I posted above - Vincent from hdtvtest solves the "output color space mystery" for both the Oppo and the Panasonic HDR to SDR color conversion.

    Oppo has a toggle to choose between

    bt709 @100nits (probably 2.4 gamma (plg))
    or
    DCI-P3@100nits (probably 2.4 gamma (plg)) ("bt2020" but not really ;) )

    Panasonic chooses depending on the Displays EDID (in most cases rec 709, in some "bt2020").

    In regards to color remapping:
    Oppos is "off" (as in problematic) according to Vincent
    Panasonic, at least in one of the 25 gamma transformation modes, does seem to do "some form of color remapping".


    What does this mean in regard to this thread?

    @asharma, wants a toggle selection between bt709 and bt2020 HDR>SDR remapping, that even the 600USD panasonic doesnt provide. Skipped entirely over all gamma remapping issues that are causing severe conversion problems, that Panasonic chose to mitigate with a 25 step slider, so the user could decide on the gamma curve thats displayed, skipped entirely over the color mapping issues - that have to be addressed, but really - cant be easily solved, and always will introduce issues - depending on the scene. Skipped over the fact, that the 500 USD Oppo player, doing a HDR>SDR conversion, doesnt have any color mapping resulting in posterization artifacts all over the image - and that he expects the same kind of color conversion the Panasonic grants - out of a chinese 100USD Android Box with no dedicated image processing.

    To be really honest - Vincent scurts on the outer edge of being deceitful, in his assessment, that the Panasonic does "usable" color mapping in its HDR>SDR conversion - by showing one high contrast scene, with dark surroundings, and no natural lighting. All thats visible in the video is, no/less posterization (color compression artifacts).

    Most of the color mapping issues he doesnt even address.

    And I bet my entire months salery, that said video is the only reason, why Panasonic is praised for its HDR>SDR conversion capabilities.

    @aharma is angry - because his TV, or projector do not display bt2020 in an info box mostly, but has no idea at all, what it means to have to change the gamma transformation setting depending on "Studio" or "Movie" - or that highlight clipping and color space compression alone will result in more inaccurate colors, than a standard Blueray (rec709 video) will ever produce on his screen.

    Zidoo for the most part is "selling" a feature, it cant deliver on. Its theoretically impossible. Even the arguably best implementation on a device six times as expensive as the Zidoo box, and from a multinational in the video processing space, has you moving a 25 pt gamma slider depending on the studio that released the content (/nits level it was mastered) - and proponents of this solution talk about "burned out detail" - but not the color mapping issues that come with selecting a gamma on the fly.

    So now - that more details have come to light - decide for yourself, If you want to buy into the promise of this "feature" - or not.

    Just - for the first time in your life - maybe decide against following the promise of a feature that marketing has sold you on to begin with. Maybe go with the explanation - that none of it is technically possibly - and everyone is winging a made up "solution" to an issue thats not solveable.

    If even one Zidoo box is sold because of this feature - the buyer has been hoodwinked. The same goes for Oppo or Panasonic Players to at least the same degree. With different issues in their implementations.
     
    Last edited: Jan 26, 2018
  13. asharma

    asharma New Member

    Thanks, I think the review u posted on the Oppo is dated...this Oppo beta FW was just released today giving sdr BT2020 4 modes:

    • Mode 1: Optimized to preserve the most highlight details
    • Mode 2: Optimized to preserve a balance between highlight and shadow details
    • Mode 3: Optimized to preserve a balance between mid-tone and highlight details
    • Mode 4: Optimized to preserve the most shadow details.
     
  14. n_p

    n_p Active Member

    Thank you for me putting up with me voicing my frustrations. :)

    To condense the information above -

    - the best implementations of a HDR to SDR conversion (which is always "non standard") allow the user to remap the gamma curve depending on the content that is displayed.
    - this becomes especially necessary, as different movies are mastered to different light levels (currently 2000 vs 4000 nits), so whats "desired to be visible" in the SDR conversion changes depending upon the movie, and you see users out there using the gamma remap feature, to remap HDR gamma to SDR gamma differently depending on the movie they are watching.
    - Every one of those gamma remappings, regardless of which you choose, introduces colorshifts, that you would have to address with 3D LUT color remapping, which probably none of those devices does. Hence color errors are introduced.
    - Furthermore, because you have to compress the colorspace, discard color information - in the Oppo case (maybe on older firmware), there were visible posterization artifacts than result from compressing a much bigger 10bit color space into a much smaller 8bit color space for reproduction, you have to do some form of color mapping to avoid that. The Panasonic player apparently does, but certainly not for all of its 25 custom gamma remap settings, it offers the user - Hence, more color errors.
    - Also "burnt" details, in that you cant prevent - and a compressed dynamic range, which means, that if you set your gamma transformation so that you'll see more bright details, you'll loose that range elsewhere in the image, probably at the shadow detail level - because SDR sadly only can cover a range of about 100 nits (at 8 bit resolution).

    - You can fight for getting "bt2020" support in your downconversion (~= to a DCI-P3 @ 100 nits colorspace), but everyone of the color errors mentioned to be introduced above, has the potential to move midtone colors in "any" direction by about the same amount you fight to get an "extension for", by having Zidoo offer you a bt2020 SDR output mode - so color accuracy simply becomes an impossible thing to begin with. A bt2020 SDR output mode would allow for more saturated colors in some scenes though - but thats all you are getting at that point. :)

    I understand that owners of projectors and non HDR capable TVs want to benefit from 4K Movie releases and the now available DCI-P3 color space as well - but sadly, there is no clean solution for them to "get it" without sacrificing many things they would not sacrifice if they were sticking to rec709 content (mainly color accuracy, but also gamma related stuff, like image depth, or highlight details).

    - If the image the Zidoo produces in this "downconversion mode" should turn out to be severely washed out looking - compared even to a normal Bluray movie - this would be an issue Zidoo would have to address (It would mostly indicate, that they might limit the colorspace, but dont do a gamma transformtion).

    - If its just you not reaching DCI-P3 colors, one could argue, that you have bigger issues - all things considered, but yes a toggle for that would be nice as well. :)

    It would be great if you could get all the DCI-P3 information your display/projector is capable off out of an HDR Movie, but sadly - this is not how that works - for all the above reasons, mainly - because there is no standard people could follow for the downconversion and backwardscompatibility never was designed to be a thing to beginn with.

    All solutions are creative mitigations at its best, and "interesting and funny" at its worst.. ;)
     
  15. asharma

    asharma New Member

    Thanks for the detailed info...Feedback on the new Oppo sdr bt2020 conversion is very positive...if we don’t let perfect be the enemy of the good, Oppo users with projectors should be very happy...if only we could get this functionality in a streamer...
     
  16. n_p

    n_p Active Member

    Here is a visualization to help you understand this issue - thanks to HDR 10 being entirely broken as a standard. :)

    [​IMG]

    What you see in the image are the "five best 4K displays of 2017" (five of them at least).

    All of them (in one case I am not sure.. ;) ) are UHD Alliance Premium Certified, which is the highest commercial HDR "standard" on the market.

    All of them are calibrated by isf and THX certified calibrators (*yawn* ;) ).

    The three on the left use the EXACT SAME PANEL.

    But. :)

    The three on the left use gamma curves that are slightly different to each other (they shouldnt, but HDR 10 never defined how to handle the "running out of brighness" case - so everyone does it differently).

    The second from the right has the highest peak brightness, but is still nowhere near where the content was mastered, and the one on the right is - well its another different one. ;)

    On all those "professionally calibrated" screens, you dont see the same color twice, which is the described combination out of "different gamma curves" and "color space compression" on a "once per movie basis".

    Converting HDR to SDR has the same issues, just more of them.. (parts of the picture details "burning out", far harsher dynamic range compression... ;) ) ;)

    Now, looking at those images - all of those TVs can display the "most saturated colors possible in DCI-P3", just at different brightnesses.

    So if you reach your goal, you'd get some version of that. No one can say what colors get displayed - but some of them are more saturated for sure.. :)

    src for the screenshot is:
    h**ps://www.youtube.com/watch?v=Jv0T5uWv7ME

    Also - none of the sales people will cop to it, and most of the journalists in the field, really dont understand much about what they are writing about... So fiascos like this can become the standard for 90% of all the 4K movies that get sold today. Great, isnt it?
     
  17. asharma

    asharma New Member

    The 2nd from the left, almost looks the most accurate as the highlights on her headpiece are not blown out all to hec...BTW, I ordered the Oppo 203 and will test inputting the Zidoo mkv output into the Oppo’s new tone mapping hdr->sdr conversion feature ...should be interesting...
     
  18. PacoRabanne

    PacoRabanne Well-Known Member Beta test group

    asharma, you have money to spend, you are lucky ;)
     
  19. n_p

    n_p Active Member

    Let me introduce you to the concept of a camera iris (called shutter) and white balancing. :)

    In reality - even the brightest TV you see in those images, cant even reproduce 2/3's of the peak brightness the movie was actually mastered in.

    Why its mastered that bright - is a different story.. Especially since in the cinema most of you saw the same movie with a 60 nits peak - and now all TVs you are looking at are showing highlights at at least 600 nits. ;)

    Of course the human eye cant handle even that contrast remotely, so your pupils are contracting instantly "deleting" lets say 30% of the darker detail thats also on screen. But thats what we want, because its more lifelike... ;)

    The main point I want to have sit with you is - that thanks to HDR10 - all of those TVs are compressing the living HECK out of the colorspace ("color volume") thats encoded on the disc - and this results in high color variation - all of it (ok, most of it.. ;) ) wrong.

    Also - if one TV "wins" in one scene, just show a different scene - and the result will be entirely different.. ;)

    Ha - made myself lough there...

    Dont play "which manufacturer is best", just be one of the first to acknowledge, that the standard is broken. ;)

    With the BEST follow up standards that are at the starting line (Dolby Vision, HDR 10+) the issue doesnt get fixed, btw. it gets "restricted" to a subset of scenes.

    With HDR 10+ trying to do this "fully automatically" with no color grading professional looking at any version of their "graded content". Great cost savings.. ;)

    Ha - there, I smirked again.. :)
     
  20. asharma

    asharma New Member

    Thanks again for the insights...you know your stuff!
     

Share This Page