Feedback [PSA] The X9S outputs wrong color levels

Discussion in 'ZIDOO X9S' started by n_p, May 31, 2017.

Thread Status:
Not open for further replies.
  1. wesk05

    wesk05 Member Beta test group

    1, 3 & 4. I have no idea why think that there is smoke being thrown around to conceal the issue or PR is trying to mitigate the issue. If this is directed at me, I have to say that I have absolutely nothing to do with Zidoo. I have declined offers of free review samples from all companies (big & small) who have ever contacted me. The devices that I comment on are all personal purchases.

    2. There is no separate image controller here. It is all contained in the SoC (RTD 1295).

    5. Zidoo can easily look into this issue. I know from @mirror 's post on Kodi forum that Zidoo has a AstroDesign VA-1809A HDMI analyzer that is perfectly capable of doing colorimetry measurements @ 1080p.
     
  2. n_p

    n_p Active Member

    @wesk05, I've already acknowledged, that your findings with Kodi/ZDMC and ONLY with Kodi/ZDMC were correct and not a PR plant. (I'm sorry for suggesting it.) But your idea of attributing the error to measuring a screen and not the signal, just so you'd have an explanation for your results was wrong and going by the data that already was posted, even a little unfair. That was a smoke screen.

    I did the last outstanding measurements - and now the picture is pretty much clear.

    Test scenarion 1:
    All measurements taken at the following settings: X9S set to YCbCr 444 output at 1080p60, TV set up accordingly (limited color space), X9S going directly into the TV.

    1. Color measurements taken in Kodi/ZDMC:
    [​IMG]
    Color errors are low, and pretty much "where expected".

    2. Color Measurements taken in Mobileforge (Android Pattern Generator):
    [​IMG]
    Color errors explode

    3. Color Measurements taken in Voodoo Screen Test Patterns 3.4 (a different Android Pattern generator)
    [​IMG]
    Color errors explode, same error profile as above (in regards to chroma).
    --

    Test scenarion 2:
    All measurements taken at the following settings: X9S set to YCbCr 444 output at 1080p60, TV set up accordingly (limited color space), X9S going through a 3D LUT box into the TV (not 3D LUT enabled).

    1. Color measurements taken in Kodi/ZDMC:
    [​IMG]
    Color error suddenly appears, same rough "amount and direction" as in the color error cases above.

    2. Color Measurements taken in Mobileforge (Android Pattern Generator):
    [​IMG]
    Color error roughly "doubles in amount and direction" to the Mobileforge measurements in test scenario 1 (when connected directly to the TV).

    3. Color Measurements taken in Voodoo Screen Test Patterns 3.4 (a different Android Pattern generator):
    [​IMG]
    Color error roughly "doubles in amount and direction" to the Voodoo Screen Test Patterns measurements in test scenario 1 (when connected directly to the TV).
    --

    Conclusion.

    When set to YCbCr the X9S outputs wrong color levels on the System Color level.

    Mobileforge for example specifically expects the Android device to take PC Level colors (0-255), like any other Android App, then downconvert them if neccessary according to the output settings. Thats not happening on the X9S. "App level" Colors are displayed wrong. This affects other Android apps as well (tested with Voodoo pattern generator)

    For some reason though - in Kodi/ZDMC, when feeding a video file pattern, the color output readings are correct, when feeding the signal directly to a TV (which most of the people out there probably are doing).

    This probably means, that the SOC handles "video level" (as in GPU assisted?) and "system level" color differently, but does it in a way that introduces significant color errors on the System/App level.

    Also -

    When going through my LUT Box, the color errors double in amount and in the same direction (chroma), and now even Kodis/ZDMCs video output shows the error that the App/System level showed before. Meaning, that theres something seriously off in terms of the device handshake and being standards compliant.

    -

    Broken down into simple words: All your apps colors are probably wrong on the X9S, but the video colors in Kodi/ZDMC are ok, when you have the device at auto/default settings (= YCbCr modes). If you feed the signal through a LUT Box or probably even an AV receiver, all bets are off, because Zidoo does "something" to the output signal, that makes it not standards compliant.

    Its as simple as that.
    --

    How to fix this.

    First, if you are Zidoo - concentrate on YCbCr (as it is the defacto AV standard) and make sure, that both system level (app) colors, and video colors are displayed correctly. Thats expected behavior.

    Second, if you are Zidoo, make sure that whatever signal you are puting out is actually standards compliant. Ie - when you are saying, that you are outputing a 16-235 signal, don't output a 0-255 signal with the colors remapped to the 16-235 range. It will confuse AV equipment.

    If you are a user and want both correct colors in apps and in video, you have to switch to RGB444 (and set your TV accordingly (black level high, full, 0-255 - or however the setting in your TV is called)) - at which point system level and video level should align (will double check this as well). The bad news is, that the X9S forgets that you've switched to this setting after reboot.

    Shorthand, its a mess - but some people might find it workable.
     
    Last edited: Jun 11, 2017
  3. wesk05

    wesk05 Member Beta test group

    1. I was making a statement about your testing methodology. I didn't quite discredit your results. I am sorry if you took that as a smoke screen.

    2. This is one thing that I can agree upon, but this again is not unusual. Amazon Fire TV does the same thing. Even Kodi as a picture viewer does it on all Android platforms.

    3. You keep mentioning about your LUT box, what brand is it?

    4. That is not what happens on Zidoo. In YCbCr mode it is 16-235 but also passes 0-15 and 236-254 and this is exactly the way it should be. In RGB mode it scales and remaps 16-235 onto 0-255 and outputs it as 0-255. Technically this is OK, but I am personally against rescaling/remapping because you are no longer watching the content true to the source.

    In my previous post, I asked you about the settings that you use on Calman/MobileForge. With the right settings YCbCr test will also be correct. If you change the data format for "stimulus" from "percent" to "levels" in Calman you will have a better idea of what is going on. Keep in mind that Zidoo will output 16-235 for YCbCr and 0-255 for RGB.
     
  4. n_p

    n_p Active Member

    @Amazon Fire TV "doing the same". In regards to having a different output set for video and system level maybe - in regards to doing it wrong and producing errors - no. I've tested both a Fire TV Stick (Gen 1) and a Fire TV (Gen 2) and they were without issues in all testing scenarios. Thank you also for the Information in regards to Kodis image viewer. .)

    @3: Its an eecolor (I think they only make one.. :) )

    @4: That would explain the second part of the issue in regards to my LUT Box.

    The LUT box looks at the signal, sees that it is 0-255, treats it as a "PC Level signal", but it already is remapped to 16-235 (or 16-255) and thats causing the issues. I'm aware that having the option to output 236-255 alongside a YCbCr signal "is a thing" - but no other device I know of comes with that enabled by default, and with no option to disable it.

    Do you know if sending "whiter than white" information alongside a YCbCr signal is "some sort of a standard" - I get that it can be technically fine, but is it standards compliant? Amazon for example goes out of their way to have 0-16 and whiter than white (?) elimited from the output, as I understand it - to be standards compliant.

    Also - sending the blacker than black information alongside a YCbCr Signal just seems strange to me...
    -

    Recap after the additional information:

    Two issues:

    1. The X9S doesn't map system level colors (most Apps, Interface) correctly, when outputting in YCbCr modes (video color levels are fine)

    2. The X9S does output the YCbCr information with blacker than black and whiter than white information (information thats outside the YCbCr color range) always active (always active with no option to disable it, to my knowledge should qualify as "not standards complient"), causing potential further issues with other AV equipment. Amazon (f.e.) goes out of their way not to do that.
     
    Last edited: Jun 13, 2017
  5. wesk05

    wesk05 Member Beta test group

    YCbCr full range was first defined in HDMI 1.4. It wasn't allowed in lower versions. Your eeColor LUT box is HDMI 1.3(?a). If I remember correctly, 1-254 is a valid YCbCr range for all HDMI versions.

    I am familiar with eeColor LUT box, but personally I have only used Lumagen video processors.

    You haven't mentioned Calman/MobileForge settings that you use when you do these tests. I believe your YCbCr issues are due to incorrect settings.
     
  6. qxy

    qxy Member

    Interesting.I notice the blue color of folder icon in bubbleupnp app installed on zidoo is very different from others,such as smart TV, mobile phone.
     
  7. n_p

    n_p Active Member

    Please offer documentation for this. Everything I've read about HDMI versioning says, that this is not the case.
    (f.e.: http://denon.custhelp.com/app/answers/detail/a_id/192/~/differences-between-hdmi-versions-1.1,-1.2,-1.3a,-1.4-and-2.0?)

    edit: Found one mention of it on the entirety of the internets.. ;) :
    http://forum.doom9.org/showthread.php?s=89f7812526eaf86d2b7d651eb2dedafe&p=1766801#post1766801

    Even if thats the case - its a defacto standard, that YCbCr delivers a limited color range (= there should at least be a toggle implemented by Zidoo. YCbCr full range is not a default anywhere in the industry), and AGAIN - there is a mismatch between Kodi/ZDMC and System/App colors, when its set to YCbCr. This can't be explained away by this suggestion. Also the missmatch is still an issue, when connecting the X9S directly to a fully compatible HDMI 2.0 TV.

    Also a black bars testpattern in Kodi/ZDMC shows, that the color level output in Kodi/ZDMC at that point is limited range (not full PC-Range), and the TV has to be adjusted adequately. So this is definitely not the X9S outputting a "correctly specified" YCbCr full range signal (Kodi/ZDMC still outputs limited in the only case - where Kodi/ZDMC Colors match up with whats expected, with the X9S on YCbCr settings.).

    I checked that afair, so I'm putting it out there that this is not the case (and that this is another smoke screen ;) ). I'll recheck it asap and will edit this thread as soon as I've done so.

    Settings for the screenshots above were "video range" with, "expand to PC Level" enabled for the tests, as Calman instructs you to do so in the MobileForge manual (http://www.spectracal.com/Documents/QSGs/SpectraCal MobileForge QuickStart.pdf - Android devices/apps use PC Level colors internally - then the device is supposed to convert that to limited range on output, if its designed correctly).

    As far as I remember, enabling/disabling "set to PC levels" didn't change the results hardly at all.
     
    Last edited: Jun 20, 2017
  8. n_p

    n_p Active Member

    Did the required tests. The System/App and video mismatch issues can not be blamed on HDMI 1.3, or the X9S outputing "YCbCr full range". In fact - it doesnt. When the X9S is set to YCbCr (== default video settings), it outputs a limited range signal (as expected for a YCbCr default setting) - its just, that colors on the System/App Level (UI, menues, apps, ...) are outright wrong.

    Test setup: X9S connected directly to the TV (to a HDMI 2.0 compatible port) - measurements taken in MobileForge (Calmans Android pattern generator). X9S set to a YCbCr output mode.

    Calman set to measure 16-235 patterns with "expand video levels to PC levels" enabled, TV set to expect a limited signal (= correct settings for the entire signal chain. :) ):
    [​IMG]
    Color errors occur
    Luminance graph shows that black level is set correctly.

    Calman set to measure 16-235 patterns with "expand video levels to PC levels" disabled, TV set to expect a limited signal:
    [​IMG]
    Color errors occur
    Luminance graph shows that black level isn't set correctly.

    Calman set to measure 0-255 patterns with "expand video levels to PC levels" disabled, TV set to expect a limited signal:
    [​IMG]
    Color errors occur
    Luminance graph shows that black level is set correctly.

    Calman set to measure 0-255 patterns with "expand video levels to PC levels" enabled, TV set to expect a limited signal:
    Within measurement tolerances to the last one, as expected - so I am just posting a link to the image -
    http://i.imgur.com/oJT0KK9.png
    -

    Just because I was curious - I also, wrongly, set the TV to expect full signal range, while having the X9S still output YCbCr signal - and the result just was that all errors increased, as now black levels were wrong in all cases. :) == X9S set to YCbCr does NOT output a pc range signal - and telling the TV that it does - DOESNT fix the very apparent color errors on the System/App level.

    limitedexpandtopc:
    http://i.imgur.com/QOk1LOE.png
    limiteddontexpandtopc:
    http://i.imgur.com/ACRKkpc.png
    fulldontexpandtopc:
    http://i.imgur.com/5sZBHpk.png
    fullexpandtopc:
    http://i.imgur.com/5WU64Z7.png
    --

    Conclusion:

    As decribed above, the X9S still outputs colors incorrectly on the System/App level - regardless of pattern input settings. Also - the cause of it is NOT that the X9S outputs "YCbCr full range", which would only be supported with HDMI 1.4 and above.

    In short. BS, BS and BS - and the color output in all Apps and Menus that arent Kodi/ZDMC is still wrong.

    Zidoo has chosen not to react to this so you can draw your conclusion from that as well.

    Short slogan to rally behind:
    If you don't want your Android TV Box to increase color error by +9 dE 2000 and above in everything but the Kodi/ZDMC video player, stay the heck away from Zidoo products.
     
    Last edited: Jun 20, 2017
  9. PacoRabanne

    PacoRabanne Well-Known Member Beta test group

    n_p, you mean that if anyone plays a video with anything but ZDMC or Media Center native player X9S renders wrong colors?
    For example with Youtube app (standard or Android TV versions) or MX Player or VLC?
    Are you able to do some tests with some or any of the many video calibration files available around (saw some on a italian forum on AVMagazine site) and any of the "other" video player app?
    (don't worry if you're non interested to, and your answer is "No" ;))
     
  10. n_p

    n_p Active Member

    I'll do those tests for youtube, MX Player and VLC if you want, but at some point you have to ask yourself what a sad state this industry is in, when we have to whitelist the apps, that color reproduction on our TV boxes isn't f*cked up in.

    I'd honestly would like to see some engineers that start to give a thought about, what they are pitching to sell the world.

    But who am I kidding, kids these days don't buy devices, because the color reproduction is correct - they buy because of the "my favourite youtube celebrity said - factor", and "whatever 3 out of 10 random people on the internet said, when I asked them what to do/buy".

    Its a freaking paradise for marketing people out there - and I'd like to not support that even further by compiling a list of where a device is still usable, despite a major f*ck up in product design.

    So thats where I am motivation wise.

    But I'll do those three apps, just because you asked and I'm interested. :)
     
    Last edited: Jun 21, 2017
  11. n_p

    n_p Active Member

    MX Player
    [​IMG]
    No significant errors

    VLC
    [​IMG]
    No significant errors

    Youtube App
    [​IMG]
    No significant errors (The still noticable increase in dE could be because I had to recycle the 100% white reading from a different target, as my test pattern here only had 75% color targets.)

    Chrome, on an image (as in not video) pattern
    [​IMG]
    AND THE ERRORS ARE BACK. *heres looking at you Zidoo*

    So whenever you have an app that plays back video (maybe connected to HW acceleration, but maybe not), the video color output seems to be "roughly ok", but whenever you have an app, or the system showing you an image, a generated color field, a menue, ... the colors are very, very wrong.

    And just to prevent the next smoke screen from being set up, this is not because of the sRGB>rec709 difference, because there is (virtually) none. :) Gamma measured was 2.2 in all cases, so thats correct for desktop publishing (the sRGB image standard) as well.
     
    Last edited: Jun 21, 2017
  12. PacoRabanne

    PacoRabanne Well-Known Member Beta test group

    Thanks sooooooo much!
    You're doing a lot of very interesting tests (from my point of view). I'm sorry I'm a completely newbie in this specific argument, But I'm curious and I do a lot of search around. Unluckily english is not my native language, so I've to work hard to understand the most :oops:
    Indeed your result (that I trust! at least until Zidoo demonstrate something different, and this seems not happening at all! Real pity!) make me sorry. This Zidoo devices appears as well designed and developed, but I'm starting to suspect that also for Zidoo business is not making old device works at the best, but sell another new device. I don't think this is the best way, but I'm a old generation dreamer...

    Anyway thanks again!
    (sorry for my bad english!)
     
  13. n_p

    n_p Active Member

    This could very well be kernel related with Zidoo hacking together a workaround so the chip they sourced can be used with YCbCr (THE AV standard) at all. Solving those issues are cost drivers, you need real devs for that that know what they are doing.

    My issue lies with problems like this (that thing simply outputs colors that are all wrong - go figure) can lie dormant in the current ecosystem for as long as they can - because our Chinese/Taiwanese manufacturers dont care about complying to standards, if it saves them money, the "youtuber" ecosystem is severely unequipped to do anything that resembles real testing (all they do is benchmarks that are useless and playing 5 videos from their video folder), and the market doesn't self correct, because of a lack of information.

    My issue lies especially with Zidoo positioning them as "the premium cheap box", because theres nothing premium about "not getting color correct on a media player", and their business model is to develop their software suite as they stumble along. At some point there is a cut off for (probably after a year), they port over their software base to the new chip and thats it.

    Still, a reaction from the company would be nice - just to see if they plan to continue with wrong color output throughout their product line, just playing the "youtube (= POS marketing) don't care" game.

    Any reputable AV manufacture would be horrified to have such an issue surface. Their reputation loss would be enormous but with Zidoo - it just ends at "who?" - and the next youtuber that wants to tell you how great a device is, based on a synthetic benchmark he downloaded from the play store...

    I'll end with - It would be great to have Zidoo publicly address this issue, as it is a big one.
     
  14. nikos_a

    nikos_a Active Member

    n_p, your findings are VERY interesting. I would also like to see an official answer from Zidoo. Just one question though, since I see you focus on Zidoo hacking the kernel. Could this be a Realtek issue from the beginning so that they "finetune" (trying to be kind here :p ) the color levels when playing video only? As I see it, the "excuse" can be that 1295 is mainly a media player chipset that has the unfortunate luck to co-exist with Android (at least for now) meaning that someone can run several apps too. Realtek has always being one step behind chips from other companies (like Sigma) in terms of picture quality.

    Just trying to say a different scenario, not trying to defend Zidoo or anything else. For me your findings are spot on and well done you spent time on that.
     
  15. boblo

    boblo Member

    @n_p: Did you try these tests with new firmware 1.4.6? I think you should repeat your tests with this version, just in case Zidoo would have addressed this problem.

    Thanks for your efforts.
     
    Pelayo likes this.
  16. boblo

    boblo Member

    It would be nice to have an official reply from Zidoo. Currently, this is the only deal breaker for me to purchase a X10. If they are not going to work on this, I'm not going to buy the box.
     
  17. n_p

    n_p Active Member

    All my tests were done on 1.3.0 - but until I read in a changelog, or a dev posting, that they have addressed it - I'm not falling for the "issues could have been updated away" gambit. ;) This most likely is kernel level stuff and hard to fix, so it seems unlikely that they would fix this "silently". ;)

    Also, the next time you hear "we could add that in software" in regard to an issue with an Android product, think "never, ever going to happen". ;)

    Not to discourage Zidoo, because at some time, they'd have get colors correct, if they want to stay in the game.. :) The next competitor, that manages to do so (and offer root :/ ), has my money. :)

    Btw, everyone with 180 USD of equipment (X-Rite i1Display Pro, or a similar probe, HCFR (free), and thee Voodoo Screen Test Patterns 3.4 apk, or an image testpattern (free) can test this (on other android boxes as well)). The errors are that obvious. ;)
     
  18. wesk05

    wesk05 Member Beta test group

    The issue here is with converting RGB Full to YCbCr. The matrix that is being used for conversion is not correct at this time. Use a proper matrix, convert and then clip to 16-235 or let 0-15 and 236-255 passthrough. This issue only affects the desktop color and for apps like games/browser that is using RGB Full. If you know that the source is going to be RGB Full, then it is better that you use RGB Full as the output.

    For all apps that play video, YCbCr output is 100% correct. The output values are an exact match to the input values.

    Test pattern images are not proper source for calibration. You have to use video test patterns for calibration.
     
    Last edited: Jun 27, 2017
  19. n_p

    n_p Active Member

    @wesk05

    Do you know where the conversion step is "situated"? Ie, who most likely wrote the code for it (the chip manufacturer, Zidoo, Google, a Linux kernel dev, way back when, ...)? If not, then a no is fine. :)

    I've other issues with using the X9S in full RGB mode (RGB 444 in the settings),

    - The first one I already mentioned, is that the RGB 444 setting didn't seem to survive a reboot and had to be toggled off and on again, several times during my testing. I remember that one time it did stick, but that might have been a reboot and not a full power cycle... And I've had enough intermittent issues for a while (didn't hunt this one down to say for sure, when...).

    - The second one is usage case related. I use my LUT box with several inputs and as the AV standard is limited, most of my other devices are set to limited. The practical example for this one is - if I set my PS4 to output full RGB (PC range 0-255), in the odd case, and then I put a DVD into that thing, the black level won't be shifted. So its back to limited again... to also go into the TVs menu everytime you switch an input on that HDMI port (I own one LUT box, not four.. ;) ) is something that isn't practical.

    - Third issue is, that it is "somewhat" common for TVs to have better color reproduction out of the factory in limited color space. Now we are talking much smaller error margins than in the "X9S sports wrong color output" case, but the cause is similar. Real time color conversion not done on a 10 bit level, rounding errors, worse accuracy, depending an the chipset. There is something to be said about supporting the industry standard (YCbCR limited range), because others are optimizing for it as well.

    - Fourth issue is, that because of that - at least with my LUT box, that device in the signal chain gets confused (might be HDMI 1.3 related, might be LUT box specific (that thing actually takes the signal, maps it to 10 bit internally, than does the conversion).

    - Fifth issue is, that people don't change defaults in their tech (they don't ;) ) , and in the case of the X9S they are warned not to do so, in the quickselect menu itself (for reasons only the manufacturer understands.. ;) ) .

    So its not really as easy as "just use the other mode" in actual use, I'd much rather "simply use a different device". :) But if no one acknowledges their design mistakes for marketing reasons, where do I start looking? I'm lost as well.

    I know Amazon doesn't have this issue, but they are the opposite of open, and root is an issue. If you go for the "enthusiasts" market with a device like this, making sure that the color output is ok, in apps AND movies, on the most commonly used setting, isn't too much to ask for, I think.

    (Also there is still the issue with the HDMI IN port always expecting it will get a full range signal... ;) Again, thats not the AV standard. :) )
     
    Last edited: Jun 27, 2017
  20. wesk05

    wesk05 Member Beta test group

    1. It is likely to be in Realtek's SDK.
    3. Is this RGB Limited or just YCbCr you are referring to? If it is YCbCr, then there is zero conversion error in Zidoo's YCbCr output from a YCbCr/YUV source (all video content). If the source is RGB, then there are some tiny errors in the output. This is what happens with Mobile Forge when Zidoo output is set to YCbCr.
     
Thread Status:
Not open for further replies.

Share This Page