Feedback [PSA] The X9S outputs wrong color levels

1. I was making a statement about your testing methodology. I didn't quite discredit your results. I am sorry if you took that as a smoke screen.

2. This is one thing that I can agree upon, but this again is not unusual. Amazon Fire TV does the same thing. Even Kodi as a picture viewer does it on all Android platforms.

3. You keep mentioning about your LUT box, what brand is it?

4. That is not what happens on Zidoo. In YCbCr mode it is 16-235 but also passes 0-15 and 236-254 and this is exactly the way it should be. In RGB mode it scales and remaps 16-235 onto 0-255 and outputs it as 0-255. Technically this is OK, but I am personally against rescaling/remapping because you are no longer watching the content true to the source.

In my previous post, I asked you about the settings that you use on Calman/MobileForge. With the right settings YCbCr test will also be correct. If you change the data format for "stimulus" from "percent" to "levels" in Calman you will have a better idea of what is going on. Keep in mind that Zidoo will output 16-235 for YCbCr and 0-255 for RGB.

 

@Amazon Fire TV "doing the same". In regards to having a different output set for video and system level maybe - in regards to doing it wrong and producing errors - no. I've tested both a Fire TV Stick (Gen 1) and a Fire TV (Gen 2) and they were without issues in all testing scenarios. Thank you also for the Information in regards to Kodis image viewer. .)

@3: Its an eecolor (I think they only make one.. )

@4: That would explain the second part of the issue in regards to my LUT Box.

The LUT box looks at the signal, sees that it is 0-255, treats it as a "PC Level signal", but it already is remapped to 16-235 (or 16-255) and thats causing the issues. I'm aware that having the option to output 236-255 alongside a YCbCr signal "is a thing" - but no other device I know of comes with that enabled by default, and with no option to disable it.

Do you know if sending "whiter than white" information alongside a YCbCr signal is "some sort of a standard" - I get that it can be technically fine, but is it standards compliant? Amazon for example goes out of their way to have 0-16 and whiter than white (?) elimited from the output, as I understand it - to be standards compliant.

Also - sending the blacker than black information alongside a YCbCr Signal just seems strange to me...
-

Recap after the additional information:

Two issues:

1. The X9S doesn't map system level colors (most Apps, Interface) correctly, when outputting in YCbCr modes (video color levels are fine)

2. The X9S does output the YCbCr information with blacker than black and whiter than white information (information thats outside the YCbCr color range) always active (always active with no option to disable it, to my knowledge should qualify as "not standards complient"), causing potential further issues with other AV equipment. Amazon (f.e.) goes out of their way not to do that.

 

YCbCr full range was first defined in HDMI 1.4. It wasn't allowed in lower versions. Your eeColor LUT box is HDMI 1.3(?a). If I remember correctly, 1-254 is a valid YCbCr range for all HDMI versions.

I am familiar with eeColor LUT box, but personally I have only used Lumagen video processors.

You haven't mentioned Calman/MobileForge settings that you use when you do these tests. I believe your YCbCr issues are due to incorrect settings.

 

Interesting.I notice the blue color of folder icon in bubbleupnp app installed on zidoo is very different from others,such as smart TV, mobile phone.

 

Please offer documentation for this. Everything I've read about HDMI versioning says, that this is not the case.
(f.e.: http://denon.custhelp.com/app/answers/detail/a_id/192/~/differences-between-hdmi-versions-1.1,-1.2,-1.3a,-1.4-and-2.0?)

edit: Found one mention of it on the entirety of the internets.. :
http://forum.doom9.org/showthread.php?s=89f7812526eaf86d2b7d651eb2dedafe&p=1766801#post1766801

Even if thats the case - its a defacto standard, that YCbCr delivers a limited color range (= there should at least be a toggle implemented by Zidoo. YCbCr full range is not a default anywhere in the industry), and AGAIN - there is a mismatch between Kodi/ZDMC and System/App colors, when its set to YCbCr. This can't be explained away by this suggestion. Also the missmatch is still an issue, when connecting the X9S directly to a fully compatible HDMI 2.0 TV.

Also a black bars testpattern in Kodi/ZDMC shows, that the color level output in Kodi/ZDMC at that point is limited range (not full PC-Range), and the TV has to be adjusted adequately. So this is definitely not the X9S outputting a "correctly specified" YCbCr full range signal (Kodi/ZDMC still outputs limited in the only case - where Kodi/ZDMC Colors match up with whats expected, with the X9S on YCbCr settings.).

I checked that afair, so I'm putting it out there that this is not the case (and that this is another smoke screen ). I'll recheck it asap and will edit this thread as soon as I've done so.

Settings for the screenshots above were "video range" with, "expand to PC Level" enabled for the tests, as Calman instructs you to do so in the MobileForge manual (http://www.spectracal.com/Documents/QSGs/SpectraCal MobileForge QuickStart.pdf - Android devices/apps use PC Level colors internally - then the device is supposed to convert that to limited range on output, if its designed correctly).

As far as I remember, enabling/disabling "set to PC levels" didn't change the results hardly at all.

 

Did the required tests. The System/App and video mismatch issues can not be blamed on HDMI 1.3, or the X9S outputing "YCbCr full range". In fact - it doesnt. When the X9S is set to YCbCr (== default video settings), it outputs a limited range signal (as expected for a YCbCr default setting) - its just, that colors on the System/App Level (UI, menues, apps, ...) are outright wrong.

Test setup: X9S connected directly to the TV (to a HDMI 2.0 compatible port) - measurements taken in MobileForge (Calmans Android pattern generator). X9S set to a YCbCr output mode.

Calman set to measure 16-235 patterns with "expand video levels to PC levels" enabled, TV set to expect a limited signal (= correct settings for the entire signal chain. ):

Color errors occur
Luminance graph shows that black level is set correctly.

Calman set to measure 16-235 patterns with "expand video levels to PC levels" disabled, TV set to expect a limited signal:

Color errors occur
Luminance graph shows that black level isn't set correctly.

Calman set to measure 0-255 patterns with "expand video levels to PC levels" disabled, TV set to expect a limited signal:

Color errors occur
Luminance graph shows that black level is set correctly.

Calman set to measure 0-255 patterns with "expand video levels to PC levels" enabled, TV set to expect a limited signal:
Within measurement tolerances to the last one, as expected - so I am just posting a link to the image -
http://i.imgur.com/oJT0KK9.png
-

Just because I was curious - I also, wrongly, set the TV to expect full signal range, while having the X9S still output YCbCr signal - and the result just was that all errors increased, as now black levels were wrong in all cases. == X9S set to YCbCr does NOT output a pc range signal - and telling the TV that it does - DOESNT fix the very apparent color errors on the System/App level.

limitedexpandtopc:
http://i.imgur.com/QOk1LOE.png
limiteddontexpandtopc:
http://i.imgur.com/ACRKkpc.png
fulldontexpandtopc:
http://i.imgur.com/5sZBHpk.png
fullexpandtopc:
http://i.imgur.com/5WU64Z7.png
--

Conclusion:

As decribed above, the X9S still outputs colors incorrectly on the System/App level - regardless of pattern input settings. Also - the cause of it is NOT that the X9S outputs "YCbCr full range", which would only be supported with HDMI 1.4 and above.

In short. BS, BS and BS - and the color output in all Apps and Menus that arent Kodi/ZDMC is still wrong.

Zidoo has chosen not to react to this so you can draw your conclusion from that as well.

Short slogan to rally behind:
If you don't want your Android TV Box to increase color error by +9 dE 2000 and above in everything but the Kodi/ZDMC video player, stay the heck away from Zidoo products.

 

n_p, you mean that if anyone plays a video with anything but ZDMC or Media Center native player X9S renders wrong colors?
For example with Youtube app (standard or Android TV versions) or MX Player or VLC?
Are you able to do some tests with some or any of the many video calibration files available around (saw some on a italian forum on AVMagazine site) and any of the "other" video player app?
(don't worry if you're non interested to, and your answer is "No" )

 

I'll do those tests for youtube, MX Player and VLC if you want, but at some point you have to ask yourself what a sad state this industry is in, when we have to whitelist the apps, that color reproduction on our TV boxes isn't f*cked up in.

I'd honestly would like to see some engineers that start to give a thought about, what they are pitching to sell the world.

But who am I kidding, kids these days don't buy devices, because the color reproduction is correct - they buy because of the "my favourite youtube celebrity said - factor", and "whatever 3 out of 10 random people on the internet said, when I asked them what to do/buy".

Its a freaking paradise for marketing people out there - and I'd like to not support that even further by compiling a list of where a device is still usable, despite a major f*ck up in product design.

So thats where I am motivation wise.

But I'll do those three apps, just because you asked and I'm interested.

 

MX Player

No significant errors

VLC

No significant errors

Youtube App

No significant errors (The still noticable increase in dE could be because I had to recycle the 100% white reading from a different target, as my test pattern here only had 75% color targets.)

Chrome, on an image (as in not video) pattern

AND THE ERRORS ARE BACK. *heres looking at you Zidoo*

So whenever you have an app that plays back video (maybe connected to HW acceleration, but maybe not), the video color output seems to be "roughly ok", but whenever you have an app, or the system showing you an image, a generated color field, a menue, ... the colors are very, very wrong.

And just to prevent the next smoke screen from being set up, this is not because of the sRGB>rec709 difference, because there is (virtually) none. Gamma measured was 2.2 in all cases, so thats correct for desktop publishing (the sRGB image standard) as well.

 

Thanks sooooooo much!
You're doing a lot of very interesting tests (from my point of view). I'm sorry I'm a completely newbie in this specific argument, But I'm curious and I do a lot of search around. Unluckily english is not my native language, so I've to work hard to understand the most
Indeed your result (that I trust! at least until Zidoo demonstrate something different, and this seems not happening at all! Real pity!) make me sorry. This Zidoo devices appears as well designed and developed, but I'm starting to suspect that also for Zidoo business is not making old device works at the best, but sell another new device. I don't think this is the best way, but I'm a old generation dreamer...

Anyway thanks again!
(sorry for my bad english!)

 

This could very well be kernel related with Zidoo hacking together a workaround so the chip they sourced can be used with YCbCr (THE AV standard) at all. Solving those issues are cost drivers, you need real devs for that that know what they are doing.

My issue lies with problems like this (that thing simply outputs colors that are all wrong - go figure) can lie dormant in the current ecosystem for as long as they can - because our Chinese/Taiwanese manufacturers dont care about complying to standards, if it saves them money, the "youtuber" ecosystem is severely unequipped to do anything that resembles real testing (all they do is benchmarks that are useless and playing 5 videos from their video folder), and the market doesn't self correct, because of a lack of information.

My issue lies especially with Zidoo positioning them as "the premium cheap box", because theres nothing premium about "not getting color correct on a media player", and their business model is to develop their software suite as they stumble along. At some point there is a cut off for (probably after a year), they port over their software base to the new chip and thats it.

Still, a reaction from the company would be nice - just to see if they plan to continue with wrong color output throughout their product line, just playing the "youtube (= POS marketing) don't care" game.

Any reputable AV manufacture would be horrified to have such an issue surface. Their reputation loss would be enormous but with Zidoo - it just ends at "who?" - and the next youtuber that wants to tell you how great a device is, based on a synthetic benchmark he downloaded from the play store...

I'll end with - It would be great to have Zidoo publicly address this issue, as it is a big one.

 

n_p, your findings are VERY interesting. I would also like to see an official answer from Zidoo. Just one question though, since I see you focus on Zidoo hacking the kernel. Could this be a Realtek issue from the beginning so that they "finetune" (trying to be kind here ) the color levels when playing video only? As I see it, the "excuse" can be that 1295 is mainly a media player chipset that has the unfortunate luck to co-exist with Android (at least for now) meaning that someone can run several apps too. Realtek has always being one step behind chips from other companies (like Sigma) in terms of picture quality.

Just trying to say a different scenario, not trying to defend Zidoo or anything else. For me your findings are spot on and well done you spent time on that.

 

@n_p: Did you try these tests with new firmware 1.4.6? I think you should repeat your tests with this version, just in case Zidoo would have addressed this problem.

Thanks for your efforts.

 

It would be nice to have an official reply from Zidoo. Currently, this is the only deal breaker for me to purchase a X10. If they are not going to work on this, I'm not going to buy the box.

 

All my tests were done on 1.3.0 - but until I read in a changelog, or a dev posting, that they have addressed it - I'm not falling for the "issues could have been updated away" gambit. This most likely is kernel level stuff and hard to fix, so it seems unlikely that they would fix this "silently".

Also, the next time you hear "we could add that in software" in regard to an issue with an Android product, think "never, ever going to happen".

Not to discourage Zidoo, because at some time, they'd have get colors correct, if they want to stay in the game.. The next competitor, that manages to do so (and offer root :/ ), has my money.

Btw, everyone with 180 USD of equipment (X-Rite i1Display Pro, or a similar probe, HCFR (free), and thee Voodoo Screen Test Patterns 3.4 apk, or an image testpattern (free) can test this (on other android boxes as well)). The errors are that obvious.

 

The issue here is with converting RGB Full to YCbCr. The matrix that is being used for conversion is not correct at this time. Use a proper matrix, convert and then clip to 16-235 or let 0-15 and 236-255 passthrough. This issue only affects the desktop color and for apps like games/browser that is using RGB Full. If you know that the source is going to be RGB Full, then it is better that you use RGB Full as the output.

For all apps that play video, YCbCr output is 100% correct. The output values are an exact match to the input values.

Test pattern images are not proper source for calibration. You have to use video test patterns for calibration.

 

@wesk05

Do you know where the conversion step is "situated"? Ie, who most likely wrote the code for it (the chip manufacturer, Zidoo, Google, a Linux kernel dev, way back when, ...)? If not, then a no is fine.

I've other issues with using the X9S in full RGB mode (RGB 444 in the settings),

- The first one I already mentioned, is that the RGB 444 setting didn't seem to survive a reboot and had to be toggled off and on again, several times during my testing. I remember that one time it did stick, but that might have been a reboot and not a full power cycle... And I've had enough intermittent issues for a while (didn't hunt this one down to say for sure, when...).

- The second one is usage case related. I use my LUT box with several inputs and as the AV standard is limited, most of my other devices are set to limited. The practical example for this one is - if I set my PS4 to output full RGB (PC range 0-255), in the odd case, and then I put a DVD into that thing, the black level won't be shifted. So its back to limited again... to also go into the TVs menu everytime you switch an input on that HDMI port (I own one LUT box, not four.. ) is something that isn't practical.

- Third issue is, that it is "somewhat" common for TVs to have better color reproduction out of the factory in limited color space. Now we are talking much smaller error margins than in the "X9S sports wrong color output" case, but the cause is similar. Real time color conversion not done on a 10 bit level, rounding errors, worse accuracy, depending an the chipset. There is something to be said about supporting the industry standard (YCbCR limited range), because others are optimizing for it as well.

- Fourth issue is, that because of that - at least with my LUT box, that device in the signal chain gets confused (might be HDMI 1.3 related, might be LUT box specific (that thing actually takes the signal, maps it to 10 bit internally, than does the conversion).

- Fifth issue is, that people don't change defaults in their tech (they don't ) , and in the case of the X9S they are warned not to do so, in the quickselect menu itself (for reasons only the manufacturer understands.. ) .

So its not really as easy as "just use the other mode" in actual use, I'd much rather "simply use a different device". But if no one acknowledges their design mistakes for marketing reasons, where do I start looking? I'm lost as well.

I know Amazon doesn't have this issue, but they are the opposite of open, and root is an issue. If you go for the "enthusiasts" market with a device like this, making sure that the color output is ok, in apps AND movies, on the most commonly used setting, isn't too much to ask for, I think.

(Also there is still the issue with the HDMI IN port always expecting it will get a full range signal... Again, thats not the AV standard. )

 

1. It is likely to be in Realtek's SDK.
3. Is this RGB Limited or just YCbCr you are referring to? If it is YCbCr, then there is zero conversion error in Zidoo's YCbCr output from a YCbCr/YUV source (all video content). If the source is RGB, then there are some tiny errors in the output. This is what happens with Mobile Forge when Zidoo output is set to YCbCr.

 

Could you elaborate this, please? Does the HDMI in app record wrong colors too?

And are colors displayed on games/browser way too much deviated? Is it noticeable at first sight (without special tools)? Could this issue be similar to displaying sRGB images on a Wide Color Gamut monitor? I had a WCG monitor in the past and I had to send it back because colors were horrible.

Thanks to all

 

The material to learn this stuff is out there. I have a chip on my shoulder with people trying to to get "simple breakdowns of everything" on the internet. If you have objectifyable data in front of you, and you ask for a subjective breakdown by a random internet user, in the end you learned nothing and still didn't understand the issue.

First HDMI IN (recording): I didn't check for color accuracy there. But if you connect any source to it, and play back a black clipping pattern (the thing you use to set your TVs brightness level), you'll see, that the X9S expects the input signal to be full range (== pc level, == 0-255). Otherwise the black level is visibly off. As in - you don't even need the black clipping pattern to see it, if you know what you are looking for. This is an "issue" as the standard for all AV equipment is limited color range. The same way that nowadays every graphics card driver has the option to choose between video level and pc level output (limited and full), recording equipment usually has a toggle for that as well. The X9S not only is missing such a toggle, it also is defaulting to the PC standard here, which hardly makes any sense, as the box uses YCbCr (limited) by default, and tries to play in the "video realm" of things. Thats an issue. black levels don't line up with default settings even in the best case scenario, and the X9S would like to force you to switch your entire video chain to "pc level" which is not standard.

The color errors on non video content (menus, apps, games, ...), even in the best case scenario for YCbCr are significant and clearly visible. Above 3 dE 2000 they are considered visible, above 5 they are considered really visible (without a direct comparison to another screen), and Cyan is ending up at 11, Green is ending up at 9 and Yellow at above 5, from a baseline (how the screen performs) thats well below dE 3.

The last time I checked, also meassuring tools were not used to see things, so when I talk about visible color differences, and I did before in this thread, ... yeah...

"But will I see them?!"
(The question thats really hidden behind the "can someone elaborate" sugarcoating..)

It depends, but yes.

dE (delta Error) is a measure of noticable difference in direct viewing (a bunch of people look at color fields in a scientific experiment setting). So by that standard - yes. (See above)

It depends also on your monitor. But chances are that it isnt wrong in the adverse direction, so - yes again.

It depends on your color vision. But the CMF (color matching function) thats used to describe the targets and the error amount is an average from a "normal" color perceiving population already - so yes, again.

It depends on your color vision still, as an average is just that, an average (in this case even weighted, so more a median), but probably - yes, again.

It also depends on the display technology you are using, as more and more "modern" screens, start to kreep into the territory, where metamerism failure starts to become an issue (color primary spectrum peaks become "too" narrow, and color perception variability explodes even within the normal color percieving public), but as this issue normally only increases perceived color error, yes - again.
-

Ok, after all of this, here is your actual answer. We make open standards in industries, so that all kind of different manufacturers can produce stuff, that then works with other stuff. If you are not adhering to a standard, and are outputing color levels "differently", your device effectively has just become junk.

Its this clear cut, because, when we connect a Bluray Player, to an AV receiver, to a TV, we don't wan't to add color error in ever intermediate step. The TV(/monitor) gets a little leeway because displays (reproduction) are difficult (thats why people test monitors), but all the intermediate devices do not get any.

If your device handles color output wrong, its just trash.

Thats why you saw wesk05 post a table with bit values before. If one of those bit levels is wrong: trash. Otherwise you just accumulate errors, the longer your (input) signal chain gets, and if you do that - none of the production/reproduction chain (camera > edit bay > colorist > codec > media format > player > TV) would work. If you remember anything out of this, remember that standards are important.

So "will I see the error" is the self actualizing question to ask (*mumbleasamillenial*), but the actual answer is, that if the math at that point is wrong and the error is there, its already game over, and everyone agrees.

That said, added error margin is big enough, that you will see it, even directly connected to a TV in almost all cases. (Thats what you post dE numbers for)
--

No, the difference will not be like "what an sRGB signal looks like on a wide gamut monitor". "Wide color gamut" means, that the color triangle you see in the Calman Screenshots gets entirely "bigger" (colors become more saturated), the X9S remains within the rec709/sRGB color space, but colors are shifted (= tint/chroma issue). So in this case, all colors in the cyan spectrum get much more blue, in the yellow spectrum they get more red, and so on and so forth. You can literally see that in the triangles in the screenshots posted. You also see the rec709/sRGB targets there.

Also, the tables contain xy (chroma) and Y (luminance) errors as well - so you can even look at the problem more granularly.

You see what the issue with "just give me a simple answer" is? (Its people who are willing to give them... )