Tutorial: Using HDD with sizes > 2 TB (Internal or USB ) and SDHC/SDXC cards

Part-1

OK finally figured out how that limit of > 2TB HDD support actually works and what can be done and what can not be done to overcome limitations.
Everything mentioned here was implemented by myself in practice. Practical experience includes using 2, 3, 4, 5, 6 and 8 TB drives.

After reading this tutorial you will see that HDD > 2TB is supported both as Internal HDD and as USB attached HDD. There are however conditions to this and formats under both conditions in fact are (mostly) conflictive with each other.

I am afraid this is a bit technical, but as many of us including support staff was often confused about things, I will try to make it clear(er). If you are only interested in the do's and don'ts to make it work you may jump straight to Part-2.

As a start one needs to understand the difference between "HDD Initiation" and "HDD Formatting".
Often this is done in one step by a program, but the technical functions are different.
- The "HDD Initiation" tells the interface driver where and how to Boot, how the HDD can physically be addressed and where the physical partitions can be found on the disc. Here Microsoft Windows MBR or GPT initiations will be used to be compatibel with various Media Players
- "HDD Formatting" puts on the filesystem. This defines how and where files are put on the disk using as specific filesystem. Here the filesystem is assumed always to be Microsoft Windows NTFS to remain compatible with most Media Players, FAT32 works functionally but has a 4 GByte individual file size upper limit. That won't do for bigger MKV files and above all ISO images of DVD and BD. BD in fact will hardly ever work.

Some technical background for the interested
- First there are the Hard Drive manufacturers which make HDD drives traditonally read by sectors of 512 bytes each. With modern discs in Terabytes that is a lot of sectors which is not very efficient any more. Filesystems like FAT32 and NTFS therefore always read/write bigger blocks of data called clusters. A cluster is by default 4 KBytes (=8 sectors) and is configurable.

- Sectors are referenced to using 32-bit registers. If one calculates right one learns that a maximum 2.2 TByte can be addressed with a 32- bit counter adressing 512 byte sectors. Changing that is far from simple as this resides deep in the OS and Drivers. Partioning does not help here as still the whole disc space needs to be accessed at the physical level. This was up till recent done via the Master Boot Record (MBR) and Logical Block Addressing (LBA) using 32-bit. Windows changed to GPT (GUID Partion Table) with Windows-7 using 64-bit addressing solving this limitation structurally.

- But even with GPT the read/write overhead remains the same. So Hardware manufactures did a trick and intruduced Physical and Logical sectors. The newer 2 Tera Byte and bigger discs all work with Physical Sectors of 4K Bytes presented to the software as 8 Logical sectors of 512 bytes. This technology is called "Advanced Format 4K" which reduces the disc internal overhead but not the total CPU processing chain for I/O. The drives should be labeled 512e which stands for 512 byte logical sector emulation. To make a 512e HDD efficient sectors must be aligned at 4K boundaries. See the matching AF logo for a 512e HDD.

- The next step is Native 4K support. See attached logo for these 4Kn HDD. With 4Kn drives Logical Sectors, Physical Sectors and Clusters are all identical 4K Bytes streamlining throughput that way. This requires drivers dealing with 4K Logical sectors and obviously bolts on support of GPT for Initiation. Windows supports Native 4K HDD starting with Windows 8.1 OS. This paves the road for hazzlefree support of > 2 TB and even > 16 TB HDD combined with improved performance. Starting begin 2016 the first Native 4K HDD's made by WD appeared on the market. These are sold as HDD or build-in into their USB3 Enclosures. Drives can easily be identified as both Logical Sectors and Physical Sectors are always 4K.

You can display all mentioned HDD format details using fsutil fsinfo ntfsinfo x: under Windows. This is a CMD to be executed with Admin rights! (Note x: replaces the actual drive being used)
You may also use "MiniTool Partition Wizard Free" (see bottom for more details).

Back to most media players around
Access can run via Internal HDD, USB HDD Docking Bay, USB HDD enclosure or a NAS.
- A NAS setup hides the total physical drive access plus format (RAID) used and as such supports > 2TB drives without the Media Player even being aware of it.

- Internal discs are SATA attached (like internal drives in your PC). Drivers for SATA are easier to adapt and most media players upgraded to GPT support. They can address > 2TB discs using Advanced Format 4K support. All external drives for mounting into a PC purchased in shops are formatted that way. There is always a buffering/memory limitation somewhere but there is not a hard limitation for the maximum disc size. We know that 6 TB and 8 TB drives do work, but I do expect drives far above 16 TB to work too.

- USB attached drives is a complete different ballgame. Drivers are in all places and often very old. Also 64-bit addressing support may be jeopardized by a structural HW limitation. So GPT support is often not yet available and worse not being planned for many products. Under the condition that GPT is handled by products on both ends then USB standards do not impose limitations. In practice only some recent USB3 Docking Bay implementations do include such support.
As a temporary fix hardware manufactures came up with a trick. They started to ship USB-enclosures (= Casing, USB/ATA bridge and a HDD as one unit) with native 4K Logical sector enabled. The USB/ATA bridge firmware presents these to the USB-side. Result MBR can now work with 32-bit adressing of 4K Logical sectors. The physical disk access is not a problem as this is done by the USB/ATA bridge here. This trick will obviously only work up to a limit of just above 16 TB capacity.

There is a size limit on each entry here. See Part-2 for the remainder.

 

Part-2

Moving a HDD between Internal <=> External <=> NAS <=> Windows PC
Please use Microsoft Windows 7/8/10 only for re-formatting with GPT+NTFS and have it (E)SATA attached for Internal HDD's. I am sure that will work. MAC and Linux equivalents may do the trick, but never tried those myself. Nobody ever reported (good) results for media players
Internal HDD and USB HDD > 2TB are not inter-changeable and need to be re-initiated and re-formatted for each deployment. As most people don't open USB Drives/Housings/Enclosures they will never be aware of the difference. Personally I am now able to move any HDD in either direction without problems. This with the exception of (still exotic) 4Kn type HDD's having 4K Physical sectors not handled by Android players.

- A HDD pulled from a USB Drive/Housing/Enclosure will work only as an internal SATA attached drive again after standard re-initialisation + re-formatting. Obviously this HDD must be (E)SATA attached during mandatory re-initialisation + re-formatting using Windows-7 or equivalent standard tools. A common reason for doing so is that a complete USB-Drive may factually be cheaper than buying the same HDD as a separate unit. You still may be able to re-use the then empty USB-External enclosure later.

- A HDD pulled from a Plug & Play USB3 Docking Bay with GPT support (like Fantec's, Icy Box and probably Lindy's) will work straight as an Internal HDD again. In fact you may move individual units freely between such a Docking Bay and Internal bays of multiple brands. Tried this with sizes up to 8 TB. This won't work with RAID supporting variants obviously as these will need a re-build of many hours when a HDD is changed (even in JBOD mode).

- Formatting with 4K Logical sectors for USB External drives requires special tooling, it also helps if you exactly know what you are doing using such tools. I am using tooling from Acronis for this just as most USB HDD manufacturers do. You will run into this whenever you want to insert a new/re-used standard SATA internal HDD into an existing External USB Housing e.g. to increase its capacity.

- If you pull a HDD from a NAS then it won't work for either deployment as it has been re-formatted the wrong way. It may be using EXT3/EXT4 but in most cases will be using a proprietary format. Re-format it first for the purpose you have in mind being Internal or External USB enclosure as explained.

- A HDD pulled from a working Windows PC might still not work despite being formatted correctly with GPT + NTFS. The reason is Windows and/or the manufacturer has put an additional (hidden) partition on it for OS recovery purposes. Media players like to see a single Partition starting at the beginning of the physical disk. You need to remove all existing partitions first and next format again with GPT + NTFS when (E)SATA attached. Doing so via USB won't work in most cases neither as resulting HDD parameters are slightly different also not interpreted by Android.

Using (EXT2/EXT3) EXT4 formatted HDD in combination with MAC or Unix/Linux/Android based platforms.
First of all this is not officially supported by some media players but it is validated to be working just fine. This does not come as a real surprise as Android is a Linux derivation and Android kernels include support for EXT2/3/4.
EXT2 is often used on USB-sticks and SD-cards, but is less suitable (=less robust) for deployment on a HDD use EXT4 by preference.

Obviously EXT formatting has to deal with the same HDD technical developments as NTFS:
- EXT3 will support a HDD up to 16 TB using a Logical 4K block size The choice of the block size to be used may be automatic or a variable to be set manually; this depending on the OS, OS-version and the factual tool being used.
- EXT4 has no implicit HDD upper limit regarding total volume. Per OS and OS-version a support limit may be applicable and specified by the manufacturer. In practice EXT4 will work with any size HDD available within the foreseeable future.

SDHC/SDXC Cards and using exFAT for HDD?
exFAT is the new defacto File System standard for SDXC cards. In fact exFAT was designed for memory cards and sticks and not for HDD. It is an excellent replacement for FAT32 removing all its practical limitations but not an advisable (=less robust) replacement for NTFS.
As SD-Cards are immensely popular this will move exFAT to almost any platform. You see already a lot of devices (above all phones) with SDXC/exFAT support included. Media Players are very slow to adopt SDXC card support. Using exFAT on SDHC cards should work in normal cases as it has become a standard Android Lollipop feature by now.

Occasionally I read that only exFAT support makes the difference between SDHC and SDXC, this is a mistake as support for the new UHS-I bus is the real technical innovation coming with SDXC. This new SDXC bus has a far higher speed . The UHS-I backward compatible UHS-II bus will again double throughput and is almost as fast as SATA600 and has 2 rows of pins making identification easy.
Some brands call everything up to 32 GByte SDHC (with FAT32) and above 32 GByte SDXC (with exFAT) which is misleading. I have seen 128 GByte SDHC cards not supporting the UHS-I bus. Look at the corresponding UHS-I/UHS-II speed labels which must be present for SDXC compatibility and performance indication. Media players will not work with native SDXC cards! Never seen any UHS-I bus enabled media player yet which is a pity.

SDHC/SDXC Hybrid Cards which also exist should always carry both a SDHC (e.g. @10) and SDXC (e.g. U1) speed indicator as they work when inserted in old SDHC slots and new SDXC slots. In reality they may not always carry both labels, specifically SANdisk is known to make a real mess with labeling. See the attachment for a correctly labeled Kingston card. The term hybrid card which I use here is never used for any SD-card advertising which adds to the confusion.
When you e.g. put a 32 GByte Hybrid card (mostly labeled as SDHC) into a card-reader for SDXC with UHS support, then it will be blazing fast. The reverse is also true, putting a Hybrid card of 64+ GByte into an SDHC card-reader slot will slow it down to @10 speeds maximum even being an ultra fast UHS-II card. This slow down will always happen when using such a hybrid card with media players as it is run in compatibility mode then.
When using a SDXC capable external USB Card Reader then it must be USB3 connected as transfer speeds are way above USB2 limits. Measured 85 MB/sec Read and 30 MB/sec Write on a 12 Euro 32 GB Hybrid card. This card works also with most of my media players. One player (Xtreamer eXpress) refuses to read hybrid cards for an unknown reason.

If you want to validate/view very detailed HDD settings yourself and/or to try various formats suitable for Media Players then you can use the excellent Windows based "MiniTool Partition Wizard Free 10.2".
https://www.partitionwizard.com/free-partition-manager.html
It allows HDD formatting with EXT2/EXT3/EXT4 using Windows too

 

Hi, @Nice Monkey and hi to all

Based on what you say, I can't put an internal hdd 8tb into a usb-case to formatting it via Usb (GPT-NTFS) on Windows, and then extract it from case for insert it like HDD internal on Zidoo X10? Am I right? Is it necessary to format Hdd from Sata?

 

It depends what the enclosure was designed to do, but in most cases that won't work indeed.

One can extract a HDD from an enclosure though, next insert it in a windows machine, delete all partitions and format it GPT+NTFS and finally insert it into a media player.

 

Thank you! This is great problem for me, cause I haven't E-Sata port...I would have to unmount pc case and connect Hdd direct to Sata on motherboard

 

There are USB/SATA adapters that are transparant and seem to work. This was reported to me. Can't recommend a specific model/make. A Docking Station may work too provided it supports GPT.

In my own PC I am using a Chieftec 3-slot HDD docking with quick loading handles. Like USB supports SATA eject and load. It occupies dual 5.25 inch bays.

 

But substansially, an usb-external case, it is not an usb sata adapter?

 

They used to be functionally the same till the HDD > 2 TB boundary was crossed. Enclosures were then made intelligent to deal with any old OS not supporting these HDD directly.

 

Ok. Thanks. When Ironwolf will arrive i'll tell you if I succeeded also for other users interest

 

Alsways good to share experiences. The Ironwolf should work as I am using one myself 8 TB.

 

Why is this thread making me so grumpy? I keep reading it over again and I don't know why

 

Why isn't this a sticky???

 

Thank you!!

 

Finding this so frustrating why does it have to be so difficult? The hard drive you put into a Dune solo is literally plug & play, with this I have spent £55 on a seagate 2tb internal hdd only for me to find out I had no way to format it before putting in my Zidoo, so I spent a further £23 just for a enclosure to format it. My pc sees it fine, I selected gtp, then formatted it (quick) to NFTS but the Zidoo still doesn’t see it, I’m now in the process of formatting it fully with quick unchecked, but I’m not holding out much hope, anyone have any advice please? Manufacturers should realise not everyone is a computer wizard and make things simpler. I feel like I’m wasting my time & money.

 

It would also help is stuff like that was in a actual user manual, but the 'cheap' Chinese are just 'cheap' by not supply a decent manual.
I realize everything can't be covered, but at least up[ to mid level functions should be there starting with what all of these damn cryptic, stupid & silly icons stand for!!

 

Hi @Nice Monkey

Thanks for the usefull info.
Personally i'm kinda lost here with all the info.

Basically i want to know if a newly 10TB harddrive in the Z10 will work, will it be recognized, ect... ?

Drives like :

• Seagate IronWolf Pro ‘ST10000NE0004’
  
• HGST Ultrastar He10 ‘HUH721010ALE600’
• WD Red Pro ‘WD101KFBX’

 

Frankly, that's a hell of a lot of data packed on a single HDD. When that goes south (it will eventually, either the controller, the heads and/or the platters) you loose the whole thing (depending on the failure). Why not split it into 2 drives?

 

I ran into this hard drive issue recently on my Zidoo X9S. I have multiple hard drives on NAS and couple connected directly to Zidoo. One is 8TB Seagate hub and second old 3TB with no name enclosure connected to Seagate hub USB 3.0 port. 3TB drive was GPT and NTFS on Windows 10 PC. The drive got corrupted during coping files to it from PC over Zidoo Gigabit network. Reformatted the drive GPT and exFAT this time and back to Seagete 8TB hub port. It worked OK for couple of months until it got corrupted again during testing and set up of my new receiver.

Zidoo has issues or wires crossed between all ports and HDMI. Plugging nad disconnecting cables on one port effect behaviour of the device on the next port. I run into the issues with HDMI 2.0 cable that worked perfect between Zidoo and TV but had serious issues between TV and Sony receiver. By the time I had it all sorted out my 3TB hard drive disappeared from Zidoo media browser and got corrupted again.

I recovered some files out of it and did fresh GPT and NTFS thinking I'm going to connect it directly to next port on Zidoo USB 2.0. Zidoo would not see this drive anymore on any port including direct SATA connection. Windows computers had no issue with the drive and files on it.

What pointed me in the right direction was the comment from Nice Monkey regarding the partitions on the disk at the beginning of the drive. Some programs would show a small partition 16MB in front of main data partition. Using GParted from Raspberry Pi Ubuntu installation I removed this small partition. The drive works now on all Zidoo ports USB 3.0, 2.0 and SATA. I didn't have to reformat it again and I didn't loose any data.

The small partition was not visible in Windows "Computer Management > Disk Management". It was transparent to Windows but no go with Zidoo. I'm not quite sure how I got it and what made it. Maybe it was some leftover corrupted data from previous exFAT, new feature from latest Windows or something generated by USB enclosure during disk initialization but the drive wouldn’t show up when connected to Zidoo. After removal of this partition the drive works everywhere.

The size of sectors or what device did the format made no difference in my case. Quite possible that those small system partition are on other drives with high end controllers or some security features. Maybe that's what stops Zidoo from seeing those drives.

 

Good morning.
I need help.
I own a Zidoo X9S.
A few days ago I bought a Seagate Barracuda hard drive, model: Exos. With a capacity of 16 TB. The operations I performed are these:

1. unboxing;
2. connection to the pc with box via usb;
3. formatting in GPT, with Windows 10.

Unfortunately when I connect it to the zidoo, it is not recognized.
I tried both with box via usb and with sata cable.
I state that the disc is new and in a few days I have almost filled it. So I hope I don't have to reformat it.
Finally, the other disks I connect, with capacities of 4TB and 5TB, work perfectly.

 

Is it better for new internal hard disk to be formated inside Zidoo , in my case UHD3000, or in Windows PC ?