Data Transfer

Ric Murray Jan 10, 2017

  1. I have 9T to transfer for an indie feature project. The data is on 3 USB3 drives and I am trying to transfer to a 4 Disk RAID5 box and the finder is telling me 3 Days to transfer! I am only getting 30-40 MBS transfer speed. I tried copying a single folder to a USB3 SSD and got 100+ MBS, not great, but at least I know the originals can be read at over 100MBS. I then tried transferring from the SSD (the SSD's are connected with hi speed USB controllers that I have tested to almost 500MBS) to my RAID, and got 200MBS. Pretty respectable. I then reconnect the originals to the RAID and still only get 40MBS. This is all on the same USB3 Bus. Tried it on both a Mac Pro (yes I know the USB bus sucks) and a Macbook Pro, same results. Can anyone explain why if I can read the disks at 100 plus, and my RAID is capable of writing 200+ I can only get 40MBS?
     
  2. Can you run a speed test with BlackMagic or AJA disk test and see the results?

    Are you plugging the USB one by one? Some ports are on the same bus and that can create a bottleneck sucking up all the bandwith

    Also use a checksum software like Silverstack or yoyotta to avoid data corruption
     
  3. 40 MB/s sounds like a USB2 controller bandwith limitation.
    Are you sure the disks are connected via USB3?

    I have an external SSD with more than 400MB/s read speeds.
    Regular cheap 5400 RPM USB drives can be much slower. I had external drives with speeds under 80MB/s but 40MB/s is pretty slow indeed.

    I have to say I don't like USB at all, because it is CPU dependent.
    Thunderbolt is always so much faster.
     
    Samuel Ouimet L. likes this.
  4. I will run some BM tests when the current transfer is over, but the source disks can read at 100+ and the RAID can write 200+.

    The RAID is plugged directly into a USB3 Port on the Mac Pro, the source disk into a USB hub. I used that path with the other disks and easily got 200+.

    I have Shotput Pro, but won't that just slow it down more?
     
  5. 40 MB/s sounds like a USB2 controller bandwith limitation.
    Are you sure the disks are connected via USB3?

    Yes same path with different disks yielded 200+

    I have an external SSD with more than 400MB/s read speeds.
    Regular cheap 5400 RPM USB drives can be much slower. I had external drives with speeds under 80MB/s but 40MB/s is pretty slow indeed.

    80MBS would be great! Would cut transfer time by half!

    I have to say I don't like USB at all, because it is CPU dependent.
    Thunderbolt is always so much faster.

    Agreed, but this RAID was bought before T-Bolt and has SATA, USB3, & FW800. It's the only thing I have that can hold 9T on one Volume.
     
  6. Doesn't what you are copying also affect the speed. Copying a feature film DPX sequence will be alot slower than copying a single QT of the same film because the drive is constantly start/stopping to copy each individual file.

    Could that be an issue here?
     
    jamie dickinson likes this.
  7. Thanks all, but I just ran a BM Disk test and yes indeedy the pocket USB drives only read about 48-50 MBS. Must be 5200 RPM cheapos. Looks like three days of watching the finder creep along. In 2017, who'd a thunk it.
     
  8. A fast interface doesn't help if the drives are shitty.
     
    Margus Voll likes this.
  9. You write that your RAID id FW800, so why not use FW800 instead?

    I have some old USB, FW800 + SATA drives and they all have USB2 only!

    And be sure to check your HUB!!
    Most HUBS are just bad. Try it without.
     
  10. I just checked the speed of two of my externals:

    2TB Samsung EVO960 SSD via USB3:

    DiskSpeedTest.jpg


    500GB El Cheapo HDD drive via USB3:

    DiskSpeedTest-USB3-HDD.jpg
     
  11. Can you skip the Hub? Plug the source drive directly into the computer, just like you have the RAID plugged in?

    Are you copying from multiple source drives simultaneously? The way Finder calculates remaining time is a bit . . . not-accurate if you are copying multiple things at a time. First and foremost, copying from multiple sources simultaneously tends to eat up more overhead, so you're losing time compared to if you are sitting there right when one ends in order to queue up the next source. The Mac Pro only has one USB bus, remember, so you're total 5Gb/s speed is divided between all USB devices.

    Though even at that, in many cases, the bus speed is not the limitation these days, but rather the hard drive itself. The BEST spinning disk on the market gives around 220MB/s transfer speed, and the average is around 120MB/s. Most external drives are around that average, with cheaper ones dropping to around 70MB/s. USB3.0 maxes at 5Gb/s, which is around 640MB/s (b=bit, B=byte, so divide 'b' by 8 to get useful numbers). If the drive is not the limiting factor, reading from a source on USB and writing to a destination on the same USB bus, the theoretical max is around 320MB/s, but you'll usually get lower than that because of overhead and whatnot. Still, if you have 4 sources and 1 destination, all on the same bus, that could easily drop you to 30-40MB/s on each. But, if you limit it to one source at a time, it will probably take you far less overall time. Your best bet *might* be 2 sources at a time, but 1 is probably ideal.
     
    Andrea Bonomelli likes this.
  12. Thanks all. It's been 3 days and the data is on the RAID. As I said, 40MBS in 2017?, who'd a thunk it.
     
  13. OK, after all the time spent transferring data, the files aren't linking up in Premiere. It turns out that some of the old OWC RAIDs are not recognized by the Premiere Media Browser. Long story short, a wasted week, so I just ponied up for a 12T ThunderBay4. It tests out at 700+ MBS in and out configged as RAID 0. Don't worry I'm using the old OWC USB3 RAID5 as a nightly backup. Now my question is this...
    When doing a large transfer like this (almost 8 TB) is there any speed difference in having the source and target drives in a T-Bolt daisy chain or plugged into different outlets on a Mac Pro 6,1?
     

Share This Page