jandrese wrote:The caveat is that he was transferring zeros to the device. Since he was going faster than the rated speed, we can only assume that there was some compression happening somewhere. Either that or he has magic USB ports that push 11MBps of data over a 3MBps link.
I would be interested to see the same test repeated with a file generated from /dev/urandom instead of /dev/zero.
It's also possible his OS is buffering up the write and dd is returning early before the bits are actually on the drive. Might want to check if the stick is mounted async or sync. Async is dangerous on a USB drive because people are bad about just yanking them out whenever they feel like, regardless of the number of outstanding buffers still waiting to be written out to the drive.
Computers take significant time to generate from /dev/random, that would be a completely worthless test to copy straight from there. That's why I was just going from zero. Also, the OS isn't buffering it, I can view direct IO speeds in activity monitor. But here you go anyway.
apple:~ nfisher $ date ; dd if=/dev/random of=rndfile bs=1024000 count=256 ; date
Wed May 16 15:18:42 CDT 2012
256+0 records in
256+0 records out
262144000 bytes transferred in 33.567270 secs (7809512 bytes/sec)
Wed May 16 15:19:15 CDT 2012
(as expected, it could only generate 7mb/sec of random data to the internal hdd)
apple:~ nfisher $ date ; dd if=rndfile of=/Volumes/Service\ Data/rndfile2 ; date
Wed May 16 15:20:44 CDT 2012
512000+0 records in
512000+0 records out
262144000 bytes transferred in 28.440215 secs (9217371 bytes/sec)
Wed May 16 15:21:12 CDT 2012
9.2MB/sec. It is a little slower, probably due to the additional file i/o of reading rndfile, which was unnecessary when reading from zero, which is why I was doing it in the first place, to prevent internal hdd influence in the test. But it's pretty minor for reading. I'll be sure to use capital B this time to help those that can't count decimal places.
I took that file and copied it 4x to another larger file, rndfile1g, for testing, because I was too impatient to wait for 1gb of random data.
apple:~ nfisher $ date ; dd if=rndfile1g of=/Volumes/Service\ Data/rndfile ; date
Wed May 16 15:26:12 CDT 2012
2048000+0 records in
2048000+0 records out
1048576000 bytes transferred in 105.852448 secs (9906016 bytes/sec)
Wed May 16 15:27:58 CDT 2012
9.9MB/sec, a little faster, again as expected. But this was creating a file too. I'll repeat the test to overwrite the file, which may actually be a little faster. Theoretically the BEST way to do this is a direct block write, but I really don't feel like reformatting it.
apple:~ nfisher $ date ; dd if=rndfile1g of=/Volumes/Service\ Data/rndfile ; date
Wed May 16 15:28:07 CDT 2012
2048000+0 records in
2048000+0 records out
1048576000 bytes transferred in 94.007138 secs (11154217 bytes/sec)
Wed May 16 15:29:42 CDT 2012
11MB/sec, a slight improvement again as expected. Now repeating with reads.
apple:~ nfisher $ date ; dd if=/Volumes/Service\ Data/rndfile of=rfile3 ; date
Wed May 16 15:30:47 CDT 2012
2048000+0 records in
2048000+0 records out
1048576000 bytes transferred in 60.143104 secs (17434684 bytes/sec)
Wed May 16 15:31:48 CDT 2012
Read speeds slowed down to 17MB/sec here, this I was not expecting. I will repeat the test with an overwrite to see if it was file creation i/o that slowed it down.
apple:~ nfisher $ date ; dd if=/Volumes/Service\ Data/rndfile of=rfile3 ; date
Wed May 16 15:33:18 CDT 2012
2048000+0 records in
2048000+0 records out
1048576000 bytes transferred in 28.833711 secs (36366321 bytes/sec)
Wed May 16 15:33:47 CDT 2012
36MB/sec, an unexpected surprise jump in read performance. Once more to null:
apple:~ nfisher $ date ; dd if=/Volumes/Service\ Data/rndfile of=/dev/null ; date
Wed May 16 15:34:18 CDT 2012
2048000+0 records in
2048000+0 records out
1048576000 bytes transferred in 5.457242 secs (192143943 bytes/sec)
Wed May 16 15:34:24 CDT 2012
Interesting, back down to 19MB/sec. It reads faster to a file than to null. Another unexpected surprise.
I don't know where you're getting your "3MBps" quote from, but clearly that is not a limit for me here. Nothing I did got anywhere near that slow. What computer are you using? USB2 is typically rated at 480mbps (yes b) which should be up in the 50's for MB/sec but it's been my experience that pretty much all the good USB hardware performs at 39MB/sec read and write. (OWC mercury enclosures, they rock) The fair stuff does 36MB/sec, cheaper gear goes around 26MB/sec, and the crap creeps along at 18MB/sec. (those are the cheap "3 in 1 usb to bare HDD" adapters and many of the usb docks, they apparently use very cheap USB bridge chips, I've done extensive testing) From the research I've done this is due to the overhead involved with USB because the protocol wasn't designed to be used for large data transfer. Firewire400 (400mbps) goes at a consistent 39MB/sec also, fw800 does 79MB/sec for me here, again both read and write (to non flash media) So I have to assume this "3mbps" limit you either have on your computer or read somewhere about this centon, and it's not the case for mine.