SD Card read/write speeds

Created Sun, 19 Jan 2014 17:38:11 +0000 by majenko

majenko

Sun, 19 Jan 2014 17:38:11 +0000

I have been doing some work on the SD and DSPI libraries in UECIDE with the aim of making them work somewhat more efficiently.

The default SD library with MPIDE uses a purely bit-banging SPI interface, even when it's connected to a hardware SPI port. As a result it is really really really slow. I mean dog slow. Dead dog slow.

A crude little benchmark program I knocked up to write then read 10MB of data to an SD card told me:

Initializing SD card...initialization done.
Deleting existing file
Writing 10MB to test.txt...done.
Time taken: 114712ms
Reading data from test.txt:
done.
Time taken: 111641ms

That's run in MPIDE. It equates to around 89KB/s write speed, or 92KB/s read speed.

Now that really is slow.

So, step one was to switch to using hardware SPI when available. That meant a slight change to how the SD library works and how you configure it. You can now pass it a DSPI object to use a specific SPI bus (defaults to DSPI0), or you can give it a bunch of IO pins to use software SPI, which has also been replaced with shiftIn() and shiftOut() instead of custom code (after all, why duplicate code?).

Already, at 10MHz SPI clock speed it's considerably quicker:

Initializing SD card...initialization done.
Deleting existing file
Writing 10MB to test.txt...done.
Time taken: 56602ms
Reading data from test.txt:
done.
Time taken: 51964ms

Now we're flying along at 197KB/s read speed and 180.9KB/s write speed. That's quite an increase in speed. But just how fast can we push it? Up the SPI clock to 20MHz, and the speed increases a little to 220KB/s read and 200KB/s write.

But there's one more little trick we can use that is available on some boards, and it's what we use on RetroBSD, and that is Enhanced Buffering. Enable that and you have a small FIFO instead of a simple shift register for the SPI data. It just smooths things out a bit - you can be loading (and emptying) the FIFO while it's doing the transferring of data. It just makes the whole thing more efficient.

Initializing SD card...initialization done.
Deleting existing file
Writing 10MB to test.txt...done.
Time taken: 45681ms
Reading data from test.txt:
done.
Time taken: 39569ms

And yes, the results are even faster. Here's a little league table so you can see all the differences:

Setup           Read        Write
MPIDE        91.7KB/s    89.3KB/s
HW 10MHz      197KB/s   180.9KB/s
HW 20MHz    220.8KB/s     200KB/s
ENHBUF      258.8KB/s   224.2KB/s

And yes, those changes are now available in the latest chipKIT core in UECIDE.

EmbeddedMan

Sun, 19 Jan 2014 22:19:01 +0000

That's pretty sweet. Any chance you can issue a pull request to MPIDE so these changes can be used there too?

*Brian

majenko

Sun, 19 Jan 2014 22:21:43 +0000

I could, but the changes to the SD code do have a bit of a knock-on effect as far as Arduino compatability goes. The Sd2Card constructor, if you want to use anything other than DSPI0 (which for some you will need to) is very different, so it doesn't (yet) work out of the box for some boards, like the WF32.

I'd like to expand the board definitions so they include the default SD settings as well really. The original SD code had the definitions within the SD code itself, which was a bit silly really. Now it completely ignores those definitions and just uses DSPI0...

EmbeddedMan

Sun, 19 Jan 2014 23:17:41 +0000

I agree- putting this in the board definitions so that the SD library can always 'do the right thing' makes a ton of sense.

*Brian

majenko

Sun, 19 Jan 2014 23:38:01 +0000

Also the SD library requires the DSPI library to be included in the sketch. This is different to "normal". Not a problem for UECIDE, which deals with recursive library requirements, but could confuse new users in MPIDE.

EmbeddedMan

Mon, 20 Jan 2014 00:04:33 +0000

Ahh, crap. The old "can't include a library within a library" problem. I can't wait until somebody figures out how to fix that in MPIDE.

Or maybe I should just join the dark side . . .

*Brian

majenko

Mon, 20 Jan 2014 01:00:39 +0000

Do not underestimate the power of the dark side...

EmbeddedMan

Mon, 20 Jan 2014 02:11:29 +0000

I just downloaded it, and will be trying it out shortly . . .

*Brian

pito

Mon, 20 Jan 2014 14:19:58 +0000

Great to see majenko finally took it in his hands and tried the EB. First time mentioned Sep2011 in this forum (Time to Market is an important aspect here as well). Also involving Bill (fat16lib) with his newest SdFat library would be a good move, I would say (if he would be interested to help). His SdFat gives 10-15x higher speeds with Due (with large buffers).. The SPI on pic32 is limited to something around 20Mhz, so the speeds around 500-1000kB/sec are to be expected (with cheap cards).

HunterR

Sun, 26 Apr 2015 09:54:57 +0000

I made a post in the Libraries subforum about my progress porting SDFat lib to allow use of hardware DSPI. (I am apparently on "post probation", so it isn't up yet, and I can't link it.) Can someone look at my modifications and make them more "official"?

http://chipkit.net/forum/viewtopic.php?f=7&t=3262

Results:

File size 5 MB
Buffer size 4096 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
957.67,12269,4088,4272
964.70,11783,4151,4242
960.25,11811,4085,4261
962.09,11789,4085,4253
962.47,11806,4087,4251
960.25,11788,4085,4260
964.51,11784,4151,4242
959.88,11804,4085,4262
960.06,11786,4085,4262
964.51,11787,4088,4242

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1176.07,4079,3388,3482
1175.79,4087,3375,3482
1175.79,4132,3372,3482
1175.79,4126,3372,3482
1175.79,4085,3372,3482