SD card library - unbuffered reads?

Created Thu, 23 Aug 2012 08:14:46 +0000 by rasmadrak

rasmadrak

Thu, 23 Aug 2012 08:14:46 +0000

I don't know if this is the right forum actually, since the SD card library has just been ported from the Arduino world, but -

How is the unbufferedReads() function intended to be used? No matter how I've done the SD card refuses to read unbuffered and simply stores everything internally before copying byte-by-byte into the destination buffer. Unbuffered reads/writes are basically a demand for handling "realtime" data to and from a SD card.

I took the liberty of modifying the library to read directly into the target buffer, which seem to speed things up quite nicely. I haven't check if this affects the library in any other way, thou, but in any case it's trivial to create additional functions for reading vs writing.

To Digilent/ChipKit staff - Are there any plans for a revised/updated SD card library? The SDfat-library the current version is based on is very old and outdated. There's has been several major releases of the library from the original programmer including support for multiple files open and support for multiple cards, along with major speed ups in the overall code.

The updated library is already available for Arduino, which is a pity since the Chipkit is superior otherwise. I would very much appreciate if you could take the time to implement an updated version of the library - and I figure a lot of Chipkit users feel the same.

rasmadrak

Sat, 25 Aug 2012 02:00:50 +0000

Hi there,

I ditched the SD library all together (no need for filenames/directories etc for my implementation anyway) and got a MASSIVE speed boost when reading raw sectors. I've also made my implementation use the DSPI library instead of the SPI, which should allow me to use "background loading" via interrupts with a little clever recoding.

Previous it took around 13 (and up to 20) ms to load a frame (1536 bytes) of the SD-card - now it's down to 5 ms. I figure this could be pushed down even further if the CRC-checks were to be disabled as well. At the moment I'm close to 307 KB/sec, which is plenty for my needs!

By separating the two SPI devices and putting them on two different DSPI-"units", I also managed to raise the performance quite a bit for the display rendering as well - Went from 8.5 to 3.9 ms!

Now I'll have to figure out a way to SLOW down the streaming video! :)

Awesome!

rasmadrak

Sat, 25 Aug 2012 11:44:48 +0000

Another nice side effect is that hotswapping cards works now! Obviously it must be an identical SD card (or special care must be taken to make sure the right sectors are being read).

In my case this allows me to preview animations in realtime by simply removing the SD card, load a new file and insert it again.

By the way, The library SDuFAT got me started, I suggest you check it out if you want more speed and is willing to sacrifice the previously mentioned parts

mhanuel

Thu, 30 Aug 2012 03:37:59 +0000

Hello rasmadrak,

Can you post your code that replace the SPI with DSPI?

Appreciated!

rasmadrak

Fri, 31 Aug 2012 21:51:42 +0000

If you base the code of the SDuFAT then simply replace all transfers with the DSPI ones instead. I added an additional setupSD() function as well, but all it really does is setting up the DPSI and then initializing the SD card. Should be simple enough to reproduce.

I should probably add that I've basterdized the orginal code to suit my needs, for instance - ignoring the return values of calls I'm not interested in. Also I've skipped the write part completely.

pito

Sat, 01 Sep 2012 16:04:20 +0000

Ask the fat16lib developer who did SDFat, he will certainly help with chipkit as well. BTW with chipkit @80MHz you have to reach ~800kBytes/sec writes and 1.7MBytes/sec reads with faster cards. Mind the sdcard's latency, therefore larger buffers are to be used (no big issue with pic32mx however).

rasmadrak

Sat, 01 Sep 2012 20:46:57 +0000

Yes, I guess I should be more specific. :)

I need to access small amounts of data (1526 bytes) in the shortest time possible. Using any kind of filesystem only delays the reading and doesn't really give me anything in return other than ease of use and "backwards compatibility" with a computer. But since the card image is 100% under my control, I'll have no problems reading the data back using a custom converter.

The speed I'm getting is currently around 7.5 ms to find and read 3 sectors (1536 bytes) from the card. I believe this could improve by using the READ_MULTIPLE_BLOCK command if I could figure out how it works... ;)

Or by using interrupted reads. But I'm a little stumped there as well.

EDIT:

If my calculations are correct I'm getting about 204 KB/sec throughput, which is ok - but not great. It should also be noted that I don't know the speed this particular card is capable of. But surely it's greater than 204 KB/sec.

This 1.7 MB you're talking about - is this a theoretical figure or something you've measured? I would very much like to get those numbers! :) Also - is it 4bit (NDA'd) SD or SPI 1bit?

brad000123

Thu, 13 Sep 2012 09:53:49 +0000

I believe this bypassed a lot of the delay in the SD card reading, but I have not yet confirmed this with any actual numbers - but at the very least it certainly runs faster now. By the looks of the new animation speed, I'm figuring it's close to 40-45 FPS streaming video now instead of the previous 30-35 FPS.

rasmadrak

Thu, 13 Sep 2012 10:34:11 +0000

That comment is looking strangely familiar... ;)

I'm up to 614 000 bytes/sec now, approaching the theoretical limit for 1bit transfers, if I consider 1 700 000 bytes/sec the max for this card with 4bit transfer (measured in a computer).

So I'm perfectly happy with the speed at the moment! :)

majenko

Sat, 15 Sep 2012 19:10:54 +0000

I fail to see what that has to do with unbuffered reads of an SD card on a ChipKit microcontroller board.

I suggest you go ask on a forum which is at least vaguely related to MACs, which this most certainly isn't.

Oh, and if you can't differentiate between a forum about electronics and microcontrollers, and one about computers, then I suggest you just quit now before you make a bigger fool of yourself.

Jacob Christ

Sat, 15 Sep 2012 19:16:11 +0000

majenko,

I think your addressing a robot. Often posts are made like this to try to look like they fit in so that the robots can get the ability to post links at some later point.

Jacob

majenko

Sat, 15 Sep 2012 19:50:04 +0000

Bah!

My sentiments still hold though.

And the one I made to Gary about the bacon slicer...