chipKIT® Development Platform

Inspired by Arduino™

more SD card problems

Created Sat, 30 Jun 2012 06:01:58 +0000 by spencoid

spencoid

Sat, 30 Jun 2012 06:01:58 +0000

I have the SD care sort of working on an Uno but there are still problems that I can not figure out. Has a tested and stable version of the SD card library happened yet?

My current problem is that I am losing a byte when reading files over about 32 k. It looks like a single byte is missed about every 32 k or so. I am reading data stored in three byte clusters and it gets out of sync at about 32 k bytes and then gets synchronized again once two more bytes are lost. I read something about this problem but thought it was fixed in the version in the test MPIDE.

are there still read problems?

any suggestions on how to speed up reading? Is the SPI read speed in need of setting at the max possible sped and how does one do that if it is necessary?

Jacob Christ

Sat, 30 Jun 2012 17:40:28 +0000

Which build of MPIDE are you using?

If your not using it, you might try the latest test build posted here:

https://github.com/chipKIT32/chipKIT-builds/downloads

I don't know if this will fix your problem, but if it still occurs then it will let us know that the problem needs to be addressed.

Jacob

spencoid

Sat, 30 Jun 2012 19:02:05 +0000

i am using the latest test IDE 20120612test. i just ran a test on a variety of files and it is not actually a specific position in the file where the byte is being lost. it occurs in different places in different files but always at the same place in the same file.

my program is reading data points consisting of three sequential bytes. all it does is read these values and instruct a servo motor to move forward backwards or pause for the duration of the integer value represented by the second and third bytes.

there are interrupts running to detect button presses but all they do is set booleans to pause the main loop if pressed.

i have the cards formatted to fat32. i have a bunch of other cards arriving to test with to see if it is a card problem but i doubt it.

i added a routine to toss out the invalid (misread) data value and reset the pointers to allow the machine to proceed. it only represents a minor glitch in the operation of the machine and it is not "life support" so ... but this is terrible to have to fake the data. at least it allows me to continue with testing in the hopes that the SD card will work soon.

i am pretty sure that just a single byte is being missed because nudging the pointer fixes the problem to the end of the file. this must have something to do with block pointers inside of nested loops or something fun like that. way beyond my ability to try to understand the arduino library code.

i can do further testing to try to locate the specifics as to where the missed byte occurs if that would help?

if the SD card (critical to this application) does not work soon, i will need to use a different controller. the regular Arduino is too slow. what other system should i try that would involve the least learning and changes to home made shields which are somewhat complicated?

rasmadrak

Wed, 04 Jul 2012 06:34:16 +0000

I am pretty sure this is the same problem that I experienced before patching the SD card library. It skipped reading the 512'th byte in each block and always returned zero. I've read files larger than 32K so the fix is ok.

Are you getting garbled values or zeros? Try reading a file with only 1's in it and check which byte becomes corrupt, I wouldn't be surprised if it was every 512'th byte...

You could also post the read code (inside code-tags) here and we'll check if there's something strange going on programmatically?

spencoid

Wed, 04 Jul 2012 07:24:10 +0000

i thought the 512 th byte problem was supposedly fixed in the test IDE that is currently available?

i can write a sketch to read a file of all ones and report where they are read as 0s but hasn't his been done as a test on the new patched version?

not sure what you are asking for in terms of code. my sketch is fairly complicated and works from files made by perl scripts. i can make a simplified version and provide a file to go with it. here is what my program does. maybe you can suggest what i should provide in terms of a sketch and file?

my perl script creates binary (binmode) files representing data points. each point has three bytes. the first byte is 1 8 0r 32 (00000001, 00001000, 00100000) which specifies the type of value to follow. i am only using three types now but will expand to many more, maybe. the next two bytes represent a 16 bit unsigned integer. i noticed a problem because i was getting numbers other than 1 8 and 32 in the first byte position. my program counts the bytes read and reports when an invalid data type byte is read. they do not seem to occur at every 512th byte. different files have the problem at different locations.

i was able to hack the sketch to work by skipping over data until i stop finding invalid bytes in the first byte position. this is allowing me to continue developing the program but it is not OK to fake it this way.

here is the main loop with unnecessary motor driving code removed. also included a simplified version of setup to make it easier to try running the sketch. not easy to read in the forum with the indenting removed so i am also attaching a

void setup() { Serial.begin(115200); if (!SD.begin(4)) { Serial.println("initialization failed!"); SD_init = false; return; } else{Serial.println("initialization success!");}

Serial.println("initialization done.");

SDfile = SD.open("motor.txt"); SDfile.seek(0); for (int i=0; i < 40; i++){ // read off the 40 character text header byte read_byte = SDfile.read(); Serial.print(read_byte); lcd.print(read_byte); } }

void loop() {
byte read_byte; int byte_one_int; int byte_two_int; int byte_three_int; three_count = 0; unsigned int uspause; unsigned long bytes_left = 0; boolean got_em = false; unsigned int offset = 0; if (paused == false){ while (bytes_left = SDfile.available()) { // step through sd card file read_byte = SDfile.read(); three_count ++; if (three_count == 3) {three_count = 0;}; // reset the count by three counter switch (three_count + offset) { case 0: // reading third byte so have all three bytes, upause is the time to delay between pulses

        uspause = (byte_two_int * 256) + byte_three_int;   
        got_em = true;       
        break;
      case 1:
        {byte_one_int = int (read_byte);}
        got_em = false; // redundant
        break;          
      case 2:
        {byte_two_int = int (read_byte);}
        got_em = false;  // redundant
        break;
     }       
     if (got_em){
        switch (byte_one_int) {
          case 1:    // delay without moving motor  
            delayMicroseconds(uspause);   
            // removed code that sends quadrature signals to motor driver system    
            break;          
          case 8:{  // run motor forward one step folowing delay  
            delayMicroseconds(uspause); 
            // removed code that sends quadrature signals to motor driver system 
            break;}
          case 32:{  // run motor backeards one step following delay
            delayMicroseconds(uspause);
            // removed code that sends quadrature signals to motor driver system
            break;}
          default:  // if not 1 8 or 32 as the first byte, this is an error so skip until valid bytes are found by offsetting the 
            Serial.print("invalid data type ");  // incrementing through the read bytes
            Serial.print(bytes_left);
            Serial.print(" offset ");
            Serial.println(offset);              
            offset ++;
            if (offset == 3) {offset = 0;};
        }          
     } 
 }  
 SDfile.seek(40);  // reset to after header currently have fixed length header
 Serial.println("restarting loop ");    
 }

}

attached is a file that does not produce errors when read by a perl script but does when run through the above sketch.

rasmadrak

Thu, 05 Jul 2012 12:53:17 +0000

Are the files binary or text? The files are .txt but what about the contents? If it's not binary you may need to account for linebreaks, spaces and other special chars.

spencoid

Thu, 05 Jul 2012 15:35:02 +0000

hte data files are binary. i open in txt mode to write the header of 40 characters and then close and re-open in binmode to write the rest of the contents. at one point i accidentally had forgotten to write the files in binary and whenever a byte of decimal value 10 was written it messed up because 10 is a line feed. not having that problem now so the files should be binary. i can try a version of the program that doesn't add the header or does so in bin mode and see if that makes any difference but i don't the problems do not occur in the 40 character header. i also think i had this probem before i added the 40 character headers.

spencoid

Fri, 06 Jul 2012 04:41:45 +0000

i made data files without headers which were written exclusively in binary mode. i modified the sketch to not read the header (which is not there)and still have the skipped byte problem.

i doubt i can make sense of the library code. short of that, any other test youcan think of to narrow down the problem?

my hardware and software are so close to working that i really want to figure this out. i have a lot more work to do with programs to create the instruction files and to edit them and want to make sure i don't have a fatal flaw in the system that i can not fix.

spencoid

Mon, 09 Jul 2012 15:17:40 +0000

i still need to have this fixed. i checked my code over and over and made simpler examples of the files to be read to be absolutely sure they are in binary. reading the SD card files on the computer does not produce errors, only when read in the uno from the SD card.

can someone who understands the library code please check to see if there is any possible error? do others have problems with skipped bytes when using this library?

is it possible that there is some sort of speed problem reading the card? i have tried seversl different brands and sizes of card and consistently have the same occasional missed byte. i would think that problems of card speed inspi mode or other problems would produce more gross errors, not just a single byte skipped now and then.

i really need this fixed and need to switch to a different brand of board if a solution is not found soon. any suggestions as to a high speed microcontroller that is arduino compatible and that has an SD card library that works?

spencoid

Thu, 12 Jul 2012 03:48:04 +0000

i think i found the solution to my problem. i had read that the chip select pin for the SD card needed to be set as output. i had mistakenly remembered that digital pins defaulted to output so i had not declared the select pin (4 in my case) as output. the fact that the SD card seemed to work in general did not make me think that this ws wrong. since i have set pin 4 to output and set it to high the SD card read does not seem to miss the occasional byte. i don't think it matters if i set the pin high or low, just that it needs to be set as output.

does this make sense as a likely explanation as to my problems?

Jacob Christ

Thu, 12 Jul 2012 04:38:21 +0000

i think i found the solution to my problem. i had read that the chip select pin for the SD card needed to be set as output. i had mistakenly remembered that digital pins defaulted to output so i had not declared the select pin (4 in my case) as output. the fact that the SD card seemed to work in general did not make me think that this ws wrong. since i have set pin 4 to output and set it to high the SD card read does not seem to miss the occasional byte. i don't think it matters if i set the pin high or low, just that it needs to be set as output. does this make sense as a likely explanation as to my problems?

Whoa, that's weird... I would say that the SD card should have worked perfectly or not at all if the chip select direction was not set. That is perfectly if it was an output and not at all if it was an input. If the chip select pin is floating maybe it was it was inadvertently being selected by the clock or something like that.

Makes me wonder how far the SD card is form the uC? Is it on a shield or just hanging off of some wires.

Jacob

spencoid

Thu, 12 Jul 2012 06:02:26 +0000

yes i never thought of looking at pin status because i thought it would not work at all if some selection was necessary but not made.

the SD card is on a fair amount of wire. i made a shield for my project but added the SD card as an afterthought so it is connected through about 1.5 inches of circuit board traces with no ground plane and also a couple more inches of wire to the sparkfun (not really a sparkfun but i forget the brand and it is deeply inside the prototype now) board.

my sketch reports all suspicious data to the serial port and i have had absolutely no reports since i changed the pin to output.