Broken malloc/free! Got random crashes? Using String?

Created Fri, 14 Dec 2012 13:10:43 +0000 by dominicclifton

dominicclifton

Fri, 14 Dec 2012 13:10:43 +0000

Not sure if this issue has already been raised here but today I stumbled into a serious issue with memory corruption when using malloc and free.

Details here:

http://code.google.com/p/arduino/issues/detail?id=857

Importantly if use use String you might get random crashes because String uses malloc and free.

This needs a new software release ASAP to fix this fundamental issue. Is anyone looking into it?

BloodyCactus

Sat, 15 Dec 2012 02:32:52 +0000

that issue is part of avr-libc which is not provided by arduino (its part of avr-gcc). what makes you think the pic32 compiler uses the same memory.c file in its version of gcc?

did you verify with a test the arduino avr-libc bug is present in the pic32 toolchain?

dominicclifton

Sat, 15 Dec 2012 16:46:40 +0000

that issue is part of avr-libc which is not provided by arduino (its part of avr-gcc). what makes you think the pic32 compiler uses the same memory.c file in its version of gcc? did you verify with a test the arduino avr-libc bug is present in the pic32 toolchain?

basically, i have some code that works fine if a large buffer is allocated up front but when the buffer is found not to be large enough a new buffer is allocated, data is copied from old to new buffer and the old buffer is freed.

the code runs fine for the first two buffer increases but fails for the third attempt. if the old buffer is NOT freed then the program also continues correctly. it is specifically the call to free the old buffer that causes memory (heap) corruption.

the problem doesn't appear right away, only later on when another method attempts to use the data in the new buffer.

i verified this by dumping the contents of the buffer at the time the data is copied to the new buffer and comparing it to the data in the same buffer later on. the data was different.

in my code the data contains a list of call-back handlers, the crash is caused by non executable code being called.

ok?

BloodyCactus

Mon, 17 Dec 2012 14:15:36 +0000

are you checking the malloc return was successful? there could be a LOT of things going on here. Remember, this is not a page remapping system, so if you have 64kb free ram, you allocate 24k, then allocate 32 and free the 24, you have a gap below and above of 24 + 8, not a flat 32.. you could be running into that form of fragmentation. the libc malloc/free/realloc has little to nothing to do with fragmentation.

the pic32 does not have the same mips MMU that the old r4k etc had.

I've done some minimal linked list stuff using calloc/malloc/free and not seen anything, I'll dig some more and see if I get any corruption when calling free.

do you know if you have a minimum heap size specified? eg: --defsym_min_heap_size=512?

how big are your mallocs? do you know your free heap size before you start? fragmentation is a huge problem on embedded chips with small ram space and no page mapping mmu etc.

dominicclifton

Mon, 17 Dec 2012 17:45:45 +0000

yes i checked the malloc return, a valid address is returned in each case.

each item in the buffer is just an address, so the amount allocated is small: sizeof(void *) * itemsToStoreInBuffer.

as i mentioned the problem only occured on the 3rd allocation where 'itemsToStoreInBuffer' was 3.

for fun I also tried allocating more than i needed, that did not help.

i'm not 100% sure where the corruption occurs. The memory isn't corrupt after the copy from the old buffer to the new buffer but it is corrupt later on.

Essentially the program was doing this:

allocating a buffer, adding data to end of buffer allocating a larger buffer, copying buffer, freeing old buffer, adding to end of buffer allocating a larger buffer, copying buffer, freeing old buffer, adding to end of buffer then reading from the serial port to a fixed size static buffer. then iterating over each item in the buffer and calling a handler's accepts()'s method which would do something like this: String(inputBuffer).equals("is-this-for-me"). if the accepts() method returned true then control would be passed back to the code doing the iteration at which point the handler's execute() method is called.

the data in the buffer is corrupted between the accepts() call and the code doing the iteration's call to execute() which then fails because the address it's trying to dereference is now incorrect.

I believe when String is used like that it may be doing a malloc and free call.

I do know for certain that if i do NOT call free() then i don't have a problem.

pretty sure i'm not running into memory fragmentation issues with the buffer sizes i'm using, i'd be suprised if i was.

i can refer back to the problematic revision of code in my codebase (via git) if you need me to try anything specific.

avenue33

Mon, 17 Dec 2012 18:20:24 +0000

Also of interest for working malloc/free, the vector type.