GFILE + chibios not opening correct files

[cpu20]

I have used gfile and fatfs together with the STM32 HAL libraries and everything worked fine.
But now I am trying to use gfile and fatfs together with chibios and strange things happen.

First the filesystem mounts correctly and gfile can open a file from the filesystem and read from it. But when closing the file and opening another file, things start to get weird. The gfileOpen function just returns the same GFILE it also gave back when opening the first file instead of the GFILE for the new file.

File1 = gfileOpen("/File1", "r"); // Opens file and reads from it correctly
gfileRead(File1, &Buf, 4);
gfileClose(File1);
File2 = gfileOpen("/File2", "r"); // The returned GFILE is the same as the
gfileRead(File2, &Buf, 4);		  // File1
gfileClose(File2);

So in this example the first file will be opened, read from and closed again. Now when the second file is opened, the GFILE returned by gfileOpen is exactly the same as the one that was returned for File1. So the same file is read again.

Now here is where it starts to get really confusing:

File1 = gfileOpen("File1", "r"); // File1 and File2 are opened
File2 = gfileOpen("File2", "r"); // and read from correctly
gfileRead(File1, &Buf, 4);
gfileRead(File2, &Buf, 4);
gfileClose(File1); // Opening files after this will return
gfileClose(File2); // the File1 GFILE...

When doing this it opens the two separate files, reads from them and closes them.
And now when trying to open another file after these two have been closed, the same GFILE will be returned as the one that was returned for File1.

Any ideas on what I could be doing wrong? Or what could be going wrong? I am using the STM32F746-disco again for this.
As opening and reading the first file works I doubt that it has something to do with the SDMMC1 configuration.

[cpu20]

Something goes wrong when allocating memory for the file object I think. It allocates memory from the heap, but I already experienced problems when chibios creates threads using memory from heap. The threads start acting very weird.
I have no idea what could be wrong with this. I'll look into it further tomorrow.

[inmarket]

Is it reading incorrect data or are you saying that the GFILE pointer is exactly the same?

If you are saying the later then this is normal. GFILES are stored as a small array of structures (the number determined by GFILE_MAX_OPEN_FILES). The GFILE is then a pointer to one of those structures. So if you open a file and then close it, the structure is released ready for the next open. Open/Close/Open is then expected to return the same GFILE on both opens. Both the contents of the internal GFILE structures will however likely to be different in the two open cases - the exception being when the underlying file system handle is also an indirect association.

Note there is no memory allocation at the GFILE layer although there may be in the underlying file-system layer.

[cpu20]

When using fatfs directly with chibios (not using gfile) everything works fine. So something is going wrong when using gfile.

Have there been other issues with gfile and chibios in the past? Or is this again platform specific (STM32F746-DISCO)?

[inmarket]

All known problems with the STM32F746-Discovery board have been resolved.

ChibiOS has no other known problems. It was in fact the os that we started the whole uGFX project with and the project was originally called ChibiOS/GFX.

If you are having strange problems like that one of the first places to look would be your stack sizes. Nothing works reliably if you overrun a stack.

Also, can you please explain the problem you are having? Is it returning data from the wrong file?

[cpu20]

So I have been playing around with the µGFX and ChibiOS configurations in a clean new project to see what happens.
What I found is that when setting CH_CFG_MEMCORE_SIZE to something like 4096 so that ChibiOS does not manage all of the RAM gfile works as expected. But when setting this to 0 (manage all RAM) things get very very weird.

I don't think I can really describe what is happening as it seems partially random. It does open a file and read from it, but the file read from is almost never the one given to the gfileOpen function. You should really test it out for yourself. Just make three files on an SD-card, and read from them one by one, closing the previous one before opening the other.

Could it be that when ChibiOS manages all of the RAM, it also manages the SDRAM but doesn't know the LTDC is using that?

[inmarket]

It could be that it is to do with the external SDRAM however I think it is more likely it is a stack overflow issue. Stacks are allocated at the top of ram (usually) with heap just below that. My guess is that telling ChibiOS not to manage all the RAM leaves enough space for the "naughty" stack to be able to overflow without crashing everything else.

For ChibiOS on STM32 don't forget the interrupt stack. It could well be that one which is overflowing.

[cpu20]

How do I check if that is the case? I am not that experienced with debugging RTOS program's.

[inmarket]

Easiest way is to fill the entire memory with a pattern eg 0x5a5a5a5a before the c runtime startup.

It is then easy to use your debugger to examine memory to see where your stacks have got to.

[Joel Bodenmann]

22 minutes ago, inmarket said:

Easiest way is to fill the entire memory with a pattern eg 0x5a5a5a5a before the c runtime startup.

ChibiOS actually supports that. There should be a setting in chconf.h when I remember correctly. CH_DBG_ENABLE_STACK_CHECK and CH_DBG_FILL_THREADS are useful.

Check out this and this.

[cpu20]

@Joel Bodenmann thanks for those recources!!! They are really usefull!
The only thing I can't get working is the generation of the .map file using the makefile approach.

When I add the flags: -Map=filename.map to the linker flags in my 'single file inclusion mechanism' projects it generates the .map file as expected. But adding -Map=filename.map to the LDFLAGS in the makefile throws me an unrcognized option. Any idea how to solve that?

[inmarket]

If you are using our makefiles there is already a OPT_xxx option that turns on producing a mapfile. Unfortunately I am not at my computer currently so I can't look it up for you but where to find the OPT_xxx documentation is listed near the top of the makefile.

Update:

The option is OPT_GENERATE_MAP. Set it to yes and it will generate the map file for you. It is described in tools/gmake_scripts/readme.txt

[cpu20]

After debugging alot I think I found the problem. See here for more information.

[inmarket]

Well done in tracking down a curly problem. I will look in detail on the weekend and see if there is anything that needs migration into our source repository.

[Joel Bodenmann]

Yep, great work indeed!