Accented characters not showing properly on Console widget

[Joshua]

Maybe there's a note about this somewhere, but when gwinPrintf-ing accented characters from my custom font to a console, they don't render correctly. gwinSetText works fine for labels.

So for example,

gwinSetText(ghLabel, "é", gFalse); // works
gwinPrintf(ghConsole, "é"); // doesn't work, prints "é"

Is there a way to make this work using the console widget?

Thanks

[Joel Bodenmann]

@inmarket any insights from your side?

[inmarket]

Hmm. Without looking at the code, it looks like a utf8 encoding problem. The reason I have this suspicion is that for the label you are getting a single character output, for the console you are getting a two character output. I suspect that while the two strings look the same they are actually encoded differently.

Both the console and the label use the same underlying font routines so the only way they would give different output for the same input is if perhaps the console is printing byte by byte instead of character by character or string print.

Try putting the string into a variable and then using that variable to print both the label and the console. If you get the same result let me know and I will go digging.

[Joshua]

Hi @inmarket, I get the same result as before with this code:

    char* accented = "é";

    gwinSetText(ghTitleLbl, accented, gTrue); // works, sets text to "é"

    gwinPrintf(ghConsole, accented);               // doesn't work, prints "é"

[Joshua]

@inmarket are you able to reproduce this?

Edited May 11, 2021 by Joshua

[inmarket]

Sorry, I have been away and unable to respond. I will be back this coming weekend and will have a good look as soon as I can. The most likely reason is gwinPrintf is processing byte by byte rather than character by character. 

[Joshua]

@inmarket Sorry to bother you again, just wondering if you had the chance to look at this yet. 

[Joshua]

Still looking if there's a solution for this

[Joel Bodenmann]

I might have a look at this in the meantime.

Could you provide the exact font you used (ideally both the font source file (if possible due to licensing) as well as the output of the font converter).

[Joshua]

Hi Joel, attached is the font converter output. It's the DejaVuSans font. I'm pretty sure I used the one that's included in the ugfx source. I converted it with anti aliasing and a custom range to include accented characters used in French and the degree symbol.

This same font is displaying accented characters fine in label widgets, but not in our console widget.

Thanks for looking into this.

DejaVuSans20_aa_c.c

Edited October 1, 2021 by Joshua
clarification

[inmarket]

I have had a decent look at this. Unfortunately it is not easy to fix. The issue is that gwinPrintf is processing the string byte by byte rather than utf8 character by utf8 character. This is completely consistent with the original C standard printf but definitely not what you need.

To get around this in the short term gsprintf it into a text buffer and then string print that. Note: even with this, %c will not work for utf8 extended chars but %s will.

I am currently deep in implementing generic rotation for text drawing for ugfx v3 so it will probably be a while before I can come back to solve this properly. 

[Joshua]

Hi @inmarket, 

There is no gsprintf function, do you mean sprintf? Even when I use sprintf with gwinPutString, I get the same result. Let me know if this is the solution you had suggested, otherwise I am misunderstanding you.

[inmarket]

The function might be called sprintg or something like that (I am off-computer presently so can't see the proper function name). It is the ugfx equivalent of sprintf. 

Note: sprintf from the c library won't work as it is also not utf8 aware at all.

Sprintg (the ugfx equivalent) may work when you use %s to include the character (obviously as part of a string). I don't expect it to work with %c or as part of the format string itself.

Note: this is a work around that should work but there are no guarantees. To be fixed properly this requires substantial changes to the printf engine in ugfx. The extra complexity is the whole reason very few sprintf implementations support it except on desktop systems like Windows or Linux.

 

[Joel Bodenmann]

The function in question is named snprintg(): https://git.ugfx.io/uGFX/ugfx/src/branch/master/src/gfile/gfile.h#L378