uGFX support for USB mouse with BeagleBone Black

[PaulB]

I am evaluating uGFX (2.8) on a BeagleBone Black, with 4D systems gen4 touchscreen.
I have got the basic display test demo working, using the linux framebuffer driver.
I have built and run the combo demo, using "gmouse_lld_linux_event_board.h" and "gmouse_lld_linux_event.c".
For other reasons, the touchscreen is not currently working, and I want to use it with a USB mouse. The mouse works with the Beagle desktop manager GUI (LXQt), but not with the uGFX app.

I have got GINPUT_NEED_MOUSE and GINPUT_TOUCH_NOTOUCH defined as TRUE.
The mouse events arrive in the driver _read function, but it seems that only touchscreen type events are processed, eg "if (ev.type == EV_ABS" etc, in gmouse_lld_linux_event.c line 85.
The mouse produces events of type EV_REL, and so no movement is processed.
For button presses, there is a similar check on the event code, where it uses BTN_TOUCH (330) and not BTN_MOUSE (272) (from "input-event-codes.h")

There is also no mouse cursor displayed.

Is a mouse (and its cursor) supported in any way ?
It seems that they should be, since files are named gmouse, and the documentation mentions touchscree/mouse for the linux framebuffer.

 

Many thanks

 

[Joel Bodenmann]

Hello @PaulB and welcome to the µGFX community!

It's correct that the Linux event board file that comes with the library only handles touchscreen events at the moment. However, writing new drivers/board files or extending existing ones is very easy.

Currently there's no support for a real visual mouse cursor. That would require a lot of resources as you'd have to either keep an entire copy of the rendered framebuffer or you'd need to figure out which parts need to be repainted which would require partial redrawing of the widgets. These are features that are currently not there because µGFX is meant to work with as little resources as possible. There are plenty of things that we'd like to add but we're currently lacking man-power. Every feature needs to be optional and managing things like this as an optional feature is extremely complex.

I'm gonna leave the explanation of the mouse vs. touchscreen stuff to @inmarket. He wrote the original version of GINPUT and I'm not sure of all the details right now - I'd have to digg into it.

[PaulB]

Ok, thanks for the information, at least I don't need to go searching any more for something that's not there.

I'll see how things go, whether to try and implement a basic mouse cursor, or wait until the touchscreen is working.

Regards.

[Joel Bodenmann]

1 hour ago, PaulB said:

or wait until the touchscreen is working.

What's the problem there? The existing GINPUT Linux event driver has been successfully used with touchscreens in many projects.
Keep in mind that every Linux distribution tends to use different device paths (eg. /dev/event0 and so on) so minor modification is necessary. If I remember correctly there's a proper #define for that in the board file.

[PaulB]

Nothing to do with uGFX, I'm sure this will work.

There are problems in the recent Beagle images regarding the 4D touchscreen, it doesn't work reliably in any situation, no way to calibrate etc, so hopefully there will be an update for this soon.

 

[Joel Bodenmann]

Do you need to use a 4D-Systems display? You can use any kind of display and touchscreen with µGFX.

[inmarket]

Also uGFX supports calibration. It is not turned on normally for the Linux event mouse driver but that is only a flag in the vmt structure in the driver. Change that flag and you will have calibration support.

[inmarket]

Sorry for the delay.

 

Try replacing the _read() routine in the Linux event mouse driver with the following code...

static bool_t _read(GMouse* m, GMouseReading* pdr)
{
	int i;
	int rb;
	struct input_event ev[64];
	privStruct* priv;

	// Retrive the private area struct
    priv = (privStruct*)(m+1);

	// Assume not touched
	pdr->buttons = priv->lastReading.buttons;
	pdr->z = priv->lastReading.z;
	pdr->x = priv->lastReading.x;
	pdr->y = priv->lastReading.y;

	// Read
	rb = read(priv->fd, ev, sizeof(ev));

	// Parse
	if (rb > 0) {
		rb /= sizeof(ev[0]);
		for (i = 0;  i < rb; i++) {
			switch(ev[i].type) {
			case EV_KEY:
				switch(ev[i].code) {
				case BTN_TOUCH:
					if (ev[i].value == 1)		pdr->z = 1;
					else if (ev[i].value == 0)	pdr->z = 0;
					break;
				case BTN_LEFT:
					if (ev[i].value == 1)		pdr->buttons |= GINPUT_MOUSE_BTN_LEFT;
					else if (ev[i].value == 0)	pdr->buttons &= ~GINPUT_MOUSE_BTN_LEFT;
					break;
				case BTN_MIDDLE:
					if (ev[i].value == 1)		pdr->buttons |= GINPUT_MOUSE_BTN_MIDDLE;
					else if (ev[i].value == 0)	pdr->buttons &= ~GINPUT_MOUSE_BTN_MIDDLE;
					break;
				case BTN_RIGHT:
					if (ev[i].value == 1)		pdr->buttons |= GINPUT_MOUSE_BTN_RIGHT;
					else if (ev[i].value == 0)	pdr->buttons &= ~GINPUT_MOUSE_BTN_RIGHT;
					break;
				}
				break;
			case EV_ABS:
				switch(ev[i].code) {
				case ABS_X:
					if (ev[i].value > 0) pdr->x = ev[i].value;
					break;
				case ABS_Y:
					if (ev[i].value > 0) pdr->y = ev[i].value;
					break;
				}
				break;
			case EV_REL:
				switch(ev[i].code) {
				case REL_X:
					pdr->x += ev[i].value;
					break;
				case REL_Y:
					pdr->y += ev[i].value;
					break;
				}
				break;
		}
	}

	// Store the current reading
	priv->lastReading.x = pdr->x;
	priv->lastReading.y = pdr->y;
	priv->lastReading.z = pdr->z;
	priv->lastReading.buttons = pdr->buttons;

	return TRUE;
}

 

Hopefully this will add support for a mouse. Note this code is untested and still doesn't display a mouse cursor. Let us know how it goes.

[PaulB]

Yes, many thanks for the suggestions !

My situation is that now I have an updated driver for the touchscreen (we need to use this particular one), so it works well enough, but there was still no calibration program, and it was off just enough to make it difficult to use. I didn't realise that uGFX had its own calibration support. I have enabled the flag as you suggested and now I can calibrate and use it, so thats good.

Thanks also for the updated mouse _read function. It works, but without a cursor its obviously difficult to use. I will keep this and come back to it when I have time, as I would like to get some mouse support with cursor working in the future.

 

If I may ask about something probably similar to mouse cursors, I have a live image which needs several labels and icons (selectable) over the top of it. I have an initial version working using a GWin image widget, with a label widget over it. Every time the image updates, I have to update the label, and this obviously causes some flicker. I have tried a custom draw for the label so it doesn't fill its background, but its only a slight improvement. 

Would you be able to point me to any ideas about the best way to update an image whilst keeping text (or whatever) on top of it ?

 

Regards

 

[Joel Bodenmann]

Glad to hear that you managed to get it working!

 

3 hours ago, PaulB said:

Would you be able to point me to any ideas about the best way to update an image whilst keeping text (or whatever) on top of it ?

The best solution is to pre-render everything into an off-screen area called a pixmap. Once everything has been rendered you can simply copy the pixmap contents to the real display. Therefore, there won't be any flickering at all. You can find the documentation about pixmaps here.

I'd also recommend you to think about writing a custom widget. That would allow you to better control the way the widget is being rendered and you could manually cache the parts of the images that are going to be overwritten by the texts on top of it. You can find an entire guide on how to create a custom widget here.

 

I hope that helps!

[inmarket]

The obvious problem here is the time delay between the background re-render and the text being redrawn.

These are several choices that could reduce that time:

1. Break up your background image so that no part overlaps a label. Then use image background labels. This way the image under the label text is small and there is no overlapping.

2. As Joel suggested above use an offscreen pixmap. This is more complex, takes lots of RAM and slows refresh but removes the visual gap.

3. A custom widget that effectively does option 1 above but in a single widget (as Joel also suggested).

I suggest #1. It is the easiest and reduces the whole background display needing to be updated when you change a label.

Overlapping windows are not handled efficiently in uGFX because of its lack of complex clip regions.

[PaulB]

Many thanks for these ideas, I will check them out.

At the moment there is enough RAM available, but I don't want to slow down the refresh if possible. There are around 10 labels and icons overlaid on the image at present.

I will have a look at both options.

Thanks again.

 

[Joel Bodenmann]

Don't hesitate to ask if you have any further questions - we're happy to help  

[PaulB]

For your info, I have gone the pixmap route for now, and I will check out custom widget rendering later.

Also for your info, after enabling pixmaps in the conf.h, I got compile errors regarding GDISPVMT_pixmap being undefined, and various other things. I am of course using the single file inclusion way of building uGFX in an IDE, and after checking a few forum posts about this, there didn't seem to be much hope. However, I moved just gdisp_pixmap.c into my local project, eventually realised I had to also remove gdisp_pixmap.c from the gfx_mk.c, and this fixed the build, and the pixmap worked with just a label.

For your info again, I tried replacing the Display in my existing gwinImageCreate so that the image went through the pixmap (uses gwinImageOpenMemory with native format memory buffer) and it failed miserably, displaying only parts of the image, very slow, and regularly crashing with memory errors.

Then I tried the better way of copying the image directly onto the pixmap surface buffer, with no gwinImage at all, and this works perfectly. The live image and label on top are displayed with no flicker at all, same speed and cpu usage as before.

So thanks for the help.

 

[inmarket]

Using pixmaps with single file make is definitely not supported currently. Even though you managed to get it to compile and partially work the crashes you are seeing will be related to the build method. Effectively the display system is not being built correctly for that scenario.

This is all being fixed in uGFX V3. All uGFX features will be available with single file make.

[Joel Bodenmann]

Just FIY: That is documented here: https://wiki.ugfx.io/index.php/Getting_Started#Single_File_Inclusion (the Note)

[PaulB]

Ok, I get the message 

I will check for other options as well...

[Joel Bodenmann]

17 hours ago, PaulB said:

Ok, I get the message 

I'm sorry, I didn't mean to be offensive or to go down the "RTFM"-road. I just meant to post it here for completion when other people come around this topic in the future. Nothing personal at all!  

[PaulB]

Not at all, I really appreciate all your feedback.

Its good to hear that pixmaps will be officially supported with single-file make in V3. 

Are you able to give an approximate time for when this will be available ?

[inmarket]

Unfortunately not. The job has proven much bigger than we planned and we have had much less time to work on it than planned.

We are currently working on API, interface and architecture issues. Once those are complete all the uGFX drivers will need to be ported. At that point we will probably ask the community to help us just because of the number of drivers that need changing and testing.