MT, Linux, and other stuff

Ryan Huffman

TUIO X Input Driver – Current Progress (Part 2)

with 4 comments

This is part 2 of my previous post which discussed the current progress of my GSoC project and also included a video that showed the driver in action.

MDs, continued

In part 1 I ended off by talking about MDs and cursors.  To create new MDs or cursors you need to use the XI2 api, which can’t (or, atleast, shouldn’t) be done in the driver.  From a users point of view, this makes sense.  If I plug a mouse in, I would be pretty pissed if the driver decided that it wanted to make a new MD and attach itself to it without me telling it to do that.  Granted, this driver is a special case (especially because we don’t have xblobevents yet), but I still think it is important to use everything as it was meant to be used.  If we start making exceptions everywhere, we will end up with a bunch of special purpose applications, which is what we were trying to avoid in the first place.

To this end, I have written the driver such that it leaves MDs untouched.  In the video, though, the second half shows multiple cursors moving around at once.  To do this, I was required to manually move the subdevices that the driver creates to their own master devices.  Let me give you quick example.

When starting X, it will parse my xorg.conf, and come across this section:

Section "InputDevice"
    Identifier      "mt0"
    Driver          "tuio"
    Option          "SubDevices"      "3"
EndSection

When reaching this, it will create a new device that uses the tuio driver, and set the “SubDevices” option for the device.  Once this device is started, it will create the number of devices specified by “SubDevices” to be used to route events through (it also routes events through itself).

Once the tuio driver is loaded and its subdevices are created, the output of `xinput list –short` will look something like this:

[ryan@namffuh ~]>> xinput list --short
⎡ Virtual core pointer                    	id=2	[master pointer  (3)]
⎜   ↳ Virtual core Xtst pointer               	id=4	[slave  pointer  (2)]
⎜   ↳ mt0                                     	id=6	[slave  pointer  (2)]
⎜   ↳ Macintosh mouse button emulation        	id=8	[slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                   	id=9	[slave  pointer  (2)]
⎜   ↳ mt0 subdev 0                            	id=12	[slave  pointer  (2)]
⎜   ↳ mt0 subdev 1                            	id=13	[slave  pointer  (2)]
⎜   ↳ mt0 subdev 2                            	id=14	[slave  pointer  (2)]
⎣ Virtual core keyboard                   	id=3	[master keyboard (2)]
    ↳ Virtual core Xtst keyboard              	id=5	[slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                  	id=7	[slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard            	id=10	[slave  keyboard (3)]
    ↳ Video Bus                               	id=11	[slave  keyboard (3)]

“mt0” is the device as specified in the xorg.conf, and the “mt0 subdev #” devices are the devices create by the “mt0” device.  At this point all input events will be routed through the “Virtual core pointer” MD and thus through the one pointer that is active.  To create new cursors/MDs, xinput can be used:

[ryan@namffuh ~]>> xinput create-master "TUIO MD 0"
[ryan@namffuh ~]>> xinput create-master "TUIO MD 1"
[ryan@namffuh ~]>> xinput create-master "TUIO MD 2"
[ryan@namffuh ~]>> xinput list --short             
⎡ Virtual core pointer                    	id=2	[master pointer  (3)]
⎜   ↳ Virtual core Xtst pointer               	id=4	[slave  pointer  (2)]
⎜   ↳ mt0                                     	id=6	[slave  pointer  (2)]
⎜   ↳ Macintosh mouse button emulation        	id=8	[slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                   	id=9	[slave  pointer  (2)]
⎜   ↳ mt0 subdev 0                            	id=12	[slave  pointer  (2)]
⎜   ↳ mt0 subdev 1                            	id=13	[slave  pointer  (2)]
⎜   ↳ mt0 subdev 2                            	id=14	[slave  pointer  (2)]
⎣ Virtual core keyboard                   	id=3	[master keyboard (2)]
    ↳ Virtual core Xtst keyboard              	id=5	[slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                  	id=7	[slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard            	id=10	[slave  keyboard (3)]
    ↳ Video Bus                               	id=11	[slave  keyboard (3)]
⎡ TUIO MD 0 pointer                       	id=15	[master pointer  (16)]
⎜   ↳ TUIO MD 0 Xtst pointer                  	id=17	[slave  pointer  (15)]
⎣ TUIO MD 0 keyboard                      	id=16	[master keyboard (15)]
    ↳ TUIO MD 0 Xtst keyboard                 	id=18	[slave  keyboard (16)]
⎡ TUIO MD 1 pointer                       	id=19	[master pointer  (20)]
⎜   ↳ TUIO MD 1 Xtst pointer                  	id=21	[slave  pointer  (19)]
⎣ TUIO MD 1 keyboard                      	id=20	[master keyboard (19)]
    ↳ TUIO MD 1 Xtst keyboard                 	id=22	[slave  keyboard (20)]
⎡ TUIO MD 2 pointer                       	id=23	[master pointer  (24)]
⎜   ↳ TUIO MD 2 Xtst pointer                  	id=25	[slave  pointer  (23)]
⎣ TUIO MD 2 keyboard                      	id=24	[master keyboard (23)]
    ↳ TUIO MD 2 Xtst keyboard                 	id=26	[slave  keyboard (24)]

Now 3 new MDs have been created, along with 3 new cursors on the screen.  Ignore the Xtst devices, they are automatically created and are used for testing.  Now we must reattach the subdevices to their new MDs:

[ryan@namffuh ~]>> xinput reattach 12 15
[ryan@namffuh ~]>> xinput reattach 13 19
[ryan@namffuh ~]>> xinput reattach 14 23
[ryan@namffuh ~]>> xinput list --short
⎡ Virtual core pointer                    	id=2	[master pointer  (3)]
⎜   ↳ Virtual core Xtst pointer               	id=4	[slave  pointer  (2)]
⎜   ↳ mt0                                     	id=6	[slave  pointer  (2)]
⎜   ↳ Macintosh mouse button emulation        	id=8	[slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                   	id=9	[slave  pointer  (2)]
⎣ Virtual core keyboard                   	id=3	[master keyboard (2)]
    ↳ Virtual core Xtst keyboard              	id=5	[slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                  	id=7	[slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard            	id=10	[slave  keyboard (3)]
    ↳ Video Bus                               	id=11	[slave  keyboard (3)]
⎡ TUIO MD 0 pointer                       	id=15	[master pointer  (16)]
⎜   ↳ mt0 subdev 0                            	id=12	[slave  pointer  (15)]
⎜   ↳ TUIO MD 0 Xtst pointer                  	id=17	[slave  pointer  (15)]
⎣ TUIO MD 0 keyboard                      	id=16	[master keyboard (15)]
    ↳ TUIO MD 0 Xtst keyboard                 	id=18	[slave  keyboard (16)]
⎡ TUIO MD 1 pointer                       	id=19	[master pointer  (20)]
⎜   ↳ mt0 subdev 1                            	id=13	[slave  pointer  (19)]
⎜   ↳ TUIO MD 1 Xtst pointer                  	id=21	[slave  pointer  (19)]
⎣ TUIO MD 1 keyboard                      	id=20	[master keyboard (19)]
    ↳ TUIO MD 1 Xtst keyboard                 	id=22	[slave  keyboard (20)]
⎡ TUIO MD 2 pointer                       	id=23	[master pointer  (24)]
⎜   ↳ mt0 subdev 2                            	id=14	[slave  pointer  (23)]
⎜   ↳ TUIO MD 2 Xtst pointer                  	id=25	[slave  pointer  (23)]
⎣ TUIO MD 2 keyboard                      	id=24	[master keyboard (23)]
    ↳ TUIO MD 2 Xtst keyboard                 	id=26	[slave  keyboard (24)]

And now the subdevices will control individual cursors.  If the goal is to control independent cursors, then it would be ideal to write a script/app to make these hierarchy changes automatically, rather than doing it manually every time.  This is something I will probably put together in the next few days.

Currently this is where the driver is at.  It will route events through the “core” device and the subdevices.  If there are more active blobs or objects than there are devices to send events through, then they will have to wait until another blob/object is removed and there is a free device to send events through.

Hopefully this has been making sense – if it hasn’t, please let me know and I can help clear it up!  Also, if anyone has any suggestions, I would really appreciate that as well.

You might be wondering what we can actually do at this point with the tuio driver.  Without a large list of Xi2 applications to play with, there isn’t much we can do with it out of the box.  This week I am going to try to modify several TUIO based applications to receive XI2 input by modifying their input layer – the layer which receives and parses tuio messages.  I will make posts as progress is made, and maybe even have a video by the end of this or next week.

Comments are encouraged!  Thanks!

⎡ Virtual core pointer                        id=2    [master pointer  (3)]
⎜   ↳ Virtual core Xtst pointer                   id=4    [slave  pointer  (2)]
⎜   ↳ mt0                                         id=6    [slave  pointer  (2)]
⎜   ↳ Macintosh mouse button emulation            id=8    [slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                       id=9    [slave  pointer  (2)]
⎣ Virtual core keyboard                       id=3    [master keyboard (2)]
↳ Virtual core Xtst keyboard                  id=5    [slave  keyboard (3)]
↳ ThinkPad Extra Buttons                      id=7    [slave  keyboard (3)]
↳ AT Translated Set 2 keyboard                id=10    [slave  keyboard (3)]
↳ Video Bus                                   id=11    [slave  keyboard (3)]
⎡ TUIO MD 0 pointer                           id=15    [master pointer  (16)]
⎜   ↳ mt0 subdev 0                                id=12    [slave  pointer  (15)]
⎜   ↳ TUIO MD 0 Xtst pointer                      id=17    [slave  pointer  (15)]
⎣ TUIO MD 0 keyboard                          id=16    [master keyboard (15)]
↳ TUIO MD 0 Xtst keyboard                     id=18    [slave  keyboard (16)]
⎡ TUIO MD 1 pointer                           id=19    [master pointer  (20)]
⎜   ↳ mt0 subdev 1                                id=13    [slave  pointer  (19)]
⎜   ↳ TUIO MD 1 Xtst pointer                      id=21    [slave  pointer  (19)]
⎣ TUIO MD 1 keyboard                          id=20    [master keyboard (19)]
↳ TUIO MD 1 Xtst keyboard                     id=22    [slave  keyboard (20)]
⎡ TUIO MD 2 pointer                           id=23    [master pointer  (24)]
⎜   ↳ mt0 subdev 2                                id=14    [slave  pointer  (23)]
⎜   ↳ TUIO MD 2 Xtst pointer                      id=25    [slave  pointer  (23)]
⎣ TUIO MD 2 keyboard                          id=24    [master keyboard (23)]
↳ TUIO MD 2 Xtst keyboard                     id=26    [slave  keyboard (24)]
Advertisements

Written by rhuffman

July 6, 2009 at 9:52 pm

Posted in GSoC, MultiTouch

Tagged with

4 Responses

Subscribe to comments with RSS.

  1. Sweet! Thanks for all of your hard work. Good luck with the rest of the project.

    andruk

    July 7, 2009 at 12:39 am

  2. […] If everything loads correctly (which, hopefully it will), then you be able to send TUIO data to your computer and it should respond accordingly.  If you want to use multiple cursors then you will need to use xinput to add new master devices and reattach the subdevices, which was described in my last post. […]

  3. How do I make the sub devices usable? I have 4 movable cursors on the screen but is there a way to make them click on things?

    mcoleman44

    March 15, 2010 at 11:20 pm

    • If inside your configuration you don’t have “PostButtonEvents” set to false (by default it is set to true), the driver should emulate a mouse button-down every time you begin to move your cursor (if you’re using a MT screen, this would be when you put your finger on the screen), and a mouse-up when you release it. If that doesn’t work, please let me know.

      Ryan

      rhuffman

      March 15, 2010 at 11:36 pm


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: