Software application development
This blog is intended to provoke discussion and exchange between like minded software application developers, engineers, architects, project managers - and keen hobbyists too.
Tuesday 11 August 2009, 8:55 AM
Do we need a multi-touch gesture alphabet?
So as this development stream becomes more widespread, my question is this: do we need to agree on a set alphabet for multi-touch gestures and label them all now so that we avoid fragmentation and incompatibilities in this space in the future?
Last week I mentioned Embarcadero’s sneaky peak ‘let’s get three press releases out of one story’ programme for releasing its RAD Studio 2010 product. This week the company is sneakily peaking its second preview of this tool, which it claims will enables developers to build touch based GUI, tablet, touchpad and kiosk applications with a flexible touch enabled framework. But how many touch gestures has Embarcadero provisioned for? The answer is 30 and these include left, right, up, down, scratch-out and interactive multi-touch gestures like pan, zoom and rotate.
Who says there are 30 multi-touch gestures then, or that Embarcadero has been intuitive enough to pick the most logical group. I’m sure they are very sensible choices. But what if we all start using touch tables, iPhones and kiosks that much more interactively and we start to develop a new set of touch behaviour? Who is going to keep track of this and try and provide some nomenclature to classify the way we work with these machines so that developers can programme with optimum efficiency?
Let me give you an example. When I am in the states I use a fantastic DVD rental service called RedBox where the movies are a dollar a night. The kiosk interface is OK, but it tends to time out and think I’ve left the scene when in fact I am just standing there still trying to decide which Robert De Niro masterpiece I am going to watch. What if we came up with a new touch behaviour? Say I move my finger in a circle meaning I am pondering. That way the system might now it needs to give me more time or more information.
Obviously this is just thinking out loud. But what if becomes a real issue? After all, there are about 100 touch receptors in each of your fingertips.We have an alphabet on our keyboards. Don’t we need a set of standards for touch - especially if we are to run with good interoperability across multiple operating systems?
PreviousComments on this post
An "alphabet" for a touch interface would be complicating it too much. For an interface like this to work, it just has to do what people expect, not what they've been taught works. Usability is never an easy job, but for such a departure from the point-and-click interface of a mouse, it's going to be even harder.
For instance, it may seem obvious that if you want to move a window, you touch it and drag it to its new position. But what if you want to select the text inside that window? Your natural instinct is probably to drag your finger across the text you want, except this will move the window, won't it?
It may turn out that a touch interface is only good for manipulating the user interface on a macro level — moving pieces of it around, altering orientation or size. Once you need to interact with the content of the interface, you'll need to revert to something a bit less whizzy and a lot more like what we have at the moment.
I don't know if it would be overcomplicated Jonathan - although I bow to your superior knowledge on this matter.
Your point about macro level interfacing is excellent - it's where I was leaning with my comments on interoperability.
I found some other discussion on this subject here:
http://forums.macrumors.com/archive/index.php/t-413573.html
... although I'm sure of its full worth.
Perhaps you need two points on your left hand to hold a text box so that the touch screen knows you want to go deeper into the interface... then (stay with me) .. then... your right hand is able to manipulate the text inside.
So this is the multi-touch gesture alphabet letter - I for Igloo. Meaning INSIDE edit.
We could go on and on...
Thanks for your comment regardless, much appreciated as always.
Regards - Adrian
Hmmmm valid points id say yes we need something that is universal in that department but it must not be to overly complicated starting off with 30 is a good start, there's nothing to say that later on that can't be expanded on or redefined.
As for the examples given above in such an instance to move the box then i use a grabbing gesture with my hand or two or more fingers, for an highlight gesture then one finger over the text or paragraph.
To switch to an editing mode then a double tab of two or more fingers.
Double tap of one finger could mimic left mouse button so on, the list goes on and on so some good decisions would haft to be made.
So who do you think should be responsible as the governing body here?
We don't want that large software company headquartered on the west coast of our former new world colony being in control - or do we?
Open sourcing this exercise might lead to too many conceptualisations, perhaps a little proprietary lock-down would be good.
;-)
haha :)
I think open source would be a good way to go but unfortunately open source relies on people having the technology to play with and that reduces the number's of which can work on it to truly make it an universal format.
Given the fact that these touch screens are still expensive I think development as it is atm is focusing on smaller things like iphones and such, through the means of application stores or open source.
But as you have said this could lead to two many forks which could either send us to the ultimate idea or kill it all in one go.
As for a single governing body for such a thing god knows? its not the manufacturers nor the developers of the equipment used, that just leaves the application layer developers to use the sensors as they see fit and we can only hope that they maintain some sort of universal format use when doing so.
Edit;
Upon further pondering with regards to the MS table one of the ways I've seen them demonstrate the resizing of window is to use two hands this is all very well but where does that leave disabled people, so as stated things need to be very well thought out.
But its certainly worth thinking about, sort of like sign language.
You make a strong point about the fact that open source relies on people having the technology and that the cost of the units may be prohibitively expensive (apart from iPhone - which anyway I still can't afford) being around.
I wonder if emulators would/could fill some of this gap? Similar perhaps to the way mobile app emulation/test systems work.
I suspect not, the whole appeal of being in the touch/surface development space is the ability to physically be able to touch these things.
AdrianB
Yes many of the small devices out there like iphones etc are still by no means cheap, as for making use of emulation techniques yeah i wouldn't be at all surprised if they was a way to utilize them in development.
But of course that would be some what limited as you say you really need to get ya hands on the actual equipment to be able see what its limitations are, such as do the sensors allow for various degrees sensitivity and if so can it be triggered by the slightest touch of feather or a straw and so on.
But lets not forget about MS's Project Natal which could over shadow all of this for home use, but maybe not for public places maybe thats another strong reason why developers need to think about it.
