[Development] Processing on Android, multi-touch and device orientation
Last update: 28-Jul-2014
|
|
Processing is a platform where you can easily write up GUI programs. It's Java-based meaning that you will be mostly writing Java code (OOP stuff and importing Java libraries). It was first designed for non-programmers so there is a lot of abstraction here, which is good for quickly prototyping some interactive graphical applications, yet at the same time support all Java code so if you want to go deep you can. I've made a list of steps which you need to take in order to install your program to an Android device here. Most of the steps are still relevent but changes were made in the recent (Jan 2014) versions of Processing. So although I won't repeat every here again, I'll cover some important points.
To set things up for Processing to work with Android requires installing a couple of tools. Here is a list of steps taken from having nothing installed. I've done this in both Windows 7 SP1 and Windows 8.1, and they work.
To set things up for Processing to work with Android requires installing a couple of tools. Here is a list of steps taken from having nothing installed. I've done this in both Windows 7 SP1 and Windows 8.1, and they work.
- Get the Java SE SDK: http://www.oracle.com/technetwork/java/javase/downloads/index.html. At the time of writing it's version 7u45. You should always get the latest version for security and efficiency.
- Get Processing (with IDE), make sure it has Android Mode: http://processing.org/download/. At this point you still can't use that mode yet (default is Java, which creates a Java application), go to the next step.
- Follow the instructions here: http://wiki.processing.org/w/Android, which gives you the Android SDK and some drivers. Note where you put the SDK folder as you'll need the path in the next step.
- Setup your Processing IDE for Android Mode, restart your computer if necessary.
- Create your Android (multitouch or not) application in the Processing IDE, and run it. If you are using code from others, make sure they work with Android
- For multitouch handling, read [1] and [2]:
[1] http://forum.processing.org/topic/a-basic-multi-touch-gesture-demo-for-android
[2] http://developer.android.com/reference/android/view/MotionEvent.html - Use Ant and command prompt to figure out compilation errors. This method is quite low level but when there is something wrong with your code, the Processing IDE doesn't always tell you exactly what is causing the issue. Hopefully the later versions will do it better.
So now you can write some Android app through Processing. Naturally you'll want the app to act like any other apps for mobile devices. And when developing something for a portable device, several additional inputs are possible. E.g. multi-touch, positional change ...etc. As Processing wasn't originally designed for Android, there is no Processing-specific code using those input, and you'll have to resort using the native Android SDK. Fortunately, most of the code sample you can find in the Android developer site are compatible in Processing, meaning you don't have to make a lot of changes (sometimes even just copy-and-paste).
To make use of the multi-touch capability of the mobile device, you'll be using this method:
public boolean surfaceTouchEvent(MotionEvent event) { ... }
instead of the onTouchEvent(MotionEvent event), which is typically used in Android apps.
For more details, check out the original blog post I made [here], particularly the part where you handle a MotionEvent when the user lifts all their fingers.
To make use of the multi-touch capability of the mobile device, you'll be using this method:
public boolean surfaceTouchEvent(MotionEvent event) { ... }
instead of the onTouchEvent(MotionEvent event), which is typically used in Android apps.
For more details, check out the original blog post I made [here], particularly the part where you handle a MotionEvent when the user lifts all their fingers.
The last piece of the puzzle is detecting the orientation of the device. This again is almost mobile device specific as one would not tilt their PC, and hence not natively supported in the Processing code. So we have to once again tap in to the native Android SDK for that. Here is a brief description of the concept:
You provide something that responds to changes of a sensor (by implementing the SensorEventListener), and ask the system to give you a manager of all sensors, and ask for a specific sensor (in this case it's Sensor.TYPE_ORIENTATION). Then you "register" your something to that sensor. One important thing to keep in mind is you'll have to "unregister" your something when you no longer need the measurements (e.g., when your pause or exit the app), otherwise the operating system will think you still need those measurements and keeps taking in values, thus draining the battery.
With these readings and some tweaks with the camera angle in the Processing code, you can get this little app running:
You provide something that responds to changes of a sensor (by implementing the SensorEventListener), and ask the system to give you a manager of all sensors, and ask for a specific sensor (in this case it's Sensor.TYPE_ORIENTATION). Then you "register" your something to that sensor. One important thing to keep in mind is you'll have to "unregister" your something when you no longer need the measurements (e.g., when your pause or exit the app), otherwise the operating system will think you still need those measurements and keeps taking in values, thus draining the battery.
With these readings and some tweaks with the camera angle in the Processing code, you can get this little app running:
To me I think the trickiest part is to figure out what those 3 axes map to (and their range). Because a mobile device has two display modes: Portrait and Landscape, and the mapping seems to change when there is a mode switch. It is possible to force the application to be at one mode all the time, but if later on one wants to do some adaptive UI, this might be a concern. There are a few good articles explaining how to do that. If you are interested you can explore this.
For more details, check out the original blog post I made [here].
For more details, check out the original blog post I made [here].
