[Research] Around-Device Interaction with Magnets
Last edit: 26-Sep-2019
Did you know that there are over a dozen sensors in your modern mobile phone? We are not talking about the obvious ones like camera or touchscreen, but the ones that sense the phone's surroundings, for example, air pressure, humidity, ambient light, etc. We don't normally use them, but some apps do to give you a better idea of what's happening around you. One of them is the magnetometer.
A magnetometer is a sensor that measures the Earth's magnetic field and gives out the field strengths along its 3 reference axes. Usually it is used by a map app to orient your map, or some toy app to detect presence of metal. What's interesting is that when you have a strong magnet next to the magnetometer it can pick up the changes in the magnetic field in a consistent and responsive manner. If you have a magnetic ring (or any round magnet) and rotate it, the changes in the field strengths is consistent enough for the magnetometer to deduce the orientation of the magnet's axis. Here is an illustration of the set up.
Did you know that there are over a dozen sensors in your modern mobile phone? We are not talking about the obvious ones like camera or touchscreen, but the ones that sense the phone's surroundings, for example, air pressure, humidity, ambient light, etc. We don't normally use them, but some apps do to give you a better idea of what's happening around you. One of them is the magnetometer.
A magnetometer is a sensor that measures the Earth's magnetic field and gives out the field strengths along its 3 reference axes. Usually it is used by a map app to orient your map, or some toy app to detect presence of metal. What's interesting is that when you have a strong magnet next to the magnetometer it can pick up the changes in the magnetic field in a consistent and responsive manner. If you have a magnetic ring (or any round magnet) and rotate it, the changes in the field strengths is consistent enough for the magnetometer to deduce the orientation of the magnet's axis. Here is an illustration of the set up.
Using this idea, I started a project that explored different ways to interact with the mobile phone using a magnetic ring, throwing in a few concepts from Around-Device Interaction (ADI, moving the interaction space beyond the device itself) and Tangible User Interfaces (TUIs, incorporating physical objects in the interaction), and presented them at TEI in 2018 as a work-in-progress.
It was really fun. Some people at the conference even described it as "magic" as I controlled the mockup interface in the mobile phone.
But then I wanted to go further and see how it impacts actual usage. So I wrote an app that uses the rotatory motions of the magnetic ring as input to play a variation of the game Space Invader. On top of that, I ran a study investigating how it affected performance and user experience, and wrote a paper on it.
The paper was accepted at MobileHCI 2019 (yay!). Here is a video summarizing the paper.
But then I wanted to go further and see how it impacts actual usage. So I wrote an app that uses the rotatory motions of the magnetic ring as input to play a variation of the game Space Invader. On top of that, I ran a study investigating how it affected performance and user experience, and wrote a paper on it.
The paper was accepted at MobileHCI 2019 (yay!). Here is a video summarizing the paper.
How to use
To help interested parties to get started, I have made the code of the app available. It is written using Unity 3D, but the intention is anyone can extract the core code that converts the rotatory gestures into an input value, and then use that for their own development.
Here is the link to its GitHub page: https://github.com/thisisvictor/MagTroGames
Limitations
While cool and so far good at what it does, there are limitations to this way of input. First, there is only a small number of gestures one can do with a magnetic ring (rotate, lift, flip; and the code only uses rotate); and it doesn't scale to multiple magnetic rings, as the magnetometer will just be measuring the resultant field from the magnets. Second, if you place the setup on a metallic table, chances are the magnetic field would be skewed, and the magnetic ring would simply be stuck to the table. It is also worth nothing that the ring is "diametrically magnetized", meaning that the magnet's polarity axis needs to be orthogonal to the z-axis of the magnetometer.
What's next
I really want to create a game that makes better use of a magnet. The Space Invader variation is a good start, but if there is time I'd like to build something original, and other than just games.
Lastly, the same set up should be viewed as a complementary input modality with touches.
To help interested parties to get started, I have made the code of the app available. It is written using Unity 3D, but the intention is anyone can extract the core code that converts the rotatory gestures into an input value, and then use that for their own development.
Here is the link to its GitHub page: https://github.com/thisisvictor/MagTroGames
Limitations
While cool and so far good at what it does, there are limitations to this way of input. First, there is only a small number of gestures one can do with a magnetic ring (rotate, lift, flip; and the code only uses rotate); and it doesn't scale to multiple magnetic rings, as the magnetometer will just be measuring the resultant field from the magnets. Second, if you place the setup on a metallic table, chances are the magnetic field would be skewed, and the magnetic ring would simply be stuck to the table. It is also worth nothing that the ring is "diametrically magnetized", meaning that the magnet's polarity axis needs to be orthogonal to the z-axis of the magnetometer.
What's next
I really want to create a game that makes better use of a magnet. The Space Invader variation is a good start, but if there is time I'd like to build something original, and other than just games.
Lastly, the same set up should be viewed as a complementary input modality with touches.
Related publication:
[1] Cheung, V. and Girouard, A. (2019). Tangible Around-Device Interaction Using Rotatory Gestures with a Magnetic Ring. To appear in Proceedings of MobileHCI2019: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services. October 1-4, Taipei, Taiwan.
https://doi.org/10.1145/3338286.3340137
[2] Cheung, V. and Girouard, A. (2018). Exploring Around-Device Tangible Interactions for Mobile Devices with a Magnetic Ring. In Proceedings of TEI2018: International Conference on Tangible, Embedded and Embodied Interactions, p. 108-114. Stockholm, Sweden, March 18-21, 2018.
https://doi.org/10.1145/3173225.3173283
[1] Cheung, V. and Girouard, A. (2019). Tangible Around-Device Interaction Using Rotatory Gestures with a Magnetic Ring. To appear in Proceedings of MobileHCI2019: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services. October 1-4, Taipei, Taiwan.
https://doi.org/10.1145/3338286.3340137
[2] Cheung, V. and Girouard, A. (2018). Exploring Around-Device Tangible Interactions for Mobile Devices with a Magnetic Ring. In Proceedings of TEI2018: International Conference on Tangible, Embedded and Embodied Interactions, p. 108-114. Stockholm, Sweden, March 18-21, 2018.
https://doi.org/10.1145/3173225.3173283