JY Tigli Homepages

• Franck Dechavanne, SI5 IAM
• Nabil El Moussaid, SI5 IAM
• Etienne Strobbe, SI5 IAM

• Phone : NONE

• IMEI : N/A

Activity recognition to set phone mode

The goal of the project is to automatically change the smartphone's audio profile depending on the user's activity and how the device is positioned.

We developed a simple Windows Phone 8.1 application (not tested on an actual phone, only the Visual Studio emulator) containing a few views to display data read from the sensors, a graphical representation of the clusters and multiple testing functions that replace the Background Agent we initially planned (canned because of issues using the WASAPI COM API to capture raw audio and WinRT projects, required for background agents).

First page of the application. We can see the current mode (Unknown as the application was just launched) and the sensors' data, periodically refreshed (details further down the document). The checkbox triggers refreshing the data.

Second page of the application. We can see all the testing buttons (features detailed further down the document).

Third page of the application. We can see a representation of the clusters, on two axes (instead of the 3 we take into account). This choice was made because, on the emulator, light sensing doesn't work, so the values would never change.

A sample notification.

A solution containing a single project, that builds a single Windows Phone 8.1 application. projetplim_-_group_4.zip

The project is built using Visual Studio 2015 and as such requires this version of the IDE. The target platform is a Windows Phone 8.1 device with an accelerometer, a light sensor and a microphone. Audio recording relies on the WASAPI COM API, that is available on the emulator but may not be available on an actual phone. As we couldn't test it on a physical device we cannot tell for sure whether or not this would work.

Load the solution in Visual Studio 2015. Press Ctrl-Shift-B. Voila, the project is built. You can then upload it to a Windows Phone 8.1 device.

Launch the application. The first screen is only used to display data, you can tick the checkbox and enable a periodic refresh of the data displayed on this first screen. There is a 5 second delay between refreshes, and each refresh takes roughly 5 seconds to complete (because of the sound recording). You may see the first refresh 5 seconds after ticking the box.

The first line of data displayed, in blue, is the value of the Z axis of the accelerometer. The second line of data displayed, in orange, is the value of the light sensor. The third line of the data displayed, in red, is the average value of the amplitude in the 5 second recording.

Select the second tab by swiping to the right (or tapping the tab at the top). You are shown a selection of buttons to be used to test the application we developed. The slider at the top controls how much time is allowed before capturing data after pressing one of the mode buttons. Both mode buttons trigger a refresh of the sensors' data and adds data to the KMeans raw data, tagged with a certain mode. In practice, and as a limitation of our application, cluster semantics are set with the first press of the button and never change afterwards. Predictably, load demo data inserts a bunch of generated values in the KMeans raw data to do quick testing of the pretend BG task function. Pretend BG Task launches a refresh of the sensor data, compares it with the clusters and sets the phone's mode. In theory at least, as a limitation of the Windows Phone 8.1 API, an application cannot set either the global audio volume or the global phone mode. Such functions are not exposed by the API. We fell back on using toast notifications to pretend we're changing modes.

https://www.youtube.com/watch?v=q1lG_j-2h3Q

The video provides a walkthrough of the application. Use the first screen to show data, use the screen window to demo, use the third screen to visualize clusters. More detail on what each button does is available before.