Getting physical

Canonical

on 14 September 2010

Introduction

With the ingress in the market of products like Nintendo Wii, Apple iPhone and Microsoft Kinect, developers finally started realizing that there are several ways a person can control a computer besides keyboards, mice and touch screens. These days there are many alternatives, obviously based on hardware sensors, and the main difference is the dependency on software computation. In facts solutions based on computer-vision (like Microsoft Kinect) rely on state of the art software to analyze pictures captured by one or more cameras.
If you are interested on the technical side of it, I recommend to have a look to the following projects: Arduino, Processing and OpenFrameworks.

Usage with Ubuntu

During a small exploration we did internally few months ago, we thought about how Ubuntu could behave if it was more aware of its physical context. Not only detecting the tilt of the device (like iPhone apps) but also analysing the user’s presence.
This wasn’t really a new concept for me, in 2006 I experimented with a user proximity sensitive billboard idea. I reckon there is a value on adapting the content of the screen based on the distance with who is watching it.

We came up with few scenarios which are far to be developed and specified, hopefully will just open some discussions or, even better, help to start some initiatives.

Lean back fullscreen

If the user moves further from the screen while a video is playing on the focused window, the video will go automatically to fullscreen.

Fullscreen notifications

If the user is not in front of the screen, the notifications could be shown at fullscreen so the user can still read them from a different location.

Windows parallax

Since this is the year of 3D screens, we couldn’t omit a parallax effect with the windows. A gesture of the user could also trigger the launcher appearance (see prototype below).

Prototype

With few hours available, I mock up something very quickly in Processing using a face recognition library (computer-vision).
Despite it could be hard to detect the horizontal position of the user’s head without a camera, we are in no way defining the technology required. The proximity could be in-facts detected with infra-red or ultra-sound sensors.

Parallax and fullscreen interaction via webcam from Canonical Design on Vimeo.

Talk to us today

Interested in running Ubuntu Desktop in your organisation?

Newsletter signup

Select topics you’re
interested in

In submitting this form, I confirm that I have read and agree to Canonical’s Privacy Notice and Privacy Policy.

Related posts

Design and Web team summary – 27 May 2020

The web team here at Canonical run two-week iterations. Here are some of the highlights of our completed work from this iteration. Meet the team My name is...

ZFS focus on Ubuntu 20.04 LTS: what’s new?

Ubuntu has supported ZFS as an option for some time. In 19.10, we introduced experimental support on the desktop. As explained, having a ZFS on root option on...

The New Ubuntu Server Guide

This article originally appeared on Joshua Powers’ blog With the release of Ubuntu 20.04 LTS (Focal Fossa) the Ubuntu Server Guide has received a major set of...