I'm not a big fan. They're environmentally wasteful, inaccessible, and a bit crap for remote workers. But some people love them, so who am I to judge?

Recently, I visited a fairly large company who are making the painful transition from providing mega-software into to being a nimble, digital supplier. Their walls were plentifully decorated with multi-coloured Post-it® notes.

Decorated being the operative word. A quick glance at them showed titles like "To-Do 2018" and "Easter Fire-Break" and "FAO Jerry".

"Who is Jerry?" I asked.

"Oh... I think he left a few months back," came the reply.

Now, not all of the Kanban Boards were outdated - some were obviously in use and had teams performing their daily rituals in front of them. But the majority seemed abandoned.

Perhaps abandoned is too strong a word. They were like cave paintings. Evidence of the hunt, sure, but now decorations to be marvelled at. A way to indoctrinate new members of the tribe.

Perhaps the Post-it® notes were good-luck charms. A steady stream of investors would have walked through the hallways and seen "evidence" of an advanced civilisation.

Perhaps the Post-it® notes were to ward off evil spirits. A cranky manager would have been mollified that his team were truly agile, and then left them alone in peace to carry on their waterfall development.

What I'm trying to say is this. You can't put up wallpaper and pretend it is structural transformation.

My first attempt used the proximity sensor. However, on all the Android phones I tried the sensor's accuracy was binary - it could sense if something was close by, but not say how close.

So, what else could I use to detect how near or far a hand was from the screen? I decided to co-opt the Light Sensor. This is normally used to automatically adjust the brightness of the screen - making it easier to see in strong light.

When the light sensor is uncovered, the total lux (that's the measure of light) may be 100. As a hand moves closer to it, that value will dip until it reaches 0 (or, on my phone, 4).

We can then represent that light value as a sound - essentially transforming lx into Hz!

This is what is sounds like

Beautiful, I'm sure you agree! You can hear an interview where I discuss this app with the BBC's Jamillah Knowles on the Outriders Podcast (22m 50s in).

If you want to have a play with it, the Optical Theremin Demo is in the Google App Store. Do note, it was coded in a couple of sleep deprived hours, crashes when you exit, and can produce "music" which scares children and animals. You have been warned!

Use The Source, Luke!

I've included the full source below, but I'd like to pick out two points which may be of interest.

Getting The Lux Value

Firstly, we need to register a listener for the light sensor.

@Override public void onCreate(Bundle savedInstanceState) {
   super.onCreate(savedInstanceState);
   mSensorManager = (SensorManager) getSystemService(SENSOR_SERVICE);
   mLightSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_LIGHT);
   mSensorManager.registerListener(this, mLightSensor, SensorManager.SENSOR_DELAY_FASTEST);
}

Every time the light sensor changes, this method will be called. It takes the light value and performs a simple mathematical transformation on it (adds 10, multiplies by 5). I found that this gave the most pleasing sound - but you can adjust it to your tastes

@Override public void onSensorChanged(SensorEvent event){
   if (event.sensor.getType()==Sensor.TYPE_LIGHT){
       mLux = event.values[0];
       freqOfTone = (mLux +10) * 5;
   }
}

Cum on Feel the Noize

So, how do we get Android to generate a tone? I faffed around with this audio generating code from StackOverflow until I could successfully generate a tone.

Essentially, this creates a WAV of a tone and gets it ready to play.

However, this sounded rather boring, so I added some reverb.

audioTrack.attachAuxEffect(EnvironmentalReverb.PARAM_DECAY_TIME);

And that's it!

Download the Optical Theremin Demo App - or use the source to create something much more melodious.

Full Source

package mobi.shkspr.android.theremin;

import java.util.Random;

import android.app.Activity;
import android.hardware.Sensor;
import android.hardware.SensorEvent;
import android.hardware.SensorEventListener;
import android.hardware.SensorManager;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioTrack;
import android.media.audiofx.EnvironmentalReverb;
import android.os.Bundle;
import android.os.Handler;
import android.util.Log;
import android.widget.TextView;

    public class TheraminActivity
        extends Activity
        implements SensorEventListener{
        // originally from http://marblemice.blogspot.com/2010/04/generate-and-play-tone-in-android.html
        // and modified by Steve Pomeroy steve@staticfree.info

        private final int duration = 5; // seconds
        private final int sampleRate = 8000;
        private final int numSamples = duration * sampleRate;
        private final double sample[] = new double[numSamples];
        private double freqOfTone = 440; // hz

        private final byte generatedSnd[] = new byte[2 * numSamples];

        private SensorManager mSensorManager;
        private Sensor mLightSensor;
        private float mLux = 0.0f;
        private String tLux = "Lux is ";

        public AudioTrack audioTrack;

        Handler handler = new Handler();

        @Override public void onCreate(Bundle savedInstanceState) {

            super.onCreate(savedInstanceState);


            mSensorManager = (SensorManager) getSystemService(SENSOR_SERVICE);
            mLightSensor = mSensorManager.getDefaultSensor(Sensor.TYPE_LIGHT);

            mSensorManager.registerListener(this, mLightSensor, SensorManager.SENSOR_DELAY_FASTEST);

        }

        @Override public void onSensorChanged(SensorEvent event){
            if (event.sensor.getType()==Sensor.TYPE_LIGHT){
                mLux = event.values[0];
                String luxStr = String.valueOf(mLux);
                TextView tv = new TextView(this);
                tv.setText(tLux);
                setContentView(tv);
                Random r = new Random();
                freqOfTone = (mLux +10) * 5;
            }

        }
        @Override protected void onResume() {
            super.onResume();
            final Thread thread = new Thread(new Runnable() {
                public void run() {

                    for (int i = 0; i < 300; i++)   {
                        genTone();

                        audioTrack = new AudioTrack(
                                        AudioManager.STREAM_MUSIC,
                                        sampleRate,
                                        AudioFormat.CHANNEL_OUT_MONO,
                                        AudioFormat.ENCODING_PCM_16BIT,
                                        numSamples,
                                        AudioTrack.MODE_STATIC);


                            try {

                                playSound();
                                Thread.sleep(505);
                            } catch (IllegalStateException e) {

                            } catch (InterruptedException e) {
                                // TODO Auto-generated catch block
                                e.printStackTrace();
                            }
                    }
                }
            });

            thread.start();
        }

        void genTone(){ // fill out the array
            tLux = "Frequency is " + freqOfTone;
            //Log.d("LUXTAG", "Lux value: " + tLux);

            for (int i = 0; i < numSamples; ++i) {
                sample[i] = Math.sin(2 * Math.PI * i /(sampleRate/freqOfTone));
            }

        // convert to 16 bit pcm sound array
        // assumes the sample buffer is normalised.
            int idx = 0; for (final double dVal : sample) {
                // scale to maximum amplitude
                final short val = (short) ((dVal * 32767)); // in 16 bit wav PCM, first byte is the low order byte
                generatedSnd[idx++] = (byte) (val & 0x00ff);
                generatedSnd[idx++] = (byte) ((val & 0xff00) >>> 8);

            }
        }

        void playSound(){
            genTone();
            try {                   audioTrack.attachAuxEffect(EnvironmentalReverb.PARAM_DECAY_TIME);
                audioTrack.write(generatedSnd, 0, generatedSnd.length); audioTrack.play();
            } catch (IllegalStateException e) {
                audioTrack.release();
            }
        }

        @Override
        public void onAccuracyChanged(Sensor sensor, int accuracy) {
            // TODO Auto-generated method stub
        }

        @Override
        public void onPause() {
            super.onPause();
            audioTrack.stop();
            audioTrack.flush();
            audioTrack.release();
        }

        @Override
        public void onStop() {
            super.onStop();
            audioTrack.stop();
            audioTrack.flush();
            audioTrack.release();
        }

        @Override
        public void onDestroy() {
            super.onDestroy();
            audioTrack.flush();
            audioTrack.stop();
            audioTrack.flush();
            audioTrack.release();
        }
    }

One of the very first things we're taught in HTML school is that we should separate content and style.

<span font="comic sans" colour="red">This is wrong!</span>

Instead, we should be doing

<span class="stylish">This is corrent!</span>

Yet, the very next thing we're taught is

<img src="example.jpg" height="120" width="90" />

Well hang on a second! We've mixed up content (example.jpg) with presentation (the dimensions of the image). The image will almost certainly have to be resized based on the screen size accessing the site. Which means all manner of crufty JavaScript and CSS hacks to get it to display perfectly.

The Right Way

Here is how I think the image tag should work.

<img src="example" class="icon" />

The first thing to note is that the image shouldn't have a file extension. As I've set out in a comment on Bruce Lawson's blog, the server should be looking at the HTTP accept headers to see what image type to serve up. If the device is capable of displaying SVG - that's what should be sent. If the device is too old to support PNG - the image should be served up as JPG (or whatever format the device accepts).

Again, the content of the image should be separated from the presentation (i.e. the file format).

Secondly, we drop the height and the width from the img tag. In the olden days, they were needed to stop the page from dramatically reflowing as images loaded. That's still a valid concern today, but the challenge is that we don't know what physical size the image will have until it is requested.

"So what?" I hear you cry "We can already do this in CSS. Images can have their dimensions set by absolute pixel size and / or relative size."

Indeed, you are correct. But, the HTML5 spec currently lists height and width as attributes which may be used. This, I believe, acts to tempt the unwary developer into using them. They should be as obsolete as "align" and "border".

Ideally, the logic should be on the server-side. Your CSS shouldn't be asking the device for its own properties, your server should be dynamically generating CSS which suits the User-Agent. The server should be adapting images on the fly (and cacheing them) depending on the resolution of the devices.

We should be writing ridiculously simple HTML5.

As I've said before

Computers are there to do the hard work for us. We shouldn’t be writing extra markup in every single new document.
Get the silicon slaves to do it all.