Content providers vs their audience, in two acts

or … It’s always been a struggle for control

Act I

Time: 1980s
Place: TV

I want you to sit still and watch these commercials.
When the commercials come on, I get up to go to the kitchen or washroom.
I’m making the commercials louder so that you’ll still hear them.
I just bought a new remote with a mute button.
I’m grouping commercials more randomly, so that you won’t know how long you can leave the room without missing something
Have I shown you my new VCR, with pause and fast-forward?
If you fast-forward through commercials, there will be no more TV in 10 years. You’ll be sorry then.

Act II

Time: 2010s
Place: the Web

You must always start on our home page and navigate from there. We want you to see the photo of our CEO in a hard hat patronisingly glad-handing workers before you get the information you want.
I’m bookmarking the pages I find useful and deep-linking to them. And your CEO is a twit.
Ha! We’re blocking deep-linking.
Look! Google’s not indexing most of your site any more.
I want you to view the site exactly the way I intend, with 9-pt fonts, grey on a purple background, and 800 pixels wide.
That looks horrible on my screen, and worse on my smartphone, and besides, my aunt can’t even read it. I’m going to my browser accessibility settings (or maybe I’ll just use Pocket).
How dare Google News show headlines from our site? We’re sending them a court order to stop.
Funny, I’m not seeing headlines from your site any more. I guess I’ll follow the links to stories on other sites.
Our ads from ad networks and hidden tracking cookies allow us to monetize you by collecting personal data and selling it to marketing databases.
If you block all our ads and tracking cookies, there will be no more web sites in 10 years. You’ll be sorry then.

(To be continued …)

Posted in General | Leave a comment

Web unusability

FedEx Canada refund form
When might a company decide that web usability is a bad thing?

If I were a conspiracy theorist, I’d suggest that companies deliberately sabotage forms for tasks like refund requests, to discourage user conversion. Practically speaking, however, I think it’s more a matter of just not bothering to invest in them past the alpha stage. Consider FedEx Canada’s refund form:

Every field on this form is required, even though many of them won’t apply (e.g. you might not have your own invoice number associated with the shipment, and you almost certainly won’t have a fax number), and some are actually contradictory (you will have either a FedEx Express Air waybill number or a FedEx Ground Tracking ID, but not both).

On a higher level, nothing in the form makes it clear whether you’re filling in your address or the address to which you were sending the package. Flip a coin!

Fortunately, the form also doesn’t bother with data-type validation, so I was able to put “n/a” in all the fields that didn’t apply (1995 called and took away my fax machine), and I was able to guess that they wanted my address, based on the presence of a picklist for Canadian provinces.

If you’re an enterprising soul, you might consider setting up shop as a Web Unusability consultant, to help companies find ways to keep customers from filling out unwelcome forms.

Posted in Design, Web | Tagged , | Leave a comment

Update: running Emacs in Android L (Lollipop)

Emacs logo. In February 2014, I posted instructions for installing Gnu Emacs in older versions of Android. These instructions no longer work in Android L (Lollipop), because of a new requirement that all executables be compiled with a position-independent (PIE) flag. I have managed to create a new binary from Michał Zieliński’s admittedly now out-of-date patched Emacs version, and have made it available on GitHub.

Installation instructions

Since TerminalIDE also fails to run under Lollipop (lacking PIE), the instructions for installation are now somewhat different.

  1. Install some kind of a terminal emulator (JuiceSSH currently crashes, but others seem to work) in your Android. If you’re not rooted, then you must choose a terminal emulator that has its own home directory under /data/, where you can make a file executable and run it (you cannot normally make a file executable under /sdcard/).

  2. Create a new directory /sdcard/emacs/ to hold your non-executable files, such as eLISP files (you may choose a different location, but you’ll need to set some extra environment variables so that Emacs can find your files; that’s outside the scope of this posting).

  3. ₣rom my GitHub repo, download emacs.bin, copy it somewhere not on your /sdcard/ (your terminal emulator’s “home” directory if not rooted, or somewhere like /data/local/bin/ if rooted) and make it executable:

    chmod 755 emacs.bin
  4. From my GitHub repo, download etc.tlzma, terminfo.tlzma, and lisp.tlzma, and unpack them all into your /sdcard/emacs/ directory — you may need BusyBox installed to get the unlzma and tar utilities required for unpacking, e.g.

    unlzma -c lisp.tlzma | tar xvf -
  5. Somewhere on your executable path (perhaps /system/xbin/), create a new shell script called “emacs” containing a command like this:

    HOME=/sdcard TERMINFO=/sdcard/emacs/terminfo /data/local/bin/emacs.bin "$*"

    If your chosen terminal emulator already has TERMINFO installed, then don’t set it in the script. You may choose any home directory you want, but it must be writeable (Emacs will want to create a $HOME/.emacs.d/ directory to save settings).

  6. Make the new script executable (and make sure it’s on your execution $PATH):

    chmod 755 emacs

Known issues

  • If your terminal type is unrecognised, you might need to add additional terminal types to emacs/terminfo; conversely, if your terminal emulator is already using a TERMINFO database somewhere, then don’t set the TERMINFO environment variable in your shell script.

  • Crashes in a local shell under JuiceSSH — don’t know why yet (use a different terminal emulator). Update: only when the font is too large — seems related to screen width.

  • Dates on *.elc files are earlier than dates on *.el files after running Michał’s build scripts; you should probably fix that with something like this:

    find /sdcard/emacs/lisp -name '*.elc' -print0 | xargs -0 touch
  • Starts with the message “WARNING: linker: /data/local/bin/emacs.bin has text relocations. This is wasting memory and prevents security hardening. Please fix.” — this seems to be a common problem in Android L, and I have not yet found the right combination of GCC flags to fix it.

Help wanted

Any volunteers to package all this up neatly, perhaps with some terminal emulator, for distribution in the Google Play Store?

Posted in General | 8 Comments

Technology design, in two comics

I refer to these two comics constantly when I’m doing technology architecture and design work.


Far Side comic: wings stay on / wings fall off

(From The Far Side, 1980s IIRC.)


xkcd comic: University Website


Posted in General | Leave a comment

A different kind of data standard

This year, the UN gave me the chance to bring people together to work on data standards for humanitarian crises (like the Ebola or Syria crisis). We put together a working group from multiple humanitarian agencies and NGOs and got to work in February.  The result is the alpha version of the Humanitarian Exchange Language (HXL, pronounced /HEX-el/), a very different kind of data standard.

What’s wrong with data standards these days?

Unlike most data standards, HXL is cooperative rather than competitive. A competitive standard typically considers the way you currently work to be a problem, and starts by presenting you with a list of demands:

  • Switch to a different data format (and acquire and learn new software tools).
  • Change the information you share (and the way your organisation collects and uses that information).
  • Abandon what is valuable and unique about your organisation’s data (and conform to the common denominator).

For HXL, we reversed the process and started by asking humanitarian responders how they’re actually working right now, then thought out how we could build a cooperative standard to enhance what they already do.


Given the conditions under which humanitarian responders work in the field (iffy connectivity, time pressure, lots to do besides putting together data reports), we realised that an XML-, JSON-, or RDF-based standard wasn’t going to work.

The one data tool people already know is the infamous spreadsheet program, so HXL would have to work with spreadsheets.  But it also had to be able to accommodate local naming conventions (e.g. we couldn’t force everyone to use “ADM1” as a header for what is a province in Guinea, a departamento in Colombia, or a governorate in Syria). So in the end, we decided to come up to add a row of hashtags below the human-readable headers to signal the common meaning of the columns and the start of the actual data. It looks a bit like this:

Location name Location code People affected
#loc #loc_id #aff_num
Town A 01000001 2000
Town B 01000002 750
Town C 01000003 1920

The tagging conventions are slightly more-sophisticated than that, also including special support for repeated fields, multiple languages, and compact-disaggregated data (e.g. time-series data).

HXL in action

While HXL is still in the alpha stage, the Standby Task Force is already using it as part of the international Ebola response, and we’re running informal interoperability trials with the International Aid Transparency initiative, with planned more-formal trials with UNHCR and IOM.

We also have an interactive HXL showcase demo site, and a collection of public-domain HXL libraries available on GitHub. More news soon.


Thanks to the Humanitarian Data Exchange project (managed by Sarah Telford) at the UN’s Office for the Coordination of Humanitarian Affairs for giving me the opportunity and support to do this kind of work, to the Humanitarian Innovation Fund for backing it financially, to the HXL Working Group for coming together to figure this stuff out, and especially to CJ Hendrix and Carsten Keßler for their excellent work on an earlier incarnation of HXL and for their ongoing support.

Posted in Aid, Data | Tagged | Leave a comment

Community data, Ebola, and the UN

The humanitarian community’s response to the Ebola crisis is more transparent than many responses in the past, and that transparency includes data. The core datasets from the UN and humanitarian partners are starting to appear here:

If you do anything potentially helpful with the data (visualisations, combining with other data, etc.) please let the UN’s HDX project know, so that they can share it around.

Posted in Aid, Data, Transparency | Leave a comment

RCIM — Right-Click-Incognito-Mode

If you’re a technically-oriented person (like most of my readers), you already know this magic spell, but I suspect it’s not obvious to most of the world: RCIMright-click-incognito-mode (or LPIMlong-press-incognito-mode in mobile) lets you open a link without persistent cookies. That does a couple of pretty-magical things:

  • It blocks tracking cookies (so Facebook’s database, for example, doesn’t know that you’re reading a piece on Unicorn herpes on another site if you didn’t follow the link from FB), and
  • It circumvents many (most?) quota-based paywalls (e.g. newspapers and magazines).

I’ve been reluctant to share links from sites like because of their quota wall (if I share a link, I want everyone to be able to read it). Browser extensions like AdBlock+ and Disconnect can solve at least the first problem, but I’m looking for something easy-enough for everyone to use (or at least, everyone using a browser with a incognito/anonymous-browsing mode, which describes most of us in 2014).

So #RCIM and enjoy a web with fewer barriers.

Posted in Web | Leave a comment