A different kind of data standard

This year, the UN gave me the chance to bring people together to work on data standards for humanitarian crises (like the Ebola or Syria crisis). We put together a working group from multiple humanitarian agencies and NGOs and got to work in February.  The result is the alpha version of the Humanitarian Exchange Language (HXL, pronounced /HEX-el/), a very different kind of data standard.

What’s wrong with data standards these days?

Unlike most data standards, HXL is cooperative rather than competitive. A competitive standard typically considers the way you currently work to be a problem, and starts by presenting you with a list of demands:

  • Switch to a different data format (and acquire and learn new software tools).
  • Change the information you share (and the way your organisation collects and uses that information).
  • Abandon what is valuable and unique about your organisation’s data (and conform to the common denominator).

For HXL, we reversed the process and started by asking humanitarian responders how they’re actually working right now, then thought out how we could build a cooperative standard to enhance what they already do.

Not JSON or XML

Given the conditions under which humanitarian responders work in the field (iffy connectivity, time pressure, lots to do besides putting together data reports), we realised that an XML-, JSON-, or RDF-based standard wasn’t going to work.

The one data tool people already know is the infamous spreadsheet program, so HXL would have to work with spreadsheets.  But it also had to be able to accommodate local naming conventions (e.g. we couldn’t force everyone to use “ADM1″ as a header for what is a province in Guinea, a departamento in Colombia, or a governorate in Syria). So in the end, we decided to come up to add a row of hashtags below the human-readable headers to signal the common meaning of the columns and the start of the actual data. It looks a bit like this:

Location name Location code People affected
#loc #loc_id #aff_num
Town A 01000001 2000
Town B 01000002 750
Town C 01000003 1920

The tagging conventions are slightly more-sophisticated than that, also including special support for repeated fields, multiple languages, and compact-disaggregated data (e.g. time-series data).

HXL in action

While HXL is still in the alpha stage, the Standby Task Force is already using it as part of the international Ebola response, and we’re running informal interoperability trials with the International Aid Transparency initiative, with planned more-formal trials with UNHCR and IOM.

We also have an interactive HXL showcase demo site, and a collection of public-domain HXL libraries available on GitHub. More news soon.

Credits

Thanks to the Humanitarian Data Exchange project (managed by Sarah Telford) at the UN’s Office for the Coordination of Humanitarian Affairs for giving me the opportunity and support to do this kind of work, to the Humanitarian Innovation Fund for backing it financially, to the HXL Working Group for coming together to figure this stuff out, and especially to CJ Hendrix and Carsten Keßler for their excellent work on an earlier incarnation of HXL and for their ongoing support.

Posted in Aid, Data | Tagged | Leave a comment

Community data, Ebola, and the UN

The humanitarian community’s response to the Ebola crisis is more transparent than many responses in the past, and that transparency includes data. The core datasets from the UN and humanitarian partners are starting to appear here:

https://data.hdx.rwlabs.org/ebola

If you do anything potentially helpful with the data (visualisations, combining with other data, etc.) please let the UN’s HDX project know, so that they can share it around.

Posted in Aid, Data, Transparency | Leave a comment

RCIM — Right-Click-Incognito-Mode

If you’re a technically-oriented person (like most of my readers), you already know this magic spell, but I suspect it’s not obvious to most of the world: RCIMright-click-incognito-mode (or LPIMlong-press-incognito-mode in mobile) lets you open a link without persistent cookies. That does a couple of pretty-magical things:

  • It blocks tracking cookies (so Facebook’s database, for example, doesn’t know that you’re reading a piece on Unicorn herpes on another site if you didn’t follow the link from FB), and
  • It circumvents many (most?) quota-based paywalls (e.g. newspapers and magazines).

I’ve been reluctant to share links from sites like economist.com because of their quota wall (if I share a link, I want everyone to be able to read it). Browser extensions like AdBlock+ and Disconnect can solve at least the first problem, but I’m looking for something easy-enough for everyone to use (or at least, everyone using a browser with a incognito/anonymous-browsing mode, which describes most of us in 2014).

So #RCIM and enjoy a web with fewer barriers.

Posted in Web | Leave a comment

Breadcrumbs: yes or no?

In web design, Breadcrumbs are those little navigational links you see across the top of some web pages, like

Home → Canada → Ontario → Ottawa

or

Media»Music»Classical»Beethoven

Good idea?

Breadcrumbs let you can see where you are in a web site’s information hierarchy, and let click to climb up to a more-abstract level. As of 2014-06-13, I use geographical breadcrumbs on OurAirports to let users climb up from looking at a specific airport to looking at the list of all airports in an administrative subdivision (e.g. province/state/governorate), country, or continent. I have haven’t changed the site design by the time you’re reading this article, you can try it out for Cairo International Airport.

I like breadcrumbs, because they reflect the way I think. I tend to organise information hierarchically, like a librarian, and breadcrumbs let me find my way around an e-commerce site (for example) without wasting time on frustrating searches. The article “Breadcrumb Navigation Examined: Best Practices & Examples” goes into great detail on different types of breadcrumbs and their benefits.

Bad idea?

While I love breadcrumbs, I notice that not many people use them on OurAirports. People almost never click up to see all airports in the same province or country; they really just want to look at a specific airport. Are my breadcrumbs just wasting screen space?

Back in 2008, Jared M. Spool posted an article “Design Cop-out #2: Breadcrumbs” suggesting that breadcrumbs represent a design failure:

The biggest problem is the lack of scent for the other areas of the site. If a user is in need of breadcrumbs because they are in the wrong part of the information tree, what they need most is good scent to the right part of the tree. However, the breadcrumbs only communicate the branch they’re on — not the branch they need to be on.

In 2011, Shanshan Ma wrote in “10 Ways Mobile Sites Are Different from Desktop Web Sites” that even if breadcrumbs belong on desktop web sites, they don’t belong on mobile ones:

However, breadcrumbs rarely appear on mobiles sites, and there is usually no necessity for them. Limited space is one reason breadcrumbs are uncommon on mobile sites. But the main factor is that the design of mobile sites prevents users from having to go too deep into a hierarchy to find what they are looking for.

What do you think?

Do breadcrumbs perform an important function in modern web design, or are they just a crutch for bad design?

Posted in Design, Uncategorized | 5 Comments

It’s never the end of history

Link: http://alistapart.com/article/fluidgrids/

Ten years ago, circa 2004, it felt like the web had found its rut and would never get out: XML and XHTML had failed to fix the browser-incompatibility mess, the horrid Internet Explorer had achieved almost total browser-market dominance, and web designers were focussing on animated pre-rolls and big screens. Even in 2009, when the rise of mobile was impossible to ignore, Ethan Marcotte still sounded like an Isaiah shouting from the wilderness when he pleaded with us to think differently:

Instead of exploring the benefits of flexible web design, we rely on a little white lie: “minimum screen resolution.” These three words contain a powerful magic, under the cover of which we churn out fixed-width layout after fixed-width layout, perhaps revisiting a design every few years to “bump up” the width once it’s judged safe enough to do so. “Minimum screen resolution” lets us design for a contrived subset of users who see our design as god and Photoshop intended.

It turns out that we weren’t at the end of web history, but just finishing the first act. After a decade-long intermission of stagnation, things started changing even faster than they had in the optimistic early days of the mid 1990s. IE has become so irrelevant that it’s barely necessary to test against it any more (unless your audience is big industry or government office workers using obsolescent desktop computers), HTML5 and CSS3 have exploded, the Javascript ecosystem has matured, the majority of page views happen on small-screen mobile devices, and those devices ensure that most users always have the latest version of their browsers (a vain hope in 2004 or 2009).

Five years after Marcotte’s article, any so-called web designer proposing to work with a fixed “minimum screen resolution” would … and should … be fired.

Except maybe in government and big industry.

Posted in Design, Mobile | Tagged , | Leave a comment

Hashtags and data standards

Link: Introducing HXL hashtags for humanitarian data

In my 16 years working with data standards, I’ve found that standards almost always ask for too much and end up getting little or nothing. If we asked for less, might we get more? Do data standards have to be tightly-managed and dirigiste, or could we learn from the success of hashtags and other simple, collaborative approaches?

The blog post linked above describes the approach we’re taking in the multi-agency Humanitarian Exchange Language (HXL) initiative to help improve data-sharing during humanitarian crises — please take a look and let us know what you think. You can also visit our HXL Showcase site to see interactive examples of how you can analyse and visualise examples of real humanitarian datasets with HXL tags added (the public-domain source code is available on GitHub).

Posted in General | Leave a comment

Simon St-Laurent originally shared this link on Facebook, and I left a comment. At his request, I’m reposting my unresearched, unedited rant here so that it can be found and held against me in the future (disclaimer: my OurAirports open-data web site uses as revenue to pay its hosting costs).

Article: Advertising is the Internet’s Original Sin (The Atlantic)

My original comment:

Good article. 

I’m going to propose another angle: what if people will pay a subscription fee for something they really want (like Netflix), but almost everything on the net is stuff people don’t actually want much (even sites that happen to have tens of millions of users)? 

That means that when you’re thinking up your new web startup, you’re actually asking the question “how do I get people to make a habit of using something they don’t actually want or need, and won’t mourn when it’s gone?” The best hope is that they’ll be bored enough to hang around (like watching a reality show because you’re too lazy to change the channel), but that can happen only as long as it’s not too much hassle and doesn’t cost anything up front. Then you fund your site by spying on those users and selling their info to advertisers.

The point is that such a site might have no other possible business model — if advertising weren’t possible, the site could never exist. So the problem wouldn’t be that we’re not finding ways to fund web sites other than subscription, but that we shouldn’t be launching so many web startups.

Link | Posted on by | Leave a comment