POST, PUT, idempotence, and self-identification

Posted on November 17, 2011 by

I realize that the title of this posting might be a bit off-putting for non-RESTafarians, but it’s about a topic that matters to anyone building an application that uses simple web standards for updates.

Background: how REST usually works

There are two HTTP methods that you can use for adding/updating information on a web site:

PUT
PUT, when you know the address (URL) of the thing you’re adding/updating.
POST
POST, when you don’t know the address of the thing you’re adding/updating (you’re implicitly asking the server to assign a URL for you).

Since PUT refers to a specific URL, people say that it’s idempotent — that is, if you repeat the same operation three times, it won’t create three separate resources on the server (I’ve simplified the HTTP headers a fair bit in these examples):

PUT http://example.org/greetings/resource01.xml

<greeting>Hello!</greeting>

PUT http://example.org/greetings/resource01.xml

<greeting>Hello!</greeting>

PUT http://example.org/greetings/resource01.xml

<greeting>Hello!</greeting>

The result is just one resource at http://example.org/greetings/resource01.xml.

POST, on the other hand, sends the request to one URL (or other address) that will place the resource at a different URL. If PUT is like placing the resource in a file yourself, POST is like handing it to a file clerk. As a result, POST is not guaranteed to be idempotent:

POST http://example.org/actions/add-greeting

<greeting>Hello!</greeting>

PUT http://example.org/actions/add-greeting

<greeting>Hello!</greeting>

PUT http://example.org/actions/add-greeting

<greeting>Hello!</greeting>

In this case, HTTP itself makes no guarantee that this three-time repetition won’t result in representations of three different resources with the same content, say http://example.org/greetings/resource01.xml, http://example.org/greetings/resource02.xml, and http://example.org/greetings/resource03.xml.

Self-identification

So that’s it, as far as HTTP goes, because HTTP is just a transport layer — with a few exceptions (like re-encoding text files), it doesn’t care whether you’re using it to send a picture, video, web page, or XML file. Because HTTP doesn’t know anything about your resource, it can’t make any guarantee about the idempotence of POST.

Let’s say, however, that you’re designing an XML-based web application where every resource has its own, internal unique identifier. To take a simple example, let’s use ISO 3166-1 alpha2 country codes, such as “US” for the United States or “DE” for Germany. Your XML files always contain the identifier in a place that the application can find it:

<country ident="CN">
  <name>China</name>
  <population>1338299500</population>
  <land-area unit="km2">9640011</land-area>
</country>

If your web application has a fixed URL scheme based on the ISO 3166 code, it will create this resource at (say) http://example.org/countries/CN.xml when you POST it. If you POST it a second time, it won’t create a second copy of the same information, because it has a fixed mapping between the code “CN” and the URL, and knows that this is a new representation of the same resource. In other words, while HTTP itself can’t guarantee idempotence for a POST operation, the web application can.

So now what?

So for a web application like this, does it make sense to have both POST and PUT operations? RDF fans would say clearly “yes”, since they would consider the URL — not the ISO 3166 code — to be the resource’s identifier, and I feel strong sympathy with that approach (URLs make great global identifiers for things). There is, however, a down-side. Let’s say I’m reading data from a legacy system that doesn’t know or care about URLs. Does it make sense to force that system to duplicate my URL-construction algorithm, and know to PUT its updates for China to http://example.org/countries/CN.xml, for Canada to http://example.org/countries/CA.xml, etc.? Or does it make sense just to let the client POST to a single URL, knowing that the application will enforce idempotence based on the domain-unique identifier in the XML file?

I actually haven’t made up my mind on this point yet. As I wrote in a 2005 posting, real-world REST applications generally stick to GET for retrieval and POST for creation/updating/deletion, however inelegant that approach may be. I hope things have improved in the last 6½ years, but I’m not confident that they have.

What do you think? Is allowing both POST and PUT unnecessarily complicating the interface for the sake of RESTful purity, or is using only POST for creation and updating breaking the implied contract of HTTP and risking confusion among users?

Posted in General | 5 Comments

MySQL wrapper script: variables from the command line

Posted on June 21, 2011 by

There’s a lot I like about PostgreSQL, and I’ll probably switch eventually, but for now, I’m still more comfortable with MySQL, especially its ability to update huge amounts of existing data in-place very quickly.

I do have some technical gripes with MySQL, though:

  • InnoDB doesn’t support fulltext indexes, so I’ve forced to give up referential integrity for any table that needs fulltext searching.
  • The mysql(1) command-line tool doesn’t allow setting user variables on the command line when invoking a script.

The first is too big a problem for me to take on right now, but the second looked like it just needed a bit of perl magic, so I wrote a wrapper script:

runmysql

This script scans each command-line option for a string beginning with “@”, and pulls it out of the arguments before passing the rest on to mysql. Any option in the format @name=value will be converted to a SQL variable declaration and passed to mysql; any other option gets passed through unmodified.

Example

Let’s say that you have a SQL script named getstuff.sql:

use mydatabase;

select * from Stuff where type=@type;

You can invoke this script like this:

runmysql @type=foo -X < getstuff.sql

The script will run the command “mysql -X” (the “-X” means XML output — it’s just there to demonstrate how other options get passed through), and prepend a declaration to the SQL script, so that MySQL actually sees this:

set @type = 'foo';
use mydatabase;

select * from Stuff where type=@type;

That’s it, really. Of course, it’s not that useful in a trivial script like this, but for scripts that load data from many different files (for example), it makes life much simpler.

Posted in General | Tagged , , , | Leave a comment

Teachers, students, and online behaviour

Posted on April 12, 2011 by

This passage recently appeared in a CBC story entitled “Ont. teachers advised not to tweet with students“:

The Ontario College of Teachers says teachers should avoid connecting with their students on Facebook or Twitter.

They are also told to avoid contacting them on LinkedIn, Flickr, YouTube and MySpace.

It’s a great idea to offer guidelines for how teachers communicate with vulnerable pre-College/University students; it’s too bad that the College revealed little other than its own ignorance in the way it wrote them. In a sense, that’s our fault in IT, because we call a whole bunch of unrelated things “social media”, so we can hardly expect teacher-bureaucrats to distinguish between broadcasting publicly over Twitter and exchanging secrets with a select group of friends over Facebook.

Not where you are, but what you’re doing

So, let’s forget about red herrings like online vs. offline, social media, etc., and instead, propose two simple questions a teacher has to answer:

  1. Is it all taking place in the public view (can the student’s parents, my colleagues, my principal, etc. follow the exchange)?

  2. Is it related to my role as a teacher (am I using this information to help teach, or do I have non-professional motives)?

If the answer to both these questions is “yes”, then the communication is probably appropriate, whether it’s through the spoken word, paper, Twitter, Facebook, or any channel.

Good and bad communication

Here are some examples of how the questions play out:

  • It is OK, and often desirable, for a student to read public school-related information posted by a teacher, whether it’s on a bulletin board in the hallway, a Twitter feed, a blog, or a posting in a Facebook group. “Public” means that the parents and principal can read it too. It’s a good way for a teacher to communicate with the students (deadlines, assignments, study hints, etc.) and encourage discussion.

  • It is usually OK for a teacher to read public information posted by a student, whether it’s a letter to the editor in a newspaper, a Twitter feed, or a posting in a Facebook group. The key is whether the information is related to the teacher’s role — for example, reading a feed of public political. literary or math tweets from a student is being a good teacher; scouring tweets (or any other public information, such as Google Streetview or the phone book) for personal information is stalking.

  • It is probably not OK for a teacher to open private communication channels with a student, where the communication can’t be seen by other teachers, parents, etc. This would include passing private notes in the student’s locker, exchanging email (unless the parent or another teacher is CC’d on it), becoming Facebook friends, chatting privately using Jabber or MSN, or exchanging SMS text messages. This is a bad idea even if the communication is meant to be strictly professional, because it leaves students open to manipulation, and teachers open to accusation.

Sound reasonable? It’s not the platform (Twitter, Facebook, etc.) but the activity that matters. As long as you keep everything out in plain view, talking and exchanging information is a good thing, online or otherwise.

Posted in General | Tagged , | 3 Comments

The last time I voted Conservative …

Posted on April 10, 2011 by

[Blushing update: a reliable source informs me that I misremembered the question I asked 23 years ago — updated below (different, but closely-related issue).]

… (Progressive Conservative at the time) was the 1988 federal election. I’d grown up in Kingston as a Red Tory (socially left, fiscally right), working on campaigns for Flora Macdonald and Keith Norton before I was old enough to vote, but by age 24, living in downtown Toronto, I’d drifted around the political spectrum and generally voted NDP or Liberal, with an occasional crush on Green.

In particular, I disliked our current Progressive Conservative Prime Minister, Brian Mulroney, because he’d pushed out Joe Clark, the last of the great Red Tories. Still, on principal, I always tried to learn something about the local candidates, because in the Westminster System that we follow in Canada, no one votes for a party or leader — we elect local representatives for each Riding, and the government is whoever can convince enough MPs (of any party) to support them.

A direct question

So one day, I was walking down Yonge Street and I ran into the local Progressive Conservative candidate, David MacDonald. MacDonald was an ordained United Church minister at a time when the church was torn apart over the issue of gay ordination; he was also running in a Riding that had both a large gay community (we called it just “Church and Wellesley” at the time — the nickname Gay Village wasn’t current yet, at least not in the hetero community) and a large, often homophobic Catholic immigrant community; finally, he was running for a party that wasn’t exactly in the vanguard of the gay rights movement.

I decided, then, as a cocky 24-year-old, to give him a chance to prove himself with a simple Shibboleth question that would piss off half his church, half his potential supporters, and some faction of his party no matter how he answered it:

Do you support same-sex marriage?
Do you support ordaining gays and lesbians as ministers?

A direct answer

The normal political instinct would be to waffle and try to come up with a wimpy compromise answer (“I support the aspirations, but it’s a difficult question and we need time …” kind of BS). The second instinct would be to be bluntly opposed, in line with the majority of Canadians 23 years ago. But MacDonald gave his answer loudly and without hesitation:

Yes, I support it fully.

That’s the kind of courage I love to see from a politician in any party. When I combined that direct and honest answer (which I happened to agree with) with the fact he had real political experience as a cabinet minister under Joe Clark, I decided that even a 24-year-old Toronto grad student could, I guess, vote Tory just once.

Aftermath

MacDonald won the election and went on to become a cabinet minister during Mulroney’s second term, acting as a moderate voice from inside the cabinet room, where he could do the most good. I had mixed feelings about Mulroney’s government, but I was always happy to know that MacDonald was there to speak for me.

Today, I am still a fan of voting local first, and national second. We need good people in Parliament, willing to represent their constituents. I do care about what party those people belong to, but it’s a secondary consideration. Sure, most MPs vote with their parties, but first, there has to be a consensus in cabinet and in caucus, and the more moderate, reasonable voices you elect there, the less likely that Layton can nationalize Canadian industry, Iggy can start a new war in the Middle East, Harper can ban abortion, or whatever nightmare scenario you happen to have in mind.

Posted in General | 2 Comments

How can I make money from writing sonnets?

Posted on March 1, 2011 by

William ShakespeareI’ve written a sonnet, and I think it’s very good. Now I’d like to get paid for it so that I can quit my job and live off the revenue. Any suggestions?

Perhaps you’ve asked this question yourself: just substitute “iPhone app”, “blog”, “web site”, “Twitter mashup”, etc. for “sonnet”. You have a right to be proud that your app/blog/site is wonderful, but why won’t anyone pay you for it?

A pat on the head for young Billy

In school, when you did good work, you got rewarded, because school is a system designed to encourage you to do your best (at least, a good school is). In a job, when you do good work, you may get noticed and rewarded, depending on how much value your manager puts on continuing to treat you like a school student.

Outside of these artificially-constructed systems, however, nobody beyond friends and family gives a shit that you did something good — as Don Draper said on Mad Men “There is no system. The universe is indifferent.”

Don’t go past the fence!

Still, many brave or foolish people venture outside the school/employment sandbox every year, and the moment they go out the gate, they are hit with two shocks at once:

  1. Nobody wants to give them money.
  2. Nobody wants to give them praise.

If you’re going to have a chance of success, you recover from this shock, forget about chasing after praise (it’s mostly worthless), grit your teeth, and learn through trial and error how to provide enough value to other people that they’re willing to give you money in exchange.

That’s hard work — very hard work — and the heart of it is not simply promoting a product or service, but spending months and years developing real business relationships with people, and even more importantly, learning (and caring about) what those people — not you — want and need.

Shortcuts

Many people aren’t ready to face that kind of a shock, though, and at that point, they’re an easy mark for anything that promises an alternative to the hard (but necessary) work of actually dealing with people and caring about what they want. Every year, there seems to be at least one fad that promises a shortcut:

  • You can run Google ads (or others) on your blog or web site, and Google will take care of finding and billing the customers, then will send you a monthly cheque.
  • You can write a Twitter mashup or Facebook app, where Twitter or Facebook members will take care of the hard work of marketing for you, and you can then get enough visitors to cash in on those adds (or sell subscriptions through PayPal, etc.).
  • You can post a video to YouTube, and get a share of ad revenue if it goes viral (insert buzzwords like “social media” where appropriate).
  • You can write an iPhone or Android mobile app, and Apple or Google will find and bill your customers for you.

All of these promise money (however little they actually deliver), but more importantly, they promise that you can stay in a safe, non-Don-Draper-esque world like the one you remember from school or work: the world where you can get an A+ for doing a very good job on your essay, or a bonus for working extra hard on the ACME account. Effectively, Google, Apple, or some other organization becomes your teacher/parent/boss, dishing out rewards in the form of money and/or pageviews.

That’s a world most people understand; it’s a system where most people feel safe; but it’s no way to sell a sonnet.

Shakespeare

To be honest, we don’t know if Shakespeare ever made money from his sonnets, but he did make a living from his plays … well, not so much from his plays, as from putting them on. It turns out that around the end of the 16th and beginning of the 17th century, Londoners sometimes wanted to get out of the house and do something, but you can go to only so many bear-baitings and Morris dances before a certain ennui sets in.

Shakespeare didn’t just sit in a lonely garret writing plays to for someone else to put on (the 1600 equivalent of writing mobile apps for the iTunes store): as an actor, as part-owner of the Lord Chamberlain’s Men (later King’s Men), and as an investor in the Globe Theatre, Shakespeare threw himself directly into the hard work of forming relationships and dealing with his customers. He made enough money to go back to his home town and buy the second-largest house — a mansion, really — and lord it over everyone he grew up with.

After all that work, perhaps he wrote his sonnets just for fun.

(This post was inspired partly by my life partner, Bonnie Robinson, and the long hours she’s putting into Sweet Tarts Takeaway.)

Posted in General | Tagged | 6 Comments

The zero-home experiment

Posted on February 15, 2011 by

Yesterday a new web series launched, Sweet Tarts Takeaway. I helped to set up its web site.

Too many clicks

Copyright © 2011 by YOW! Productions. Used with permission.

After making one mockup of the home page, we realized that it would take three clicks for a user simply to start watching this week’s episode:

  1. Click on “episodes” on the home page.
  2. Click on the latest episode at the top of the episodes page.
  3. Click on the video to start it playing.

For a web series, where the main point is to let people watch a new episode every week, this made no sense. We thought about a couple of options:

  • We could embed a direct link to the current episode on the home page.
  • We could embed the current episode itself on the home page.

Broken links and bookmarks

The first option still required two clicks. The second option had more subtle problems: let’s say that I went to sweettartstakeaway.com, watched the current episode, liked it, and sent you the link. Two days later, you open your email, click on the link, and see an entirely different episode, because the current episode has changed.

This isn’t a merely abstract problem. Blogging sites (like those based on WordPress) typically put copies of the newest content on the home page. I have followed hundreds of links to blog postings over the past few years, only to find that someone had linked to the blog’s home page instead of the specific blog posting, and the posting I wanted had long fallen off the home page. Even search engine results do that sometimes.

The zero option

The solution we finally came up with seems simple only in retrospect — throw out the home page.

When you direct your browser to sweettartstakeaway.com, instead of yawning at a promotional page telling you how wonderful the web site is, you get redirected automatically to the current episode page (on February 14, 2011, that was /episodes/01). Next Monday, you’ll be redirected to a different episode, and so on, for all 18 episodes. (We’re using a 307 redirect, for anyone who cares about HTTP status codes.)

Give them what they want, not what you want

With a web series, you know there’s one activity most visitors care about: watching this week’s episode. When you do know that, why not let them start there, instead of forcing them to crawl through an ego-driven “home page” telling them how wonderful you are?

When the series is finished, and we archive it, we may add a home page then to give historical context for the whole series, with links to episodes. For now, though, it will be interesting to see how our zero-home experiment works out.

Posted in General | Tagged , , | 1 Comment

Is fluid layout old-fashioned?

Posted on February 9, 2011 by

No one would mistake me for a graphic designer — I believe that no merely visual design can match the breathtaking beauty of a properly-normalized database schema — but still, I recently found myself helping someone design a simple theme for a Drupal web site.

The site owner was visually-focussed (poor woman, stuck with me as designer!), and had very specific ideas about where layout items and whitespace should go, so I decided that while I have issues with fixed-width layouts like the [in]famous 960 grid, I shouldn’t make her suffer for my beliefs: I designed a fixed-width, em-based layout for her.

Two weeks later, she got in touch and said “I’m really sorry, but …”. She had been looking at the web site on a laptop up to that point, but the first time she opened it on her big monitor at work, it looked like garbage, a tiny strip surrounded by huge bands of whitespace.

Window size: an obsolete problem?

I mention this story because I just read a web page that suggests that fluid layouts — web-page layouts that adjust themselves to fit the width of the browser window — are passé:

With the advancement in displays and the cheap prices of large 22″ and 24″ displays fluid layouts have quickly become more a hinderance than a help because you don’t run your browser full screen anymore. You generally run your browser alongside your mail client, your IM client, iTunes, etc. You find a common size for your browser that accommodates most sites, and you leave it at that width.

Well, apparently not. At least, not in this case. The argument seems to be that users will conform to the needs of designers, by carefully resizing their browser windows to make the designs look good, but the web site owner I’m working with wasn’t one of those people, and I suspect that not all her visitors will be, either.

Rogues’ gallery

The designer of the web site proclaiming the demise of fluid layout probably hoped his or her site would look like this:

This is what you’d see on a typical laptop screen, or on a window sized smaller on a big screen. It’s beautiful — I freely admit that I could never design such a nice-looking web site.

Now, here’s how the same site would look in a large browser window on a large monitor:

OK, even I (and probably my guinea pig) can design something nicer-looking than that. Damn those uncooperative users! Why do they forget to size their browser windows properly?

Now, let’s go down-market, and see how the same site might look in a browser window on a tiny netbook (where the user doesn’t have the option of choosing a bigger window size):

Not nearly as bad as the big screen, but awkward — I can sort-of see what the designer was trying to do, but I have to scroll around to get the whole effect: it’s like looking at the Mona Lisa in 5 cm² segments.

Fluid vs fixed-width is a trade-off

I’m not going to conclude by saying “fixed-width bad, fluid layout good”, or any such oversimplification, but instead, to suggest that the tradeoff that existed between the two ten years ago still exists today:

  • If you want a web site that’s a respectable B- in a wide range of screen sizes, use a fluid layout (it’s not an accident that that’s what popular sites like Google use).
  • If you want a web site that’s a spectacular A+ in one screen size, and a dismal D- in all others, use a fixed-width layout.

If you need to demonstrate that you are capable of stunning visual beauty, if you know that most of your users will be viewing the site on a certain type of device, or if you just want to stoke your ego by winning design awards (and you know the resolution the selection jury will be using to view sites), then fixed-width can be a great choice. For me, however, usability trumps style, so I’ll stick with fluid-width when I can.

And yes, for the record, you’re reading this blog (as of 2011-02-09) in a fixed-width layout. I just use the default theme on wordpress.com with a custom picture in the banner, because I’m not really all that visually-focussed. Maybe some day I’ll get around to seeing if I choose a WordPress theme with a fluid layout here, too.

Posted in General | Tagged , , | 4 Comments