Roy T. Fielding, author of the REST thesis wrote an article recently: REST APIs must be hypertext-driven. I liked this article, it fit with an intuition I’ve had. Then he wrote an article explaining that he wouldn’t really explain the other articles because, I guess, he wanted a conversation with the specialists, and it seems like a kind of invitation to reinterpret his writing. So since others are doing it I figured I’d do it too.
I’d summarize his argument thus:
- Focus on media types, i.e., resource formats, i.e., document formats. The protocol will flow from these if they are well specified.
- URL structures are not a media type. They are some kind of server layout. You can’t hold them, you can’t pass them around, there is no notion of CRUD. Media types have all sorts of advantages that URL structures do not.
An example of a protocol based on a URL structure would be something like:
- Do GET /articles/ to get a JSON list of all the article ids, with a response like [1, 2, 3]
- Do a GET /articles/{id} to get the representation of a specific article.
JSON is a reasonable structure for a media type. It is not itself a fully explained type, because it’s just a container for data, just like XML. In this example you have a document, [1, 2, 3] which isn’t self-describing and just isn’t very useful. A more appropriate protocol would be:
- You start with a container, in our example /articles/. Do GET /articles/ to get a JSON document listing the URLs of all the articles. These URLs are relative to the container URL. You’ll get a response like ['./1', './2', './3'] (actually ['1', '2', '3'] would be fine too).
- Do GET {article-url} to get the article representation.
It’s a small difference. Heck, the communication could look identical in practice, but by putting URLs in the JSON document instead of this abstract "id" notion you’ve created a more flexible and self-describing system. You could probably give a name to that list of URLs, and then just talk about that name.
An example in Atompub is rel="edit". An Atom entry can look like:
<entry>...
<link rel="edit" href="/post/15" />
</entry>
Instead of the client just somehow knowing where to go to edit an entry, it’s made explicit. Thus you can move the entry around, while still pointing back to the canonical location to edit that entry.
There’s nothing really that complicated about this, the rule is really quite simple: link to other things, don’t just expect the client to know or guess where those other things are.
For a more concrete example of where this linking works well, OpenID uses <link rel="openid.server" href="..."> and <link rel="openid.delegate" href="...">, which allows you to add a little information to any HTML homepage so that the login can happen at a third location. If OpenID used something like looking at {homepage}/openid for a OpenID server then you couldn’t select whatever OpenID service you liked, or change services, or apply OpenID to hosted locations where you couldn’t install an OpenID server.
I’ll add my own little opinion in here: this is why the URL structure of applications doesn’t affect their RESTfulness, nor is URL structure all that important of a concern generally. Pretty URL structures are a nice thing to do, like indenting your code in a pleasant way, but it has nothing to do with your API, and if you can’t use a crappy URL structure with that same API then probably something is wrong with that API.
Automatically generated list of related posts:
- Atompub & OpenID One of the thinmgs I would like to do is...
> You start with a container, in our example /articles/. Do GET /articles/ to get a JSON document listing the URLs of all the articles. These URLs are relative to the container URL. You’ll get a response like [’./1′, ‘./2′, ‘./3′] (actually [’1′, ‘2′, ‘3′] would be fine too).
Mmmm, I don’t think the return type here would be JSON. Your requesting a list of URIs. Happily there’s a wonderful very parsable method for returning a list of URIs already. text/uri-list, which tells you that what is being returned IS a list of URIs and how to handle it. At that point I think it would be a lot closer to the mediatype based REST being described. For example, AtomPub documents aren’t sent as application/xml instead they have their own mimetype that lets client know what it is that they are reading.
For ad hoc APIs I suppose you could make up your own mimetype, like
application/x-myapplication+json(or+xmlor whatever). As to a mimetype for a list of URIs, this is a trivial example, and it’s quite likely that in a real case it would contain other information (e.g., last modified dates, titles, paging information, etc), so you’ll probably need something a little more extensible than just a list of URIs, i.e., JSON or XML.text/uri-listwon’t work for everything, but it does work very nicely for a list of URIs. Also I’ve found that very often a simple list of URIs which the client can then fetch in batches and in order is a very useful tool to have around when dealing with JSON and paging APIs anyway. And for those cases where you do want a bit more information then just a URI, perhaps a URI and last modified dates and titles RDF makes a great data model. Eg.Now, that in mind… OMG that’s a huge amount of XML going over the wire, and OMG Javascript is horrible in browsers when it comes to dealing with namespaces and RDF serialized in XML. So think what MIGHT make a great deal of sense from a pragmatic perspective might be application/rdf+json. We have N3, Turtle and RDFa already. I think some standards work around expressing RDF in JSON would make perfect sense, and be an excellent go to mediatype for random bits of data. It would also tend to make those JSON provides far more reusable then they currently are.
PS. Can’t figure out a good way to type XML in comments ;)
“this is why the URL structure of applications doesn’t affect their RESTfulness”
not entirely true- query parameters can affect caching.
“if you can’t use a crappy URL structure with that same API then probably something is wrong with that API.”
Right. So in the AtomPub world we love service documents and the link construct. Clients never need to know URL structure. Except where they do and we revert to documenting parameters (like google charts) or using domain specific discovery languages (like opensearch)
But I think Dave Johnson caught flak here for no good reason and I find both of Roy’s posts lacking to some extent. Something like “Do a GET /articles/{id} to get the representation of a specific article.” is a perfectly good way to explain an “API” to a developer or an implementor. Lesson – don’t confuse explanation with specification. Alternatively, try to define an API for a developer or an implementor, without referring to methods, url structures or response codes. It’s hard to do well.
The real problem – calling these things “APIs” is confusing the heck out of a lot of people.
“So think what MIGHT make a great deal of sense from a pragmatic perspective might be application/rdf+json”
Yes; most JSON structures are dictionaries so the trick is to give those dictionaries a URL and treat each key/value as RDF property/values. But I still think it could be tricky to do. For adoption purposes it’s probably optimal to not say it’s RDF; the “no notation with denotation” crowds (Atom, JSON, uF) tend to have issues imvho with RDF tax.
I totally agree that having json documents containing links to other json documents is a nice way to design apis. It becomes natural to think of the application as a graph that you can crawl to discover more. Links really are what made the web, not the restricted number of verbs, since browsers to this day still don’t support form posts with method PUT or DELETE. Idempotency is an important concept but it’s mostly an accident that most of the web is accessed by GET that mostly happens to not have side effects. As we move to a more massively concurrent read-write web instead of just a passive tv-style web we’ll have to start designing with a greater understanding of idempotency.
As you mention, having documents that contain links lets you ignore the format of the url. A link is a link. It’s nice to have links that are human parse-able, but it’s only nice from the standpoint of the humans learning about the structure of the system. When we added some REST-style resources to Second Life, the first thing we did was bootstrap a single url into the legacy login and teleport protocols — the “seed capability”. This was simply a resource which had named links to other resources that agent could use. All of the actual links were just UUIDs. Since UUIDs are unguessable it was enough to just transmit all of the resources over HTTPS and the capabilities would stay secure, while at the same time allowing extensibility from any of the resources in the graph without any other part of the system knowing.
I do think we need some more widely supported content types for common things. There are plenty of content types for various things lying around like iCal, so it’s probably just a matter of important services supporting useful content types, like we are seeing lots of services usefully provide RSS and ATOM representations of certain resources. Library support in a number of languages is also important, but since many formats are based on xml (or perhaps new formats could be based on json instead?) this may not be much of an issue.
Great post.
Bill:
Query parameters are actually part of the URL, so each different set of query parameters is actually a new resource and has to be cached separately. If this turns out to be a problem in the real world, try stabilizing your query parameters — sort your query parameters alphabetically for example, and refuse to serve requests without sorted query parameters. Also, make sure to move the pieces that are specifying the resource out of the query parameter into the path part and reserve the query parameters for alternate “views” such as sort by descending or another “view transform” like that.
And if you are tempted to use session ids in query parameters — sessions are just gross and shouldn’t be used. Ever. If you need to store some data that is scoped by some time interval or some other logical interval such as the duration of using a shopping cart, PUT a new resource at the beginning of the session and arrange to have DELETE called after the session times out. Then just link to it from the page where the session was created. You don’t really have to literally do the PUT if you use a technique like encrypting data into the url, since all you have to do is give out the encrypted url and if it decrypts again later, you know that the resource really should exist.
The most “disturbing” outcome of Fielding’s very enlightening post is that JSON, as a media type, is totally unadapted to convey explicit information on possible application status transfer.
You could use, as you suggest, a json list of urls for a collection or, even more explicity, do something like:
{“urls”: ['./1', './2', ...]}
but this is NOT a standard.
Confront this with HTML, a standard that explicitly specifies the A tag with HREF attributes. A browser+human user can easily and completely infer API usage information as well as any HTML-aware automata (i.e. http://twill.idyll.org/) can be easily programmed for interaction without “customization” (I don’t have to explain that the links are to be found in the “urls” key).
HTML allows conveying all the API semantics without the need of out of band documentation (i.e. the documentation of the specific Rest API)
JSON then could only be useful for “leaf” information…
That’s true about the unlabeled nature of JSON, and it’s certainly an appealing feature of HTML that you can understand it pretty well. That’s one of the big selling points of microformats. XML, though, is largely as undefined as HTML (
xml:baseis handy though). Atom is more defined, but its extensions aren’t, so even if you understand the link structure of Atom you can’t be sure about any extensions (the exception being<link>elements with a new kind ofrelattribute).Yep Ian, I totally agree on XML and other media-types. So then, at the state of the art, an API that doesn’t use HTML as it’s media-type cannot be considered a “pure” RestFull API. … and for a good reason … That’s “disturbing” ;-)
I guess I wonder why the list of article URLs should be JSON; a single representation of that resource — in HTML — can serve machines and humans just as well if it’s properly written (especially if the consumer knows the semantics of the various “rel” values in the HTML spec, which make for extremely easy automated navigation).
I always thought
/articles.jsonand/articles/1.jsonmade the most sense. That way you could also implement/articles.htmlif you wanted. Some people prefer/articles?format=json, the former is just a little less text :)Hear, hear. URI design is a nice-to-have, not the critical factor here. People should not sweat over these things.
WRT caching and query parameters — it affects it only in a minor way; it disallows a cache from using heuristics to determine freshness; i.e., it has to be explicit, such as with Cache-Control: max-age. Since I hate heuristics, I’d say this isn’t a bad thing.
The only reason you should concern yourself with URI design really is to look for concrete benefits like being able to use relative URIs easily. Making them human-readable and intuitive are again just nice-to-haves.