42
RESTful Service Best Practices
essence, the requester adds a query-string parameter (e.g. jsonp=”jsonp_callback”) to the request,
where the value of the “jsonp” parameter is the name of a javascript function that will be called when
the response is returned.
There severe limitations to the functionality enabled by JSONP, since GET requests do not contain a
request body and, therefore, information must be passed via query-string parameters. Also, to support
PUT, POST and DELETE operations, the effective HTTP method must also be passed as a query-string
argument, such as _method=POST. Tunneling the HTTP method like this is not recommended and can
open services up to security risks.
JSONP works on legacy browsers which preclude CORS support, but affects how services are built if
they're going to support it. Alternatively, JSONP can be implemented via a proxy. Overall, JSONP is
being de-emphasized in favor of CORS. Favor CORS whenever possible.
To support JSONP on the server side, when the JSONP query-string parameter is passed in, the
response must be manipulated a bit as follows:
1. The response body must be wrapped as the parameter to the given javascript function in the
jsonp parameter (e.g. jsonp_callback(“<JSON response body>”)).
2. Always return HTTP status 200 (OK) and return the actual status as part of the JSON response.
Additionally, it's also often necessary to include headers as part of the response body. This enables the
JSONP callback method to make decisions on response handling based on the response body since it's
not privy to the information in response headers and status.
An example error response following the above wrapped response recommendations is as follows
(note: HTTP response status is 200):
jsonp_callback(“{'code':'404', 'status':'error','headers':[],'message':'resource XYZ not
found','data':'NotFoundException'}”)
A successful creation response looks like this (still with an HTTP response status of 200):
jsonp_callback(“{'code':'201', 'status':'error','headers':
[{'Location':'http://www.example.com/customers/12345'}],'data':'12345'}”)
Querying, Filtering and Pagination
For large data sets, limiting the amount of data returned is important from a band-width standpoint.
But it's also important from a UI processing standpoint as a UI often can only display a small portion of
a huge data set. In cases where the dataset grows indefinitely, it's helpful to limit the amount of data
returned by default. For instance, in the case of Twitter returning a person's tweets (via their home
timeline), it returns up to 20 items unless otherwise specified in the request and even then will return a
maximum of 200.
Aside from limiting the amount of data returned, we also need to consider how to “page” or scroll
through that large data set if more than that first subset needs retrieval. This is referred to as pagination
—creating “pages” of data, returning known sections of a larger list and being able to page “forward”
and “backward” through that large data set. Additionally, we may want to specify the fields or
properties of a resource to be included in the response, thereby limiting the amount of data that comes
05/29/12
www.RestApiTutorial.com
Page 23 of 34
42
RESTful Service Best Practices
back and we eventually want to query for specific values and/ or sort the returned data.
There are combinations of two primary ways to limit query results and perform pagination. First, the
indexing scheme is either page-oriented or item-oriented. In other words, incoming requests will
specify where to begin returning data with either a “page” number, specifying a number of items per
page, or specify a first and last item number directly (in a range) to return. In other words the two
options are, “give me page 5 assuming 20 items per page” or “give me items 100 through 120.”
Service providers are split on how this should work. However, some UI tools, such as the Dojo JSON
Datastore object, chooses to mimic the HTTP specifications use of byte ranges. It's very helpful if your
services support that right out of the box so no translation is necessary between your UI toolkit and
back-end services.
The recommendations below support both the Dojo model for pagination, which is to specify the range
of items being requested using the Range header, and utilization of query-string parameters. By
supporting both, services are more flexible—usable from both advanced UI toolkits, like Dojo, as well
as by simple, straight-forward links and anchor tags. It shouldn't add much complexity to the
development effort to support both options. However, if your services don't support UI functionality
directly, consider eliminating support for the Range header option.
It's important to note that querying, filtering and pagination are not recommended for all services. This
behavior is resource specific and should not be supported on all resources by default. Documentation
for the services and resources should mention which end-points support these more complex
capabilities.
Limiting Results
The “give me items 3 through 55” way of requesting data is more consistent with how the HTTP spec
utilizes the Range header for bytes so we use that metaphor with the Range header. However, the
“starting with item 2 give me a maximum of 20 items” is easier for humans to read, formulate and
understand so we use that metaphor in supporting the query-string parameters.
As mentioned above, the recommendation is to support use of both the HTTP Range header plus query-
string parameters, offset and limit, in our services to limit results in responses. Note that, given support
for both options, the query-string parameters should override the Range header.
One of the first questions your going to ask is, “Why are we supporting two metaphors with these
similar functions as the numbers in the requests will never match? Isn't that confusing?” Um... That's
two questions. Well, to answer your question, it may be confusing. The thing is, we want to make
things in the query-string especially clear, easily-understood, human readable and easy to construct and
parse. The Range header, however, is more machine-based with usage dictated to us via the HTTP
specification.
In short, the Range header items value must be parsed, which increases the complexity, plus the client
side has to perform some computation in order to construct the request. Using the individual limit and
offset parameters are easily-understood and created, usually without much demand on the human
element.
05/29/12
www.RestApiTutorial.com
Page 24 of 34
39
RESTful Service Best Practices
Limiting via the Range Header
When a request is made for a range of items using a HTTP header instead of query-string parameters,
include a Range header specifying the range as follows:
Range: items=0-24
Note that items are zero-based to be consistent with the HTTP specification in how it uses the Range
header to request bytes. In other words, the first item in the dataset would be requested by a beginning
range specifier of zero (0). The above request would return the first 25 items, assuming there were at
least 25 items in the data set.
On the server side, inspect the Range header in the request to know which items to return. Once a
Range header is determined to exist, it can be simply parsed using a regular expression (e.g.
“items=(\\d+)-(\\d+)”) to retrieve the individual range values.
Limiting via Query-String Parameters
For the query-string alternative to the Range header, use parameter names of offset and limit, where
offset is the beginning item number (matches the first digit in the items string for the Range header
above) and limit is the maximum number of items to return. A request using query-string parameters
that matches the example in the Range Header section above is:
GET http://api.example.com/resources?offset=0&limit=25
The offset value is zero-based, just like the items in the Range header. The value for limit is the
maximum number of items to return. Services can impose their own default and maximum values for
limit for when it's not specified in the query string. But please document those “invisible” settings.
Note that when the query-string parameters are used, the values should override those provided in the
Range header.
Range-Based Responses
For a range-based request, whether via Range HTTP header or query-string parameters, the server
should respond with a Content-Range header to indicate how many items are being returned and how
many total items exist yet to be retrieved:
Content-Range: items 0-24/66
Note that the total items available (e.g. 66 in this case) is not zero-based. Hence, requesting the last
few items in this data set would return a Content-Range header as follows:
Content-Range: items 40-65/66
According to the HTTP specification, it is also valid to replace the total items available (66 in this case)
with an asterisk (“*”) if the number of items is unknown at response time, or if the calculation of that
number is too expensive. In this case the response header would look like this:
Content-Range: items 40-65/*
However, note that Dojo or other UI tools may not support this notation.
05/29/12
www.RestApiTutorial.com
Page 25 of 34
38
RESTful Service Best Practices
Pagination
The above response-limiting schemes works for pagination by allowing requesters to specify the items
within a dataset in which they're interested. Using the above example where 66 total items are
available, retrieving the second “page” of data using a page size of 25 would use a Range header as
follows:
Range: items=25-49
Via query-string parameters, this would be equivalent to:
GET ...?offset=25&limit=25
Whereupon, the server (given our example) would return the data, along with a Content-Range header
as follows:
Content-Range: 25-49/66
This is works great for most things. However, occasionally there are cases where item numbers don't
translate directly to rows in the data set. Also, for an extremely active data set where new items are
regularly added to the top of the list, apparent “paging issues” with what look like duplicates can occur.
Date-ordered data sets are a common case like a Twitter feed. While you can still page through the data
using item numbers, sometimes it's more beneficial and understandable to use an “after” or “before”
query-string parameter, optionally in conjunction with the Range header (or query-string parameters,
offset and limit).
For example, to retrieve up to 20 remarks around a given timestamp:
GET http://www.example.com/remarks/home_timeline?after=<timestamp>
Range: items=0-19
GET http://www.example.com/remarks/home_timeline?before=<timestamp>
Range: items=0-19
Equivalently, using query-string parameters:
GET http://www.example.com/remarks/home_timeline?after=<timestamp>&offset=0&limit=20
GET http://www.example.com/remarks/home_timeline?before=<timestamp>&offset=0&limit=20
For timestamp formatting and handling in different cases, please see the Date Handling section below.
If a service returns a subset of data by default or a maximum number of arguments even when the
requester does not set a Range header, have the server respond with a Content-Range header to
communicate the limit to the client. For example, in the home_timeline example above, that service
call may only ever return 20 items at a time whether the requester sets the Range header or not. In that
case, the server should always respond with content range header such as:
Content-Range: 0-19/4125
or Content-Range: 0-19/*
05/29/12
www.RestApiTutorial.com
Page 26 of 34
44
RESTful Service Best Practices
Filtering and Sorting Results
Another consideration for affecting results is the act of filtering data and/or ordering it on the server,
retrieving a subset of data and/or in a specified order. These concepts work in conjunction with
pagination and results-limiting and utilize query-string parameters, filter and sort respectively, to do
their magic.
Again, filtering and sorting are complex operations and don't need to be supported by default on all
resources. Document those resources that offer filtering and sorting.
Filtering
In this case, filtering is defined as reducing the number of results returned by specifying some criteria
that must be met on the data before it is returned. Filtering can get quite complex if services support a
complete set of comparison operators and complex criteria matching. However, it is quite often
acceptable to keep things sane by supporting a simple equality, 'starts-with' or contains comparison.
Before we get started discussing what goes in the filter query-string parameter, it's important to
understand why a single parameter vs. multiple query-string parameters is used. Basically, it comes
down to reducing the possibility of parameter name clashes. We're already embracing the use of offset,
limit, and sort (see below) parameters. Then there's jsonp if you choose to support it, the format
specifier and possibly after and before parameters. And that's just the query-string parameters
discussed in this document. The more parameters we use on the query-string the more possibilities we
have to have name clashes or overlap. Using a single filter parameter minimizes that.
Plus, it's easier from the server-side to determine if filtering functionality is requested by simply
checking for the presence of that single filter parameter. Also, as complexity of your querying
requirements increases, this single parameter option provides more flexibility in the future—for
creating your own fully-functional query syntax (see OData comments below or at
http://www.odata.org
).
By embracing a set of common, accepted delimiters, equality comparison can be implemented in
straight-forward fashion. Setting the value of the filter query-string parameter to a string using those
delimiters creates a list of name/value pairs which can be parsed easily on the server-side and utilized
to enhance database queries as needed. The delimiters that have worked as conventions are the vertical
bar (“|”) to separate individual filter phrases and a double colon (“::”) to separate the names and values.
This provides a unique-enough set of delimiters to support the majority of use cases and creates a user-
readable query-string parameter. A simple example will serve to clarify the technique. Suppose we want
to request users with the name “Todd” who live in Denver and have the title of “Grand Poobah”. The
request URI, complete with query-string might look like this:
GET http://www.example.com/users?filter="name::todd|city::denver|title::grand poobah”
The delimiter of the double colon (“::”) separates the property name from the comparison value,
enabling the comparison value to contain spaces—making it easier to parse the delimiter from the value
on the server.
Note that the property names in the name/value pairs match the name of the properties that would be
returned by the service in the payload.
05/29/12
www.RestApiTutorial.com
Page 27 of 34
42
RESTful Service Best Practices
Simple but effective. Case sensitivity is certainly up for debate on a case-by-case basis, but in general,
filtering works best when case is ignored. You can also offer wild-cards as needed using the asterisk
(“*”) as the value portion of the name/value pair.
For queries that require more-than simple equality or wild-card comparisons, introduction of operators
is necessary. In this case, the operators themselves should be part of the value and parsed on the server
side, rather than part of the property name. When complex query-language-style functionality is
needed, consider introducing query concept from the Open Data Protocol (OData) Filter System Query
Option specification (see http://www.odata.org/documentation/uri-
conventions#FilterSystemQueryOption
).
Sorting
For our purposes, sorting is defined as determining the order in which items in a payload are returned
from a service. In other words, the sort order of multiple items in a response payload.
Again, convention here says to do something simple. The recommended approach is to utilize a sort
query-string parameter that contains a delimited set of property names. Behavior is, for each property
name, sort in ascending order, and for each property prefixed with a dash (“-”) sort in descending order.
Separate each property name with a vertical bar (“|”), which is consistent with the separation of the
name/value pairs in filtering, above.
For example, if we want to retrieve users in order of their last name (ascending), first name (ascending)
and hire date (descending), the request might look like this:
GET http://www.example.com/users?sort=last_name|first_name|-hire_date
Note that again the property names match the name of the properties that would be returned by the
service in the payload. Additionally, because of its complexity, offer sorting on a case-by-case basis for
only resources that need it. Small collections of resources can be ordered on the client, if needed.
Service Versioning
Services should be versioned as early as possible in the development cycle. Any initial public or
internal release should be versioned right out of the gate.
As a developer, it's easiest to use an API when that API is accessible via simple tools like a browser,
browser plug-ins or command-line tools like 'curl'. Therefore, in the case of versioning, the concept of
'visibility' comes very much into play. This means that it should be very obvious to the API consumer
which version of that API they are consuming. Consequently, to enhance that visibility it is
recommended to place a version number directly in resource URIs, very high in the URI node
hierarchy.
This technique flies in the face of much academic REST conversations as it doesn't embrace the built-in
header system of the HTTP specification using etags, or that a new URI should be added only when a
new concept is introduced. Furthermore, another argument against it is that resource URIs aren't meant
to change over time—when something unrelated to the resource, like a new API version comes out.
However, putting the version in the URI makes the API easier to use, test, and verify that the
05/29/12
www.RestApiTutorial.com
Page 28 of 34
41
RESTful Service Best Practices
appropriate resource representation version is being requested. It could also be argued that since
returned values are actually representations—not the resource itself—the version number reflects a
resource representation appropriately. Additionally, many of the “big boys” such as Twitter, Yammer,
Facebook, Google, etc. frequently utilize version numbers in their URIs.
The current recommendation is to support versioning via version numbers directly in resource URIs. It
makes the version visible and a versioned API easier to understand and use correctly.
Version numbers in URIs should be high in the node hierarchy, preferably as the first node, for
example: api.example.com/v1/users or www.example.com/api/v1/users.
Date/Time Handling
Dates and timestamps can be a real headache if not dealt with appropriately and consistently.
Timezone issues can crop up easily and since dates are just strings in JSON payloads, parsing is a real
issue if the format isn't known, consistent or specified.
Internally, services should store, process, cache, etc. such timestamps in UTC or GMT time. This
alleviates timezone issues with both dates and timestamps.
Date/Time Serialization In Body Content
There's an easy way around all of this—always use the same format, including the time portion (along
with timezone information) in the string. ISO 8601 time point format is a good solution, using the
fully-enhanced format that includes hours, minutes, seconds and a decimal fraction of seconds (e.g.
yyyy-MM-dd'T'HH:mm:ss.SSS'Z'). It is recommended that ISO 8601 be used for all dates represented
in REST service body content (both requests and responses).
Incidentally, for those doing Java-based services, the DateAdapterJ library easily parses and formats
ISO8601 dates and time points and HTTP 1.1 header (RFC 1123) formats, with its DateAdapter,
Iso8601TimepointAdapter and HttpHeaderTimestampAdapter implementation classes, respectively. It
can be downloaded at https://github.com/tfredrich/DateAdapterJ
.
For those creating browser-based UIs, the ECMAScript 5 specification includes parsing and creating
ISO8601 dates in JavaScript natively, so it should be making its way into all mainstream browsers as
we speak. If you're supporting older browsers that don't natively parse those dates, a JavaScript library
or fancy regular expression is in order. A couple of sample JavaScript libraries that can parse and
produce ISO8601 Timepoints are:
http://momentjs.com/
http://www.datejs.com/
Date/Time Serialization In HTTP Headers
While the above recommendation works for JSON and XML content in the content of and HTTP
request or response, the HTTP specification utilizes a different format for HTTP headers. Specified in
RFC 822 which was updated by RFC 1123, that format includes various date, time and date-time
formats. However, it is recommended to always use a timestamp format, which ends up looking like
05/29/12
www.RestApiTutorial.com
Page 29 of 34
38
RESTful Service Best Practices
this in your request headers:
Sun, 06 Nov 1994 08:49:37 GMT
Unfortunately, it doesn't account for a millisecond or decimal fraction of a second in its format. The
Java SimpleDateFormat specifier string is: "EEE, dd MMM yyyy HH:mm:ss 'GMT'"
Securing Services
Authentication is the act of verifying that a given request is from someone (or some system) that is
known to the service and that the requestor is who they say they are. While authentication is the act of
verifying a requestor is who they say they are, authorization is verifying the requestor has permission to
perform the requested operation.
Essentially, the process goes something like this:
1. Client makes a request, including authentication token in X-Authorization header or token
query-string parameter in the request.
2. Service verifies presence of the authorization token, validates it (that it's valid and not expired)
and parses or loads the authentication principal based on the token contents.
3. Service makes a call to the authorization service providing authentication principal, requested
resource and required permission for operation.
4. If authorized, service continues with normal processing.
#3 above could be expensive, but assuming a cacheable access-control list (ACL), it is conceivable to
create an authorization client that caches the most-recent ACLs to validate locally before making
remote calls.
Authentication
Current best practice is to use OAuth for authentication. OAuth2 is highly recommended, but is still in
draft state. OAuth1 is definitely an acceptable alternative. 3-Legged OAuth is also an option for certain
cases. Read more about the OAuth specification at http://oauth.net/documentation/spec/
.
OpenID is an additional option. However, it is recommended that OpenID be used as an additional
authentication option, leveraging OAuth as primary. Read more about the OpenID specification at
http://openid.net/developers/specs/
.
Transport Security
All authentication should use SSL. OAuth2 requires the authorization server and access token
credentials to use TLS.
Switching between HTTP and HTTPS introduces security weaknesses and best practice is to use TLS
by default for all communication.
05/29/12
www.RestApiTutorial.com
Page 30 of 34
Documents you may be interested
Documents you may be interested
