What is REST exactly? Representational State Transfer
, aka REST
, is an architectural style for web-based data access, an alternative to other techniques like SOAP Web Services and Remote Procedure Calls. Or as Todd Bleeker so aptly put it, it's "Query by URL." It's simpler than SOAP or RPC, and is catching on everywhere. OData
is a protocol - a standardized way to implement REST to surface, query, and manipulate data. REST isn't necessarily focused on data, but OData is, so think "Oh DATA!". This post is a thumbnail sketch of REST/OData in general, and SharePoint REST/OData 2010 and 2013. Note that all SharePoint 2013 comments are based on the SharePoint Server 2013 Technical Preview, a preliminary version subject to change.
OData: Query by URL, Answer by RSS
OData is an open
web-data protocol developed by Microsoft. OData is all about web-based data access; it's a natural evolution of Web Services and Object-Relational Mappings. It's like having LINQ to SQL classes (or some other Object-Relational Mapping) available by URL.
OData can make almost any kind of structured data collection available to any kind of platform, because all data access is via plain old HTTP, and the data is served up as XML (or JSON) in an Atom-style RSS feed. Imagine - a database (or other structured data) can be available for virtually any ad hoc query from almost any internet-connected device. And not just queries- all CRUD operations are available.
SharePoint REST/OData is client-side data access. For code targeting SharePoint 2010 or SharePoint 2013 data, if any of the Client-Side Object Models (CSOM) are available, those are likely the best choices. But if the code is in a non-Windows context, maybe a LAMP application or an Android/iOS phone App, then SharePoint REST/OData is likely the way to go. On the other hand, REST/OData could be a means of bringing external data into SharePoint; Netflix, ebay, twitpics, StackOverflow, and NuGet all have OData feeds. A general understanding of REST could also be useful for consuming Twitter, LinkedIn, or Facebook's REST APIs.
OData.org lists a number of OData services
available for playing around with REST queries right in the browser. If LINQ is comfortable, then another way to wrap your head around REST-speak (or at least get help translating) is LINQPad
. It's a free tool for exploring LINQ, OData, and code in general. Point LINQPad at an OData service (select "WCF Data Services" as the Data Context to build automatically, enter URL, set Database), write and run a LINQ query, then click the SQL button to see the REST URL that was generated. (Or the RequestLog button if it's LINQPad targeting .NET Framework 4.) Here's LINQPad 4 querying an Announcements List in a SharePoint 2010-style ListData.svc service - the URL is in the bottom right pane:
(Sadly, pointing LINQPad at the new SharePoint 2013 OData service _vti_bin/Client.svc results in an error.) Another gotta-have free tool for OData is Fiddler2
. This Web Debugging Proxy lets you inspect each HTTP request and response.
Entity Data Model
OData is built on the Entity Data Model - yes, think .NET Entity Framework and WCF Data Services, which can be used to create OData services in a flash. It's like any Object-Relational Mapping (ORM); one just needs to learn the lingo. In a RDBMS like SQL Server, tables contain rows; in OData, Collections contain Entries. In a database, tables can be related; in OData, Collections can be associated. A row has columns. An Entry has properties. Tables may have keys; Collections always have keys. Browsing to the service root of an OData service usually displays an Atom+XML list of all available Collections. Here's the Microsoft Northwind database as OData:
To see the metadata for this service, append $metadata to the service URL and OData responds with EntityTypes and Properties and NavigationProperties.
Wait a minute- Collections have Entries, so why does the metadata contain EntityTypes? As is often the case, there are multiple sets of terminology at work. Entity Data Model (EDM) and OData have parallel terms:
This returns a feed full of Entries - just like rows in a data table. There's a bunch of metadata for each Entry, like id, title, author, date-time updated, and there are links to related items, which contain relative URLs like "Customers('ALFKI')/Orders". Note in the screenshot above, in the metadata for the first entry, the id is a URL: http://services.odata.org/Northwind/Northwind.svc/Customers('ALFKI'
) - that's how to retrieve just that one Entry. The actual data for this Entry is contained inside the entry/content/properties element at the bottom of the screenshot: CustomerID, CompanyName, ContactName,…
An OData Url has three parts: a Service root, a resource path, and (optionally) query string options. We've seen the service root, which typically returns a list of all available Collections. The resource path is kind of like a relative URL, and identifies a Collection or a single Entry or a property of an Entry. Basically the resource path drills down through the entity model to get to a particular object. OData URLs are usually case-sensitive, so take care with spelling. For example, http://services.odata.org/Northwind/Northwind.svc/Customers('ALFKI')
returns just the ALFKI Customer, which is a single Entry identified by its key:
Query string options, like $value and $links used above, begin with "$" and are part of the query language.
The query language provides
- options ($filter, $sort, $orderby, $top, $expand, $skip, $take, $metadata, …)
- operators (eq, ne, gt, ge, lt, le, and, or, not, mod, add, sub, …)
- functions (startswith, substring, replace, tolower, trim, round, ceiling, day, month, year, typeof, …)
Since these will be part of a URL query string, symbols like "+", "-", and "=" already have assigned meanings, hence the need to represent them by text like "add", "sub", and "eq". Most of these options, operators, and functions have familiar names, and do just what their names suggest.
When the browser receives the JSON, it will probably offer to save it rather than show it. Opening that saved JSON file in Visual Studio looks like this:
Note: appending $format=JSON does not work with SharePoint REST/OData, but there are other ways to get SharePoint REST/OData to return JSON.
CRUD via HTTP Verbs
OData uses the Atom Publishing Protocol, (AtomPub), as the means of Creating, Reading, Updating, and Deleting (CRUD-ing) content. Typically the payload of the HTTP Request is Atom+XML or JSON formatted Entries, like the Entries returned from a query. The HTTP verb is what determines the CRUD action.
- Create = HTTP POST
- Read = HTTP GET
- Update = HTTP PUT or HTTP MERGE
- Delete = HTTP DELETE
Update is an HTTP PUT or MERGE; a PUT replaces an existing entry, by updating all values with the new ones in the request, and setting to default all others. MERGE replaces old values, but leaves anything not specified untouched. Concurrency is maintained by ETags, but we're getting beyond the scope of this post.
SharePoint and REST/OData
In SharePoint 2010, only Lists were available as OData services. The service URL was based on the site and looked like this: http://server/site/_vti_bin/ListData.svc
. Probably the most common way for a SharePoint developer to interact with SharePoint 2010 REST/OData was via a service proxy (a DataContext). Working with a service proxy is easy-peasy; just add a Service Reference to a Visual Studio project, and a tool will generate the proxy for you. Instantiate the proxy class, passing the constructor the URL of the site, set credentials, and issue LINQ queries. Under the covers, that LINQ query will be translated into a REST URL and sent to the server, where it gets translated into a LINQ to SharePoint query, which in turn gets translated into CAML. The proxy will translate the returned XML back into .NET objects.
Here's a short C# code example that uses a proxy to retrieve a SharePoint 2010 List as a Generic .NET List:
public List<OperationsProxy.ProjectRoleItem> RoleList
OperationsProxy.OperationsDataContext ctx =
ctx.Credentials = CredentialCache.DefaultCredentials;
In SharePoint 2013, REST/OData access is no longer limited to Lists - it covers virtually everything the CSOM does, including Site Collections and Webs. This expanded REST/OData service is Client.svc, which can be accessed in two ways:
The "_api" friendly name is preferred, since it's easier to read. The old SharePoint 2010 /_vti_bin/ListData.svc is still there in SharePoint 2013, and is handy if only List data is needed.
There are 5 access points for the _api service:
- Site Collections - http://server/site/_api/site
- Webs - http://server/site/_api/web
- User Profiles - http://server/site/_api/userProfiles
- Search - http://server/site/_api/search
- Publishing - http://server/site/_api/publishing
Happily, many OData URLs can be constructed by starting with the corresponding CSOM method, then replacing "." with "/". For example:
Digging just a little deeper, retrieve just the Title and Author fields from List Item with ID=2 of the Announcements List, as HTML:
Note that the Author field is formatted with presence information, including a link, which navigates to the My Site for that user: