Separate REST JSON API server and client?

Question

I'm about to create a bunch of web apps from scratch. (See http://50pop.com/code for overview.) I'd like for them to be able to be accessed from many different clients: front-end websites, smartphone apps, backend webservices, etc. So I really want a JSON REST API for each one.

Also, I prefer working on the back-end, so I daydream of me keeping my focus purely on the API, and hiring someone else to make the front-end UI, whether a website, iPhone, Android, or other app.

Please help me decide which approach I should take:

TOGETHER IN RAILS

Make a very standard Rails web-app. In the controller, do the respond_with switch, to serve either JSON or HTML. The JSON response is then my API.

Pro: Lots of precedent. Great standards & many examples of doing things this way.

Con: Don't necessarily want API to be same as web app. Don't like if/then respond_with switch approach. Mixing two very different things (UI + API).

REST SERVER + JAVASCRIPT-HEAVY CLIENT

Make a JSON-only REST API server. Use Backbone or Ember.js for client-side JavaScript to access API directly, displaying templates in browser.

Pro: I love the separation of API & client. Smart people say this is the way to go. Great in theory. Seems cutting-edge and exciting.

Con: Not much precedent. Not many examples of this done well. Public examples (twitter.com) feel sluggish & are even switching away from this approach.

REST SERVER + SERVER-SIDE HTML CLIENT

Make a JSON-only REST API server. Make a basic HTML website client, that accesses the REST API only. Less client-side JavaScript.

Pro: I love the separation of API & client. But serving plain HTML5 is quite foolproof & not client-intensive.

Con: Not much precedent. Not many examples of this done well. Frameworks don't support this as well. Not sure how to approach it.

Especially looking for advice from experience, not just in-theory.

Solution 1

At Boundless, we've gone deep with option #2 and rolled it out to thousands of students. Our server is a JSON REST API (Scala + MongoDB), and all of our client code is served straight out of CloudFront (ie: www.boundless.com is just an alias for CloudFront).

Pros:

• Cutting-edge/exciting
• A lot of bang for your buck: API gives you basis for your own web client, mobile clients, 3rd party access, etc.
• exceedingly fast site loading / page transitions

Cons:

• Not SEO friendly/ready without a lot more work.
• Requires top-notch web front-end folk who are ready to cope w/ the reality of a site experience that is 70% javascript and what that means.

I do think this is the future of all web-apps.

Some thoughts for the web front end folks (which is where all the new-ness/challenge is given this architecture):

• CoffeeScript. Much easier to produce high-quality code.
• Backbone. Great way to organize your logic, and active community.
• HAMLC. Haml + CoffeeScript templates => JS.
• SASS

We've built a harness for our front-end development called 'Spar' (Single Page App Rocketship) which is effectively the asset pipeline from Rails tuned for single page app development. We'll be open-sourcing within the next couple of weeks on our github page, along with a blog post explaining how to use it and overall architecture in greater detail.

UPDATE:

With respect to people's concerns with Backbone, I think they are over-rated. Backbone is far more an organizational principle than it is a deep framework. Twitter's site itself is a giant beast of Javascript covering every corner-case across millions of users & legacy browsers, while loading tweets real-time, garbage collect, display lots of multimedia, etc. Of all the 'pure' js sites I've seen, Twitter is the odd one out. There have been many impressively complicated apps delivered via JS that fare very well.

And your choice of architecture depends entirely on your goals. If you are looking for the fastest way to support multiple clients and have access to good front-end talent, investing in a standalone API is a great way to go.

Solution 2

Very well asked. +1. For sure, this is future useful reference for me. Also @Aaron and others added value to discussion. Like Ruby, this question is equally applicable to other programming environments.

I have used the first two options. First one for numerous applications and second one for my open source project Cowoop

Option 1

This one is no doubt the most popular one. But I find implementation are very much http-ish. Every API's initial code goes in dealing with request object. So API code is more than pure ruby/python/other language code.

Option 2

I always loved this.

This option also implies that HTML is not runtime generated on server. This is how option 2 is different from option 3. But are build as static html using a build script. When loaded on client side these HTML would call API server as JS API client.

• Separation of concerns is great advantage. And very much to your liking (and mine) backend experts implement backend APIs, test them easily like usual language code without worrying about framework/ http request code.

• This really is not as difficult as it sounds on frontend side. Do API calls and resulting data (mostly json) is available to your client side template or MVC.

• Less server side processing. It means you may go for commodity hardware/ less expensive server.

• Easier to test layers independently, easier to generate API docs.

It does have some downsides.

• Many developers find this over engineered and hard to understand. So chances are that architecture may get criticized.

• i18n/l10n is hard. Since HTML is essentially generated build time are static, one needs multiple builds per supported language (which isn't necessarily a bad thing). But even with that you may have corner cases around l10n/i18n and need to be careful.

Option 3

Backend coding in this case must be same as second option. Most points for option 2 are applicable here as well.

Web pages are rendered runtime using server side templates. This makes i18n/l10n much easier with more established/accepted techniques. May be one less http call for some essential context needed for page rendering like user, language, currency etc. So server side processing is increased with rendering but possibly compensated by less http calls to API server.

Now that pages are server rendered on server, frontend is now more tied with programming environment. This might not be even a consideration for many applications.

Twitter case

As I understand, Twitter might does their initial page rendering on server but for page updates it still has some API calls and client side templates to manipulate DOM. So in such case you have double templates to maintain which adds some overhead and complexity. Not everyone can afford this option, unlike Twitter.

Our project Stack

I happen to use Python. I use JsonRPC 2.0 instead of REST. I suggest REST, though I like idea of JsonRPC for various reasons. I use below libraries. Somebody considering option 2/3 might find it useful.

• API Server: Python A fast web micro framework - Flask
• Frontend server: Nginx
• Client side MVC: Knockout.js
• Other relevant tools/libs:
  • Jquery
  • Accounting.js for money currency
  • Webshim : Cross browser polyfill
  • director: Client side routing
  • sphc: HTML generation

My conclusion and recommendation

Option 3!.

All said, I have used option 2 successfully but now leaning towards option 3 for some simplicity. Generating static HTML pages with build script and serving them with one of ultra fast server that specialize in serving static pages is very tempting (Option 2).

Solution 3

We opted for #2 when building gaug.es. I worked on the API (ruby, sinatra, etc.) and my business partner, Steve Smith, worked on the front-end (javascript client).

Pros:

1. Move quickly in parallel. If I worked ahead of Steve, I could keep creating APIs for new features. If he worked ahead of me, he could fake out the API very easily and build the UI.

2. API for free. Having open access to the data in your app is quickly becoming a standard feature. If you start with an API from the ground up, you get this for free.

3. Clean separation. It is better to think of your app as an API with clients. Sure, the first and most important client may be a web one, but it sets you up for easily creating other clients (iPhone, Android).

Cons:

1. Backwards Compatibility. This is more related to an API than your direct question, but once your API is out there, you can't just break it or you break all your clients two. This doesn't mean you have to move slower, but it does mean you have to often make two things work at once. Adding on to the API or new fields is fine, but changing/removing shouldn't be done without versioning.

I can't think of anymore cons right now.

Conclusion: API + JS client is the way to go if you plan on releasing an API.

P.S. I would also recommend fully documenting your API before releasing it. The process of documenting Gaug.es API really helped us imp

http://get.gaug.es/documentation/api/

Solution 4

I prefer to go the route of #2 and #3. Mainly because #1 violates separation of concerns and intermingles all kinds of stuff. Eventually you'll find the need to have an API end point that does not have a matching HTML page/etc and you'll be up a creek with intermingled HTML and JSON endpoints in the same code base. It turns into a freaking mess, even if its MVP, you'll have to re-write it eventually because its soo messy that its not even worth salvaging.

Going with #2 or #3 allows you to completely have a API that acts the same (for the most part) regardless. This provides great flexibility. I'm not 100% sold on Backbone/ember/whatever/etc.js just yet. I think its great, but as we're seeing with twitter this is not optimal. BUT... Twitter is also a huge beast of a company and has hundreds of millions of users. So any improvement can have a huge impact to bottom line on various areas of various business units. I think there is more to the decision than speed alone and they're not letting us in on that. But thats just my opinion. However, I do not discount backbone and its competitors. These apps are great to use and are very clean and are very responsive (for the most part).

The third option has some valid allure as well. This is where I'd follow the Pareto principle (80/20 rule) and have 20% of your main markup (or vice versa) rendered on the server and then have a nice JS client (backbone/etc) run the rest of it. You may not be communicating 100% with the REST api via the JS client, but you will be doing some work if necessary to make the suer experience better.

I think this is one of those "it depends" kinds of problems and the answer is "it depends" on what you're doing, whom you're serving and what kind of experience you want them to receive. Given that I think you can decide between 2 or 3 or a hybrid of them.

Solution 5

We use the following variant of #3: Make a JSON-only REST API server. Make an HTML website server. The HTML web server is not, as in your variant, a client to the REST API server. Instead, the two are peers. Not far below the surface, there is an internal API that provides the functionality that the two servers need.

We're not aware of any precedent, so it's kind of experimental. So far (about to enter beta), it has worked out pretty well.

View more solutions

Comments

[Rhb123]

A minor point to add: While I have only built option #1, I know multiple mobile app developers who are starting to use parse.com as their backend in order to enable a fast path to #2.

[Aaron]

Things like Parse and Kinvey are very interesting, can't say I've had a chance to play w/ them yet. Depends if your value is in the front or back of the stack I suppose

[MartinodF]

I'm thinking about this option to avoid some problems related to being a proper API client, such as authentication. I'd like to know more about how you structured the whole thing and how you manage separation and communication between the three different parts. Is there anything I could read? Thanks!

[Admin]

@MartinodF We host on Google App Engine, which limits to Java or Python. Wanted to use Python, but were forced into Java because we crunch numbers (can't extend Py with C/C++ on GAE). We chose Stripes (Stripes, not Struts, not Spring) for the presentation framework. Very happy with that. The whole thing is one Java app on GAE. The core functionality is implemeted in a bunch of Java packages and exposed in an internal API. There is a servlet that provides the JSON REST service, and another that is configured as a Stripes web app. Since it's all one GAE Java app, communication is trivial.

[MartinodF]

Thanks for the insight, it's very useful!

[Shannon]

So you detect whether the request is coming from a search bot, and serve pre-rendered HTML if it is, and JS+Templates if it's not?

[Shannon]

So you are rendering markup server-side but you are still giving templates to the client and using Backbone?

[Sebastian Wramba]

May I ask how you authenticate the web frontend with the REST API? I saw that you need an API key to communicate with the API which is obtained by logging in to your user profile. But how does the web client get its API key, if you know what I mean?

[Blue Smith]

I also like option 2, but option 3 has a lot of advantages that we can't get rid from. I'm trying to find some hydrid-solution combining both opt2 + opt3, but it will lead to the headache like Twitter.

[Nicolas Goy]

I use the same approach with spinejs for the frontend.

[Erich]

How do you handle a single domain running two separate applications? Eg. I have www.mysite.com and I want to expose a public API and serve a front-end on that URL. True to REST principles, mysite.com/product/24 accessed from a web browser should return an HTML page by looking at the HTTP Accept header, and a GET with JSON in the Accept header on mysite.com/product/24 should return JSON.

[Ankan-Zerob]

How would AngularJS pan out for this?

[Ujjwal Ojha]

+1 to hybrid of 2 and 3

[Saltymule]

FYI: There is an isomorphic javascript framework that will run your javascript client on the server, allowing you to serve html pages, which are SEO friendly. When the html/javascript hits the client, it starts behaving like a client application as you would expect. bloomberg.com and bloombergview.com use this: github.com/bloomberg/brisket

[AmaChefe]

I love option 3, and aiming to use if for a current project. Any eg or git repo you can point to for help?

[Shekhar]

@AmaChefe I wish. For current project where SEO is crucial we use option 3. But code isn't open source. We use flask+jinja2 and knockout/react.js.

[Katrina]

@SebastianWramba This is late, but since your comment got 12 upvotes... I would look at something like OAuth2's password authorization. If you are the creator of the app calling the API, this is the approach you probably want, since it doesn't use the API key directly. If it's a third party app, you have the user login to your website to get their API key, and then the user uses that key (and any other needed credentials) to access the API via their app, website, etc.