RESTful web service - how to authenticate requests from other services?

Question

I am designing a RESTful web service that needs to be accessed by users, but also other web services and applications. All of the incoming requests need to be authenticated. All communication takes place over HTTPS. User authentication is going to work based on an authentication token, acquired by POSTing the username and password (over an SSL connection) to a /session resource provided by the service.

In the case of web service clients, there is no end user behind the client service. The requests are initiated by scheduled tasks, events or some other computer operations. The list of connecting services is known beforehand (obviously, I guess). How should I authenticate these requests coming from other (web) services? I want the authentication process to be as easy as possible to implement for those services, but not at the cost of security. What would be the standard and best practices for a scenario like this?

Options that I can think of (or have been suggested to me):

1. Have the client services resort to having a "fake" username and password, and authenticate them in the same way as users. I do not like this option - it just doesn't feel right.

2. Assign a permanent application id for the client service, possibly an application key as well. As far as I have understood this is just the same as having username + password. With this id and key, I can either authenticate each request, or create an authentication token to authenticate further requests. Either way, I do not like this option, because anyone who can get a hold of the application id and key can impersonate the client.

3. I could add an IP address check to previous option. This would make it harder to perform fake requests.

4. Client certificates. Set up my own certificate authority, create root certificate, and create client certificates for the client services. A couple of issues come to mind, though: a) how do I still allow the users to authenticate without certificates and b) how complicated is this scenario to implement from the client service point of view?

5. Something else - there must be other solutions out there?

My service would be running on Java, but I deliberately left out information about what specific framework it would be built on, because I am more interested on the basic principles and not so much on the implementation details - I assume the best solution for this will be possible to implement regardless of the underlying framework. However, I am a bit inexperienced with this subject, so concrete tips and examples on the actual implementation (such as useful third party libraries, articles, etc.) will be much appreciated as well.

Solution 1

Any solution to this problem boils down to a shared secret. I also don't like the hard-coded user-name and password option but it does have the benefit of being quite simple. The client certificate is also good but is it really much different? There's a cert on the server and one on the client. It's main advantage is that it's harder to brute force. Hopefully you've got other protections in place to protect against that though.

I don't think your point A for the client certificate solution is difficult to resolve. You just use a branch. if (client side certificat) { check it } else { http basic auth } I'm no java expert and I've never worked with it to do client side certificates. However a quick Google leads us to this tutorial which looks right up your alley.

Despite all of this "what's best" discussion, let me just point out that there is another philosophy that says, "less code, less cleverness is better." (I personally hold this philosophy). The client certificate solution sounds like a lot of code.

I know you expressed questions about OAuth, but the OAuth2 proposal does include a solution to your problem called "bearer tokens" which must be used in conjunction with SSL. I think, for the sake of simplicity, I'd choose either the hard-coded user/pass (one per app so that they can be revoked individually) or the very similar bearer tokens.

Solution 2

After reading your question, I would say, generate special token to do request required. This token will live in specific time (lets say in one day).

Here is an example from to generate authentication token:

(day * 10) + (month * 100) + (year (last 2 digits) * 1000)

for example: 3 June 2011

(3 * 10) + (6 * 100) + (11 * 1000) = 
30 + 600 + 11000 = 11630

then concatenate with user password, example "my4wesomeP4ssword!"

11630my4wesomeP4ssword!

Then do MD5 of that string:

05a9d022d621b64096160683f3afe804

When do you call a request, always use this token,

https://mywebservice.com/?token=05a9d022d621b64096160683f3afe804&op=getdata

This token is always unique everyday, so I guess this kind of protection is more than sufficient to always protect ur service.

Hope helps

:)

Solution 3

There are several different approaches you can take.

1. The RESTful purists will want you to use BASIC authentication, and send credentials on every request. Their rationale is that no one is storing any state.

2. The client service could store a cookie, which maintains a session ID. I don't personally find this as offensive as some of the purists I hear from - it can be expensive to authenticate over and over again. It sounds like you're not too fond of this idea, though.

3. From your description, it really sounds like you might be interested in OAuth2 My experience so far, from what I've seen, is that it's kind of confusing, and kind of bleeding edge. There are implementations out there, but they're few and far between. In Java, I understand that it has been integrated into Spring3's security modules. (Their tutorial is nicely written.) I've been waiting to see if there will be an extension in Restlet, but so far, although it's been proposed, and may be in the incubator, it's still not been fully incorporated.

Solution 4

I believe the approach:

1. First request, client sends id/passcode
2. Exchange id/pass for unique token
3. Validate token on each subsequent request until it expires

is pretty standard, regardless of how you implement and other specific technical details.

If you really want to push the envelope, perhaps you could regard the client's https key in a temporarily invalid state until the credentials are validated, limit information if they never are, and grant access when they are validated, based again on expiration.

Hope this helps

Solution 5

5. Something else - there must be other solutions out there?

You're right, there is! And it is called JWT (JSON Web Tokens).

JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA.

I highly recommend looking into JWTs. They're a much simpler solution to the problem when compared against alternative solutions.

https://jwt.io/introduction/

View more solutions

Comments

[Tommi]

Hmm, I'm not sure if I was able to follow the explanation. Which user are we talking about? I am talking about application communicating with another application. I suppose you understood this, but I still can't quite understand. What happens when this "token" expires, for example?

[Tommi]

I have nothing against option 2 - I think it's fine solution in a RESTful app - but where does the client service get the token from in the first place? How do they authenticate the first time? Maybe I'm thinking this wrong, but it seems odd that the client service would need to have its own username and password for this.

[Tommi]

Between the end user and the backend service? Wouldn't that leave the client services unauthenticated altogether? It's not what I want. I can of course place that middle layer between my web service and client services, but that still leaves the question open: what would the actual authentication model be?

[Dagang]

The middle layer can be web server such as Nginx, you can do authentication there. The authentication model can be session based.

[jwismar]

If the end-user is a user of yours, then the intermediary service can pass their credentials along to you on the first request, and you can return a cookie or other token.

[jwismar]

Similarly, in the OAuth scenario, the end-user is delegating to the intermediary service their access to your web service.

[Devin M]

Well the token that you generate and send back to the other application is a persistent token, it is tied to the user and to the application. Here is a link to foursquares docs developer.foursquare.com/docs/oauth.html and it is just basically oauth2 so look into that for a good authentication solution.

[Tommi]

There seems to be a misunderstanding - there is no end user at all behind the client service. I have updated my question to explain the situation better.

[Tommi]

As I have tried to explain in my question, I do not want a session-based authentication scheme for these client services. (Please see my updates to the question.)

[Tommi]

There seems to be a misunderstanding - there is no end user at all behind the client service. The foursquare documentation you linked to does briefly mention userless access, though, so it was at least somewhat helpful - thanks! But I'm still unable to form a complete picture of how it would work in reality.

[Devin M]

Just generate keys for the applications then, if all you are doing is allowing access for applications then a simple application_id and application_key should work for authentication. If you want to have them authenticate with a token look into using devise's token authentication options as it would just be a parameter passed with the url request to your application.

[Tommi]

But isn't this then exactly the same as the username + password = session authentication token scenario then? Aren't the application_id and application_key just synonyms for username and password? :) It's perfectly fine if this really is the standard practice for a situation like this - as I said, I'm inexperienced on this - but I just thought there might be other options...

[Devin M]

Its not relaying on a session though, the token wont create a session if you dont want to.

[Tommi]

Ok, perhaps there is no "session" per se, but my web service must still store the token to identify further requests from that client. And expire that token at some point unless I want it to be permanent (which I don't want). It's starting to sound a lot like session to me... :)

[Dagang]

I suggest you use white ip list or ip range instead of name&password. Usually, client service ip is stable.

[Donal Fellows]

I find that the authenticate-every-time method is good because it makes it a cinch to write the client-side code. A variation on it (only usable because we're talking about trusted tasks anyway, and assuming it's over HTTPS of course) is to use a custom HTTP header to contain simple auth information (e.g., a pre-configured UUID) and a custom SpringSec handler on the server side to understand it.

[Devin M]

No the app just submits the token on each request.

[mr-sk]

I would just add that the #1 option listed above should only be done over HTTPS.

[Ankur Verma]

I really like the way you appended the security token in every request but what happens to this when programmer already created 100's of jsp pages and after that have t0 implement the security in previously created 100pages as well as the pages that are going to be created.In that case appending token in each request in not a correct choice.Anyway +1 for your technique. :)

[Tim]

Client certificates are NOT a shared secret. That's why they exist. The client has a private key and the server has a public key. The client never share's its secret, and the public key is not a secret.

[NickG]

What happens if the clocks are not synchronised? Won't the client generate the wrong token in this instance? Even if both generate the datetime in UTC, their clocks could still be different resulting in a Window of time every day, when the token would not work?

[kororo]

@NickG, I had this problem before, the only way to secure this by requesting server time. This will 99% kills the problem of UTC. Of course the downside is extra one call to the server.

[Marjan Venema]

The tutorial link doesn't lead to a tutorial article but to some Java index page on the Oracle site...

[Mina Gabriel]

but i still can consume your web service using this token for a day right ? i don't see how this will help

[Fábio Duque Silva]

@MarjanVenema Well, that's because you're trying the link +2 years after newz2000 answered, but you can always try WayBack Machine: web.archive.org/web/20110826004236/http://java.sun.com/…

[Marjan Venema]

@FábioSilva: Which is known as link rot and that is why I called attention to it. Thanks for the link and food on you for finding the cache of it, but I'd rather that the link were updated to point to the new location of this content, or to a more up-to-date version of it.

[Fábio Duque Silva]

@MarjanVenema: I'm sorry, but you're expecting newz2000 to come here and update the link after it went dead? Like you said, it's link rot, so it happens sooner or later. Either you try to access the archive to see what the author was seeing at that time, or you find the new link and make a positive contribution. I don't see how your comment helped anyone. But here, follow this link: oracle.com/technetwork/articles/javase/… (beware that it'll eventually rot too)

[Marjan Venema]

@FábioSilva: No. I don't expect him to do that. I did read the archive but I didn't have time to go hunt for a new link, so I did the next best thing: put in a comment that it is dead so someone else from the community could find the new location and update the post. As you obviously had the time to find the new link, why didn't you update the link in the post instead of putting it in a comment griping to me?

[kororo]

@MinaGabriel you can add more time frame in the token generation. (minute * 10) + (hour * 100) + (day * 1000) + (month * 10000) + (year (last 2 digits) * 100000)

[mikijov]

Sorry, I am late to the party. I am still security noob, but the suggested answer seems to break two points I keep reading about; 1) server has to store "my4wesomeP4ssword" in recoverable form and 2) the server has to do a DB lookup to verify token (compared to token issued during authentication that does not have user secret, but rather server secret). Am I missing something?

[kororo]

Its better late than never, I guess. From the original quesiton, it mentioned that "How should I authenticate these requests coming from other (web) services? I want the authentication process to be as easy as possible to implement for those services, but not at the cost of security". With easy and still secure in terms of mentioning brute force attack and replay attack. The solution that I provided will consider those. Since you can eliminate the replay attack by adding minute and the brute force will not possible since you always generate it.

[kororo]

And for your question, once client and server know the secret password "my4wesomeP4ssword" by phone/sms/other. It should be still secure because you are behind HTTPS channel. And of course, security for the client and server is out of topic such as if attacker can breach the system either client/server.