How do I discover RESTful Web services?

Question

When I want to find a "Big" Web service (WS-*/WSDL based) of required functionality I can simply google for ".... filetype:wsdl" because Google indexes also WSDL definitions. Or I can simply use some registries that already crawled the Web for WSDL definitions for me, e.g. SeekDa.com or ServiceFinder.

When I want to find RESTful Web service (RESTful Web API) I may count only on the community, because it is not possible to distinguish between URI of RESTful Web service and other Web resources (e.g. Web sites URLs) and hence any focused crawler cannot tell for 100% sure: this is URI of RESTful Web service. So, I come e.g. to the ProgrammableWeb.com and I hope someone already used/found the service I'm looking for and registered it there.

Recently, I was looking for the Web service for abbreviation expansion. I couldn't find it in portals like ProgrammableWeb, because no-one has put it there. Finally I was forced to browse through several abbreviation dictionaries sites, dig and drill there and I finally found Abbreviations.com Web service API in RESTful form. That took me a lot of time!

Is there any other Way to discover RESTful Web services on the Web?

Solution 1

The REST ideal is that services are as consumable as Web Pages are, they don't need registries. [Got to admit I don't fully "get" the implications of this RESTful world, no WSDL, no registry feels like riding a bike with no stabilisers.]

In the old Web, pre-Google, how did we find pages? Effectively word-of-mouth and a few key starting points. REST services so far as I can see are pretty much in the pre-Google stage.

I don't agree that "it is not possible to distinguish between URI of RESTful Web service and other Web resources" - if we follow the link we get certain types of content application/xml and application/json would be pretty strong indicators wouldn't they?

Solution 2

There is a convention (recommended in REST API Design Handbook for example) that you expose a /api endpoint at the root of your service. This returns an XML or JSON response containing the "child" resources that your service supports e.g. /api/products

Solution 3

There's a WSDL-like file for REST webservices: it's called WADL.

Solution 4

SoapUI now can discover the REST services. It works as a proxy, writing down all requests/responses that pass through. Having requests and responses SoapUI recreates descriptions/definitions of the services. Now the definitions can be stored in WADL and WSDL formats (as WSDL and XML-Schema in XML world). It also can be stored in Swagger format. I prefer Swagger. Swagger can be stored in SwaggerHub right from SoapUI, which is like GitHub for source code. SwaggerHub is one of many API management systems.

View more solutions

Comments

[dzieciou]

Thank you. That's a good idea assuming RESTful Web services are described with WADLs. But currently it's just idea, not commonly practices. So coming back to the real world: How can I find interesting RESTful Web service NOW?

[Stefano Borini]

unless you know them, you can't. That's what WADL tries to solve.

[dzieciou]

Thank you. I haven't considered MIME types of the service response :-)

[dzieciou]

Just a comment: you're never sure whether application/xml has been generated on call (as a service response) or its a static XML document. So this is just one of the possible indicators. Any other suggestions?

[djna]

You don't need to know that. Provded it satisifies the URI you can't demand how it should be served. Some answers could be precalculated and cached.

[dzieciou]

I may need to know that to try to extract functionality from the associated Web sites to automatically generate WADL definition or at least try to do it. I don't want to waste my time on resources that are not Web service API. For instance, the following appears to be some static data: data.eastmoney.com/flash2/data2_1100.xml?r= (with application/xml MIME type). Though it may be also generated on call. I see no explicit feature of the HTTP response to say it comes from the Web service (even pre-calculated and cache) or it's a static document.

[dzieciou]

OK. BTW, do you know any research on extracting WADL definitions from the document of the API, associated Web pages? Any trials? That would be nice way generate WADL-s semi-automatically and provide some smooth transition (evolution instead of revolution) from the current lack of interfaces for RESTful Web services.

[djna]

My point is that this is a restful service! In the sense that you have a confirming URI and some conforming result. If the service implementation is some JAVA running or we suck from disk the "interface" is indistinguishable. It walks and quacks like a REST service, for your purposes it is a service. If some aspect of the response doesn't conform to the shape of a service then it ain't a service - even if some Java did produce it. You can only look at URI and response, that's it. You cannot tell difference between static and dynamic. Consider "universe/ended" -> {"no"}

[Donal Fellows]

Do you have a link to that guidebook? It would greatly enhance your answer…

[Gusdor]

This is a bit of a necro but it is impossible to write generic tooling to consume a REST service. WSDL is more complex, but it allows my tools to just generate a client endpoint for me. Anecdotal: I'm currently writing to consume a rest service with no documentation. It is a total ballache. Looking at the web browser logs and making educated guesses...

[mobby]

This is just a convention and totally up to developers if they like exposing their endpoints.