Middleware & SOA by Example

Question

I am an inexperienced Java developer trying to wrap my head around some fundamental middleware/SOA concepts and technologies, specifically:

• Service-Oriented Architecture (SOA)
• Message-Oriented Middleware (MOM)
• Message Queue
• Apache Camel
• Mule
• EJBs
• Endpoints & Routes
• Service Bus/ESB
• JMS

After looking each of these up online/on Wikipedia, I was able to get (for the most part) decent definitions for each of these. What I am not understanding is how all of these technologies/concepts work together on the backend to provide a 2nd/business tier solution.

Can someone please give an example of an architecture that would use all of these technologies/concepts, and explain what role each of them play in the overall solution? Once I see a working example I'm sure it will help me connect most of the dots.

Edit: Since I added the bounty, I've had several answers that suggest reading books. Although I appreciate all feedback here, I simply can't part ways with 300 reputation points for an answer that, essentially, boils down to "RTM" (especially when I'm flat broke and can't afford the manual!) To reiterate, the bounty and definitive answer will go to someone who can hit all of these bullets in a meaningful, practical example. This does not have to be a middleware compendium!!! Just a paragraph or two that shows how all these can be used together in harmony to produce a Java business-tier solution. Thanks again.

Solution 1

SOA main principles: Build systems as set of services where each service is

• Coarse-grained
• Interoperable
• Loosely coupled

A company offers a lot of business services (coarse-grained) developed over many years and exposed to the users (human or other systems) in some form. There are more chances that each of these features have been designed and developed not keeping the above three principles in mind. Moreover, each of those features might be running on disparate heterogeneous platforms, using different technologies etc.

What if you want to integrate these disparate features thus creating new solutions (For e.g. Amazon store front is a new service composed of their catalog service, shopping cart service etc)?

You have two choices:

1. Building the new feature from scratch keeping the 3 principles in mind. But it is a very costly endeavor, and one that’s almost never successful.
2. An effective and less risky alternative is to assemble/compose it from existing, proven (well tested) services.

Option 2 is where ESBs can help with their support for routing, transformation, monitoring etc. Apache Camel, Mule are open-source ESB's. Endpoints & Routes are the terminology used in EIP (Enterprise Integration Patterns) that these ESB's implement. ESB's can take help of MOM's (Message-Oriented-Middleware) when they want to route/integrate services that are running on heterogeneous platforms (For e.g. the catalog service might be running on a mainframe system but the shopping cart is implemented using stateful EJBs running in a Java Application server). Message queue is a concept in MOM that acts a temporary storage of the message between the sender and receiver. This temporary storage provides lot of benefits like asynchronous delivery, guaranteed delivery etc. There are several different MOM vendors like IBM (WebSphere MQ), open-source ActiveMQ etc. We can use JMS to keep your code independent of the vendor.

I tried to relate all the concepts with an example. I also tried to keep it short. Please ask follow up questions to gain more understanding.

MOM is not a requirement to implement SOA. For e.g. if all of your services are exposed over SOAP via HTTP then you don't need a MOM in this case.

Solution 2

Java classes/example for every tech. might not be possible in single post because what you asked is evolution industry went through in last decade and still evolving. So, what happened over last decade can not be covered in one post. However, its good to understand how it went through this phase and why new technology stack required and what kind of problem it solves.

• EJBs Enterprise Java Beans serverside component architecture. It enables rapid and simplified development of

  1) distributed(where multiple application server talks to each other, server components(e.g. service calling other service hosted on different server).

  2) transactional - persistance bean (DB TXNs), most important part of any simple/web/distributed application. Easy development e.g configuration base. Write XML which takes care of the transaction e.g. when to commit, when to roll back(on exceptions) etc. JPA Java Persistance APIs provide object relationship maping. Such as your table row is mapped to your java object through the xml configuration.

  3) secure - authentication(uid/pwd) and authorization(role based- who is logged in user and what all task he can do?).

This looks good at one point to develope any enterprise application however it has got some disadvantages e.g. its very heavy (all jars included in it.). Classes used as bean should confirm to EJB standards (classes should have implemented certain interface for EJB engine to understand which type of bean it is).

To overcome such scenarioes, there are many alternatives are available in industry for EJBs e.g Hibrnate does same things such as OR mapping, TXN handling same provided by persistance bean in EJB. Spring, light weight framework and simplifies business logic (you can use your already build class which need not to implement any interface, checked exceptions or extends some of the mandatory abstract classes).

Now a days, most of the compnies realy on the light weight frame work such as Spring, Hibernate, IBatis, Axis-2.

• Service-Oriented Architecture (SOA) Service Oriented Architecture is an answer to the platform independence at the enterprise level. OR To integrate your application faster, to communicate between different application server.

  Just think that you want to implement solution where you are providing option for hotel booking all over the world. Your requirement is to check availability of rooms in those hotels. Now, it means that you need to interact with multiple hotel applications at a time. It's not necessary that every hotel is using same standard or their application(server, programming language) may be deployed on the different application servers. At a same time, its not practical to write different applications which can talk to all different type of the application server. We need some standard based solution which can solve this problem. It's possible through the Web services.

It is possible because web services are sending message in the SOAP(Simple Object Access Protocol), based on the XML. XML is used to interchange data across any language, platform or network protocol.

Web services can be classified SOAP Based and REST. SOAP based service JAX-RPC and JAX-WS (http://www.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc/index.html)

Web services can be developed contract first - first write WSDL. code first - first write code.

Now, lets talk how to start for the web services practically.

Simplest web service or hello world(JAXWS) can be written as follows:- http://svn.apache.org/viewvc/cxf/trunk/distribution/src/main/release/samples/java_first_jaxws/

• Message-Oriented Middleware (MOM)
• JMS

• Message Queue (Point-to-Point)

  MOM is required to over come disadvantages of request-response style communication. Server need to be alive when client sends response. Client wait for response till server executes and respond back.

  Request response - application will fail if server or client is down. MOM - Either of the end point is not required to be on time you send the request message for the processing.

  MOM is concept and JMS is specification on this concept. Many vendors have implementation of this specification e.g IBM has MQ, OpenJMS open source implementation, EMS from Tibco etc.

JMS specification has majorly two patterns. Pub/sub and ponin-to-point.

Pub/sub is topic, your application want to publish certain information to all the interested parties. e.g. dash board. (Stock application want to notify certain message to all the registered listeners).

Point-to-Point communication is done through message queue.

Business use case- think you have application e.g customer request to the customer care. Other side you have multiple customer care representatives and other side customers sometimes more than customer care representatives, at a time one and only one representative will get the request to be processed and he/she will not get next request until finishes the task. (Same one queue multiple windows and which ever window is free will process the request). You can think of other complexity in this e.g what if one of the node is failed, request not processed and particular type of request needs to be process by particular node. etc.

Produce code:- http://docs.oracle.com/javaee/1.4/tutorial/examples/jms/simple/src/SimpleProducer.java

Consumer synchronous code:- (POJO classes) http://docs.oracle.com/javaee/1.4/tutorial/examples/jms/simple/src/SimpleSynchConsumer.java

http://www.java2s.com/Code/Java/J2EE/ThisexampleisasimpleJMSclientapplication.htm

Consume asynchronous code:- (Spring by example - reads message from destination till program will not be stopped.) http://www.springbyexample.org/examples/simple-spring-jms-listener-config.html

Though, its just basic there are many aspects to be covered in this MOM e.g. what's fail-over mechanism, what is selector, durable message, message acknowledgement modes etc...

• Service Bus/ESB
• Endpoints & Routes
• Apache Camel
• Mule

Now, lets say you have adopted SOA and MOM long back and you have bunch of services which talks to each others to accomplish enterprise wide task. Imagine to manage logic such as multiple destination whom should be redirected from where will be very cumbersome. Some people call this application logic. Service buses will be used to reduce application logic and focus more on the business logic(functionality provided by app).

In Simple words, consider End point as URL exposed on server. You will use this url/end point to invoke your service.

e.g. http://localhost:8888/Context/MyService?wsdl

in code:-

    String endpointAddress = "http://localhost:8080/jaxws/services/hello_world?wsdl";

    // Add a port to the Service
    service.addPort(PORT_NAME, SOAPBinding.SOAP11HTTP_BINDING, endpointAddress);

    HelloWorld hw = service.getPort(HelloWorld.class);
    System.out.println(hw.sayHi("World"));

Routes When service bus receives particular message, it will route it through no of services/broker destinations such as queue/topics. This path is known as route.

E.g your stock application has got some input by analyst, it will be processed through the application/web component and then result will be published to all the interested/registered members for particular stock update.

Apache Camel and Muel http://camel.apache.org/how-does-camel-compare-to-mule.html provides solution for the enterprise integration.

Solution 3

Enterprise Integration Patterns can help you understand how everything fits together.

[update:] Your follow-up question on another answer made me realise that you're confused about specific products. That's partly because software in practice tends to map to more than one concept and partly because different companies argue that they provide "everything", when really they don't.

The ESBs are toolkits / libraries that let you connect everything together. They are neither the services themselves, nor the messaging implementations, but the goo that fills the odd little gaps in-between. If you were writing everything from scratch you might not even need one, because what they are best at is fixing the mismatch between a whole pile of different technologies, and if you are starting from scratch you can avoid that mess.

The services are, well, the services. You might use some EJBs when implementing one (I only mention this because for some reason you include them in your question).

The messaging middleware is software that gets messages from A to B. That's extremely useful, but also complex, and everyone and their brother has invented their own. So you need some abstraction that lets you avoid lock-in. That can be an ESB or, if you are all-Java then it can be JMS. But even when you are all-Java with JMS you may still want to use an ESB because they are libraries of all the bits of Java code you would still need to write (random bits of routing logic, message reformatting, etc etc).

Hope that helps. My original answer is more about the abstract patterns that you build with these tools - when you're wiring things together the same problems come up again and again.

Solution 4

Endpoints & Routes: where the information comes from and goes to. Message Queue is one type of endpoint. The other type is a Message Topic.

An endpoint is a 'logical name for a thing', e.g. PRICE.MSFT, that is used by a publisher or consumer application to get things from or send to. Topics deliver information to everyone subscribed (one-to-one or one-to-many), queues deliver messages to the first one that gets it (usually one-to-one). Forget about Queues, everything can also be done with Topics and topics have several advantages.

Message-Oriented Middleware (MOM): the software infrastructure that delivers information between topics or queus. It is 'message-oriented' not 'packet oriented' as TCP is. So each information blob is delivered in one, hopefully self-describing, message. The implementation of your MOM then gives you an API where you can do things like msg.get("bid")

JMS and AMQP are examples of a MOM specifications. MOM implementations are the real products that implement these specs: TIBCO EMS, Websphere MQ, MSMQ, Solace, and many, many others

Apache Camel - very interesting approach on how to configure workflows in this MOM world. But a more advanced concept.

Service-Oriented Architecture (SOA), Service Bus/ESB are just new buzzwords words for what used to be called EAI (Enterprise Application Integration). They are recommendations on how to use 'MOM's and a way to pay high-priced consultants. What 'ESB' adds to a MOM is the idea of thinking of your publishers as 'services' providing a service. In other words: don't think too much about what one consumer wants right now. There might be 5 consumers in the future and that publisher should provide a service, not 'create information that consumer A wants'. (It'll become clearer once your architecture has grown to 5+ applications). Also you should have some common object model, maybe in XML to make things simple between applications.

Mule - one form of ESB, but its' not exactly main-stream. In 5 years, most middleware action might have been moved to AMQP or something else entirely.

EJBs: Sun's idea of sophisticated Java classes that run in a container. Supposed to make application development easier. But in many cases it made things more complex. A better alternative would be 'Spring' - but EJB's are about something else (not just MOM). Its more about how to develop bigger applications (see IoC pattern).

If you are looking on where to start: I would recommend learning about JMS (all other MOM's are simliar and JMS is the basis of EJB's/ Mule, ...) and, unless you have super-high performance requirements, consider messages to be a TextMessage containing XML. Most tools are available in that area. Or even simpler but less sophisticated, a MapMessage with key/value pairs.

Solution 5

Taking all of your requirements and packaging them into a query, I came across an excellent case study that should meet your needs:

• Service Oriented Architecture: An Integration Blueprint

I went ahead and fulltext searched the book using Amazon's "Search Inside This Book" feature. It covers all of the integration cases you've discussed, appears to be thorough, and steps you through the entire design and implementation process.

I'm embarrassed to state I haven't read through this myself, but I highly recommend using the same tools I did to see if it fits your needs before investing in a copy. It seems more thorough, more complete, and more helpful than simply foisting you on a whole lot of incomplete documentation or spooling out content into an answer here.

View more solutions