Separation of Concerns and Replication

In an integration scenario that involves the replication of items from one system (the master) to another (the slave) the direction of the communication is an interesting issue.

First, there is the question, whether the initial creation of a replicated item in the slave should be performed by a request from the master to the slave or by a request from the slave to the master. The former will often be chosen when there is a business rule involved that should trigger the replication (e.g. replicate all employees in the slave system whose contract ends in 6 months). The latter is more likely to be used in a scenario where the slave replicates all new items in the master, in other words, where the slave keeps the overall set of items in sync.

Second there is the question how updates of the items in the master are made available in the slave. This is the classic distinction between polling and publish/subscribe: Should the master notify the client of updates or should the slave poll for updates?

In addition, there is sometimes a desire to notify the master about changes the slave makes to the replicated items. The intention of this scenario is not the bidirectional synchronization of master and slave but reflects the master's interest in any changes the slave makes.

What is the guiding principle to make an informed design decision regarding the direction of communication in such a replication scenario? The answer is separation of concerns with the goal of simplicity and avoiding unnecessary coupling. This leads to the question which of the systems should for which communication play the server role and which one should play the client role?

Every communication is initiated to achieve some goal and the correct separation of concerns puts the component that acts to achieve the goal into the client role and the other component into the server role.

Applied to the above scenarios this means that for initial replication and subsequent updates the slave system should act as a client and the master should act as a server. All the necessary resources to be provided by the sever (e.g. providing list of all items that match a business rule) can be derived.

Applied to the scenario where the master is interested in updates from the slave it means that the master should act as the client because it is the master that has the goal (the slave does not care at all). This leads to certain resources that need to be exposed by the slave.

I have seen at least three production systems that had this issue and all of them did it wrong and would have greatly benefited in terms of simplicity and loose coupling from following the above principled design path.

Related is Roy's post Paper tigers and hidden dragons.

History of Fragment Identifiers

Everything you ever wanted to know about fragment identifiers - an in-depth coverage by Roy back in 2002.

300 and 406 Response Bodies

Reading through some older postings, I came across this by Roy:

That is why we have a WebDAV working group.  Both the 300 and 406 response bodies were left unspecified because the intention was that they be specified by a group that actually had time to study the problem in detail and come up with a [hopefully] better solution than some off-the-cuff invention of mine.  It was one of the WebDAV to-do items, last time I checked.

Has this ever been done? Maybe it is time to create some standard 300 and 406 response media type?

UPDATE: Umm, should have searched a bit more I guess: RFC2295. What is left is to tell clients to look for Alternates headers in 300 and 406 responses.

ESB Decoupling Capabilities and the Elegance of REST

In his presentation "The Role of the ESB" Mark Richards makes the point that the most significant capability of an ESB is that it enables the separation of "Business Services" from "Service Implementations" (at 00:11:25). "Business Services", he says, "are the ones that make sense to the business" while "the implementation service [...] is the one we actually code". As an example he contrasts "PlaceTrade" and "saveTradeOrder()" and talks about how PlaceTrade is much more stable than saveTradeOrder() because the former is a business semantic and not an implementation detail.

While it is truly astonishing what amount of tooling and complexity the slide conveys in order to provide a business service 'PlaceTrade' it is something else that struck me as being really interesting:

It is obvious that, in order to provide a 'PlaceTrade' service, there needs to be implementation details that should not be conveyed to the user of 'PlaceTrade'. (According to Richards, SOA wonderfully solves that problem by using an ESB that translates the business service into an implementation service call. Can't help it...enterprisey comes to mind...)

REST takes a different approach: in a RESTful system, the implementation details that correspond to the translation of 'PlaceTrade' to 'saveTradeOrder()' manifest themselves in the form of the resource model of a service (the actual resources the service chooses to make available). This is a direct consequence of realizing business functionality through a uniform interface.

Interestingly, in a RESTful system, clients are inherently decoupled from the services' resource models - the contract between client and server is expressed solely through hypermedia specifications, independent of any actual service implementation.

In other words: SOA makes use of ESBs to achieve a decoupling that is already inherent in REST per design.

POST /trades
Content-Type: application/trade-order

<trade-order>
....
</trade-order>

Service Descriptions and Legal Contracts

The intermediate result of the thoughts triggered by the recent interchange on the SOA list are here. Comments welcome!

Formal Contracts

When clients in a RESTful system are not ultimately driven by a human then the expectations they make about the resource model of the services and the available representation formats will inevitably be manifested in the client side code to some extend.

While the client developer can go along way in limiting the assumptions made about the server at some point the client code just needs to explicitly assume that there is for example a 'place order' state traversal (aka 'link' or 'form') when interacting with a service were you can order stuff. The client can plan for various ways in which the server could express the link or form but ultimately the client code will contain some form of orderProcessorUri = clientLib.lookupUri(...expression to find order processor URI...).

If the traversal option cannot be found the client will break without a possibility to automatically react. This is where a human user would change the online store or call the hotline.

Steve Jones is concerned about the development process related- and legal implications of the lack of a formal means for documenting such implicit contracts created between client and server once the client manifests its assumptions in its own code or configuration. I think he is making some valid (and too often ignored) points.

I have been working on related questions during the past weeks and currently see the following issues:

• While the freedom that is usually (deliberately) given to servers in Internet specifications improves evolvability it is not a sufficient basis for establishing legal contracts between service consumers and service providers. If you make a serious investment into implementing a client to some service then RFC 2119's SHOULD is probably not the best contractual basis.
• If a service makes use of a variety of media types, extensions, link relations, categories, etc. then how would one express this in a formal way?
• If a client is dependent on the presence of some Atom extension in Atom entry documents for example, how could the service provider state that they are committed to providing that extension?

It is important though to emphasize the general goal of maintaining the flexibility induced by REST's constraints and at the same time to make explicit and to formalize the inevitable coupling that occurs when non-human clients interact with services.

This is important not only for development-process and legal matters but also for IT governance and Service Management issues and critical for the adoption of REST inside the enterprise.