How to SOA an existing project

SOA or service oriented architecture is a way of building a complex project so that each individual unit can stand alone with a documented API.  There are many benefits to taking an SOA approach to designing a complex software system.

To those familiar with programming, SOA provides the same sorts of benefits as OOP but at a higher level of abstraction and with a network interface rather than a code interface.  The benefits of providing encapsulated logical units that can hide the complexities is well understood by most experienced programmers.

For a business taking a SOA approach to building a complex project makes a lot of sense.  It allows teams to be divided into smaller groups to reduce the communication load and help limit the scope of the problems that any one developer has to concern themselves with.

So how do you take a monolithic piece of existing software and change it to SOA?

The first and perhaps most critical decision to make is where to cut your application into pieces which can become isolated services.  The most obvious first approach might be to cut between the GUI and the business logic.  That is usually NOT the best thing to do for any real complex project.  Deep coupling of the UI usually makes this division much to big for an initial undertaking.

Likewise starting with something too small can lead to an abandoned project.  A simplistic project that can be ‘finished’ such that no developer ever has to be familiar with how it was built can lead to problems down the road when it breaks and no one knows how to fix it.

So an SOA unit should be logical,  it should be not too big that it is destined to fail, and not too small that it can be forgotten.

A further guide to finding an aspect of the code to pull into a service is whether it will have more than 1 customer or could have more than 1 if it was a service.  By customer I mean anything that might want to call into it’s API.  For a HR system the customers might be the GUI used by HR department, tools used by managers, excel spreadsheets used by the CFO and a business intelligence dashboard for the CEO.  With that many potential customers of an API there is a strong benefit to having an SOA approach compared to asking HR for quarterly reports or sending them emails to update an employees phone number.

One further option to consider is to wrap your entire application in an API so that it can be leveraged somewhere new.

On one project I worked on we had a complex requirement for managing encryption keys. We needed a way to expire old keys, provide redundancy against server failures, and have a nuclear missile style 2 person/2 password login process.  It made sense to develop this as a stand alone service.   It paid off with a project that gives us confidence that some newbie programmer can’t easily create a critical security hole in an application that uses it.

Amazon is perhaps the best case study for a successful SOA deployment.  Their SOA approach has spawned entirely new billion dollar businesses (AWS).  One of the benefits they’ve seen is that because of their highly used SOA APIs the requirement for testing is greatly reduced.  Incremental updates get slowly rolled out providing time to listen for errors.  On these highly exercised APIs regressions can found quickly before any harm is done.  The actual use of the service provides a much more complete test of it than any test suite could possibly perform.

Perhaps the best place to start on an SOA conversion project is to see if there is already some natural divisions in the developers knowledge and experience within the existing code base.  Creating SOA services based on these natural lines will ensure that the new teams will already have an expertise in their service and reduce the learning/ramp up time.  Domain knowledge is the biggest hurdle for any programmer (ie it take a lot more time to switch from enterprise software to video games than it does to switch from Javascript to C#).

So you’ve got an idea for where to start.  The next thing to decide is what the API should look like.  I prefer RESTful interfaces because of the lower barrier required over something like SOAP for clients.  SOAP would make things like a bash shell script difficult to do.  I also prefer JSON over XML for similar reasons but also because XML allows for some style inconsistencies between developers such as should something be an attribute or child.  Should authentication be done with simple username/password or are there requirements for something like OAuth?

Your team should probably develop a couple of libraries in some key target languages for their API.  This hides things like URLs and encodings behind a more obvious code API.

Documentation is also very important.  An undocumented API will probably never be used by anyone.  Both the raw API and any libraries should be documented with examples.  If it’s a public API, put your libraries up on github.com so that other developers can easily find them and keep up to date.

Work in some analytics if you can for the new API.  Tracking usage and performance of your service is fascinating and can provide some key insights into where improvements could be made.

Lastly, a new SOA project will include some new infrastructure and deployment requirements.  You’ll need to decide on server requirements, uptime and redundancy for this new service.  As well as write any new deployment scripts for it.

So an SOA conversion is not a trivial task.  It does come with some overhead which is usually made up for by growth of the service as it finds new customers and new use cases.  Reducing the scope of a large project into separate services provides less easily measurable benefits such as reducing communication costs, reducing the scope of future changes and making the project easier to understand.