Decoupling Processes For Fun And Profit

I’ve been working with a friend on the architecture of a new app. It is designed to scale out horizontally and to have less points of failure. To this end we have tried to keep all processes decoupled by using a message bus (specificly RabbitMQ)

This isn’t too difficult as long as you keep in mind a few basic ideas…

• Keeping state in a database isn't great, but is needed sometimes, but often you can simply send along the data with the work submission to the queue.
• Mirrored queues for critical jobs
• Front-end shouldn't do any lookups itself (besides basic auth/rate limiting)
• Internal processes need a basic internal SLA that you can alert on. ie. 95% of responses come back within 20ms. This means you can define a contract with each service and change the implementation without breaking anything as long as the interface and contract doesn't change.
• There should never be one of anything where possible. A single point of failure is bad. Where not possible, active/passive HA should be considered.

This isn’t exhaustive, but is simply a starting point. I am working on something I hope to publish soon that should expand on this pretty dramaticly.