As we discussed here in a previous post, the use of event notifications allows us to develop elegant solutions to solve potentially complex problems.
How can two or more applications (potentially services or microservices) that use data from a particular entity type work as an eventually consistent system, with minimal coupling, without overloading network and without introducing critical points of failure?
The answer to the above question, of course, is a resounding “it depends “.
Generally, having all applications use the same database is not the best choice. After all, this decision would make all systems operate in an extremely tightly coupled manner, hence can create problems as the software becomes large. In addition, the database itself would become a critical point of failure. it is the solution with the lowest development cost and seems “OK” in scenarios where the schema is stable and the application development team is the same.
Developing a service that operates as a “source of truth” for a particular entity type, being consumed by all other services, does not seem to be a very smart strategy either. After all, the coupling would not be due to the database schema, but to the service contract. In addition, the service would also be a critical point of failure (experiencing instability would compromise the system as a whole). Anyway, it would be “OK” if the volume of consumption were small.
The best strategy seems to have each service keeping a copy of the data it needs to work, at least for online operations. This would require more storage space (which costs little), but there would be no implications for external instabilities (no points of failure) and no unnecessary overhead on the structure, especially if the frequency of data modification is small. However, it is the most complex approach, and as we know, complexity always has a cost.
When we have data from an entity, replicated across multiple sources, we can ensure consistency through notification events, which are posted every time an entity is changed. These events would “carry” the updated data and, being “listened to” by all applications that need this data could serve as the source for the update without having to consult the source.
This approach is significantly more potent if triggered events carried the modification context (for example, AddressUpdated, SalaryIncreased, etc.). After all, not all services would have to keep “all” data from the entities of interest.
Importantly, this approach becomes quite complicated if more than one application or service operates as a modification interface for the same subset of data.
Have you used or considered using notification events to give data consistency across distributed systems? Have you come across the challenge indicated in this post?
