Decoupling: You're gonna need it

One of the key properties of Clean Architecture (also known as hexagonal architecture, ports and adapters, or onion architecture) is that the domain model of an application or service is decoupled from all external integrations, including the persistence mechanism. (The latter decoupling also goes by the name persistence ignorance.) This means that all external integrations are hidden behind interfaces, so that there is no source code dependency from the business logic or domain model to any external system.

Some developers hear this and object with YAGNI: adding extra interfaces increases complexity and is a code smell, they claim. No interface should ever exist unless there are at least two implementations used in production, they say.

Of course, YAGNI is a real concern, but I claim that this kind of objection is not its intention.

What is the difference?

My answer to the YAGNI objection is, "But what if you do need it?" After all, you have no crystal ball to see what will change and what won't.

Take persistence ignorance as example. The persistence layer and the domain model are different things which change for different reasons. This makes them exactly the right targets for decoupling.

When arguing that one should decouple the domain model and the persistence layer because one or the other could change, one often hears, "But that will never change," normally thinking of a large scale change such as swapping out the entire database. Can one really guarantee that the two will never change independently of each other? Over 2, 5, 10 years? Remember, no one has a crystal ball.

Consider the simplest case of a change to the two components: adding a new field to the domain model to be persisted directly. This is a case where the two appear to change in lock-step. In fact, this type of change is commonly cited as a reason to use a tool like ORM to avoid explicit mappings: if the mapping is automatic, then this change is trivial, right?

But is it?

The change in the domain model might involve updating the business rules to make use of the new data. In the persistence layer, one must worry about data which predate the addition of the field. One needs to make sure the right thing happens when persisted domain objects are deserialized into the new domain model. Does one prepopulate the data? With a default value? With a value calculated by some rule? Or should one add some trigger when loading the data so that a suitable value is set when it is absent? Does all of this satisfy the assumptions of the business logic?

These are all solvable problems, but tackling these at the same time as worrying about the domain model is a lot on which to chew.

And this is only the simplest case! What if one is reorganizing the database by moving columns between tables or moving tables between schemata? Or refactoring the domain model? Replacing an int with a CustomerId? Changing the key of a Map? Each of these should have no effect on the other side, but suddenly one needs to think about both components at once and how they interact. Any refactoring in the domain model could suddenly be a change in the persistence layer, and any change in the persistence layer could affect how data are represented in the domain model.

And one tiny mistake could lead to data corruption.

I would wager that any serious application will undergo significant changes to its domain model, persistence mechanism, or both during its lifetime. And if it doesn't, you probably shouldn't be developing it. For that means that it's a dead area: the problems have already been solved, and someone has solved them better than you will. Use their solution and be done with it.

So why use an interface?

It should be clear why we decouple the persistence mechanism from the domain model, but why do that via an interface? Why not encapsulate the persistence mechanism in a class and leave it at that? Some developers would object that an interface is unnecessary at this point.

The reason to use an interface is that a class would merely encapsulate the persistence layer, but would not decouple it from the domain model. Encapsulation is weaker than decoupling -- it tends to leak. To understand this, consider the question: what language would the public interface of such a class speak?

In order to achieve a true decoupling, the domain model must drive the public interface provided by the integration with the external service, not the other way around. When the persistence layer is a concrete class, it tends to drive its interface.

This could be visible in the code itself. Let's take a concrete example. Suppose we have a domain object Customer. We access the persistence mechanism for Customer through a class called CustomerRepository, whose public interface appears initially as follows:

class CustomerRepository {
  void persist(Customer customer);
  Customer load(CustomerId id);
}

Now suppose that some query functionality is to be added to the repository. One way to do this is to add one method for each kind of query:

class CustomerRepository {
  void persist(Customer customer);
  Customer load(CustomerId id);
  List<Customer> findByName(String name);
  List<Customer> findByCountry(CountryCode country);
  ...
}

Perhaps there is a need for more advanced queries, such as filtering by multiple criteria, sorting, pagination, and so on. If the repository is implemented by an SQL database, the temptation grows to add a method like this:

class CustomerRepository {
  void persist(Customer customer);
  Customer load(CustomerId id);
  List<Customer> findByQuery(String query);
}

The parameter query is a plain SQL statement. Now the domain model is tightly coupled to the persistence layer. A change in the persistence layer may require updating all parts of the domain model which use this method.

Even if one avoids this kind of direct coupling, more subtle forms of coupling can (and will) appear in the form of implicit assumptions about the behaviour of the persistence layer by the domain model. For example, suppose that one of the query methods is implemented so as to return the results in a particular sort order -- perhaps due to the implementation of the underlying database. The domain model simply assumes that the results are sorted in that way, but this assumption is not made explicit anywhere. A change in that sort order could cause a production defect without any test preventing it.

How to prevent this?

Using an interface to decouple the two layers is only part of the solution. It is also necessary to have a reference implementation of this interface to be used in tests. It is then a fake, a specific type of test double, which constitutes the simplest possible implementation which satisfies the contract. No external integrations, no database -- just a hash map in memory.

To ensure that the behaviour of both the fake and the database-based implementation used in production satisfy the contract assumed by the domain model, it is also necessary to write contract tests which run against all concrete implementations of the interface.

Sound like a lot of work? It's not really that much: absent some mechanism to support complex flexible queries, the fake is nearly trivial to write. And the contract tests are tests one should have in any case, irrespective of whether one has an interface or just a concrete class.

This model reverses the natural inclination towards coupling which occurs when one uses a concrete class to encapsulate the persistence layer: it becomes most natural to let the domain model drive the interface to the persistence layer, and let the persistence layer implement that implementation to serve the domain model. A change in the persistence layer is contained to that layer and has little or no effect on the domain model. A change in the domain model might imply some changes to the code which translates from it to the language of the persistence layer (i.e. serializing and deserializing domain objects), but those changes are transparent and straightforward, and have no unintended effect on the persistence format. Life for developers and maintainers becomes much easier -- over the entire lifetime of the product.

So where does YAGNI really apply?

The most common mistake developers make by which the YAGNI response is appropriate is probably premature generalization. One wants to solve a relatively straightforward problem and decides on a complex solution which can solve a far greater range of problems. Going back to the question, "But what if you do need it?" one has a simple answer: if one discovers the need, one can refactor the simple solution to a more general one later. Or one can add another simple solution to solve the next problem which comes. Or one can add a complex solution alongside the existing simple one. All of these solutions work just fine, and having settled for a simple solution doesn't hinder the introduction of another solution later. There is no risk to picking the simple solution.

If one does not decouple components which change for different reasons, then one is introducing a strong assumption into the core of the application, namely, that the two components will remain in close connection with one another for the entire lifetime of the product. And the cost of changing an assumption which underlies the architecture of an application is huge.

Of course, one can't avoid all assumptions. And one does not have a crystal ball to see what will change and what won't. But inappropriate coupling under the assumption that some components will never change -- or will only change in certain ways -- over the lifetime of an application is likely to cause a lot of pain down the road.