The Dependency Injection Hype Gets it Wrong
I spent about 10 years working in C#, where OOP, SOLID, and Dependency Injection are all entries in the religious texts. It took me a long time to grasp these concepts and apply them effectively, but when I did, it was like the puzzle pieces started to fit themselves together. I then followed it with 3 years immersed in the Zen of Python and haven’t touched much .NET at all, except to Google things like “how to do LINQ in Python?”. While I love both languages, the respective communities seem to be on opposite ends of the spectrum on things like Dependency Injection. Recently, I’ve spent a bit of time (re)training on .NET and C# in case I have to use it for something productive, and the first topic I decided to hit was Dependency Injection (DI) in .NET 6.
What I immediately experienced were all of the wrong reasons why dependency injection is important. The first example given is a situation where we’ve written a simple app of 3 classes (MVC View, Business Class, and Data Access) and all of a sudden we have a need to switch from SQL Server to MySQL. I think this comes across as a highly unlikely scenario, and misses the opportunity to advertise the greatest values that disciplined DI offers. Ignoring the potentially prohibitive licensing costs of SQL Server (or at least what I knew of them 3 years ago) the persistence layer is the component least likely to change during an application’s lifetime.
If we haven’t lost the physical attention of our potential converts, we’ve at least lost their faith that their coming moments are going to be worthwhile. A suspicious and inquisitive student is good, but the chances a cynical student will gain any value from a training are very low.
The Metrics Aren’t the Goal
One of the more contentious areas of software development is time and effort estimation. I’ve spent countless hours in grooming and refinement meetings where the ultimate goal of the team is to assign one form of estimation to a body of work: story points, a t-shirt sizes, a timebox, or, at worst, hours. Over the course of these meetings, it’s a near certainty that a member of the team will be dissatisfied with the act of estimation — whether it’s an engineer feeling like story points are a meaningless fiction, or the product manager annoyed that 10 small tasks (at ~2-4 hours each) weren’t all completed in someone’s 40 hour week.
These complaints are missing the value of the grooming activity, and focusing on the physical output rather than the journey. The longer I’ve been engaged in building systems, the more I know that task estimates are rarely valid building blocks for timelines, even though I do still fall victim to that trap. The value is in the mental process, and the grooming sessions are forcing functions for each member of the team to ensure that risks are identified, potential functional scenarios are discussed, and that the groomed work represents the best documentation of the collective knowledge the team can compose around a given task or requirement. When I’m asked “how much time do we need to complete this?” I try to fast forward through the film that is testing, then building, then testing again, and then delivering a particular body of work. That activity helps me, and everyone else that’s willing to give the exercise a little bit of faith, mentally enumerate the necessary steps, and miss plenty of them along the way. With a healthy enough team, most of the time, most of the gotchas and risks are identified, and it is mostly helpful to whomever ends up working on the task. This is what successful grooming feels like.
Thinking back to dependency injection’s purported valuable metrics: Low-coupling, better reusability, easier testing, improved readability. Those all sound like good things, but the suspicious side of me wants to know why they are good things. Each of them, when pushed to their extremes, can become counter productive — and we see this in the ranting tirades against OOP and SOLID from the potential converts we lost along the way. The most recent one I read declared that Java would require 1,000+ classes to replace a handful of lines of Lisp, and that anything that increased the number of lines of code is a decrease in quality. 😐
These are all metrics that may be used, in conjunction with one another, to describe some characteristics of a particular code base. We can measure afferent and efferent coupling, code reuse (or duplication), test coverage… and for readability, maybe at least adherence to a standard formatter for the given language like Black for Python. None of those tell us about the suitability of a codebase for a given set of use cases, nor do they help us ascertain whether the codebase is sufficiently adaptable to be a cornerstone asset of an organization’s journey into the future. The pattern of DI is a forcing function that asks the developer to think of things like:
- What information is on hand when I need to invoke this function?
- What do I really need out of this function?
- Do those things still make sense if I replace the inner implementation of that function?
If we don’t feel good about our answers to these questions, the chances are high that the abstraction we’re conceiving of is poorly conceived. It may be mixing implementation details in with business concerns, or isn’t logically cohesive with other functionality in the component where it’s being added. It may be too fine-grained, which ends up requiring too much orchestration. One concern we can likely defer is if there is a seemingly high degree of complexity to transform the inputs into the outputs — handling that concern is a problem for subsequent implementation, which itself can also be decomposed, or even better, deferred to emergent architecture.
And lastly on metrics, the idea that adding things like interfaces and DI containers to an app that previously had one or two large functions “simplifies” anything will bring more than a chuckle or two. It may in fact reduce the cyclomatic complexity, but it will also dramatically reduce human comprehensibility. That is, until it’s compared to the solution that actually demands such DI frameworks and multiple implementations of interfaces that has not undergone such rigorous treatment but instead been left to meander on its own.
Sales, not Training
Training exists to equip the motivated with the techniques and awareness to pursue their motivations, but training is generally not motivating itself. I enjoy learning new languages and tools to solve new problems, but I find myself frequently glazing over or annoyed at the pace of both instructor-led and self-guided training. If anything, the training is demotivating, and too much of it will snuff out the innate motivation that I may have entered with. I need to have enough motivation to solve the problem in order to get through the training I need.
The examples we often use in training are highly simplistic, idealized versions of what we encounter in the real world. Without any hard data on hand, I’d venture a guess it’s relatively rare that so many of the problems we’re solving day to day fit neatly into one Model, one View, and one Controller. These relatively simplistic patterns and categories are excellent for training because they give us a low barrier to entry to some hands-on exercise with what may be a new concept, or maybe one we’re brushing up on. What they are not is a sales pitch to convince anyone why they should care. I doubt the strength of my own motivation to build yet another Hello World or To-Do List demonstration app for anything that I’m not already convinced is going to be valuable to me. The convincing has to be done before the training. For that, we have to talk as sales people!💸
Don’t worry… not really (sorry, sales friends). But, what we do need to think about is how to convince someone to engage their finite time and energy in learning, and trusting in something that is currently unknown to them and that they are rightfully suspicious of. For that, we need to empathize with the highly complex problems, or the long running solution-less problems, that our audience is likely struggling with.
Take our simple MVC Hello World app and fast-forward it 5 years, 4 company re-orgs, and 2 microservice slice-offs. Without disciplined application of DI, this app has likely turned into a nested smorgasbord of development patterns, and it may even be the second or third time our audience has seen this (see: it’s okay, every company is mess). Instead of pitching left-field use cases like swapping SQL database technologies or lists of code metrics, we could instead ask our audience things like:
- How easily could you handle the deprecation of a built-in hashing function in your chosen database? (SQL Server 2016 actually did deprecate some of the older hashing functions)… this one is probably not too bad. My guess is not many teams are hashing in the database engine.
- How do you know when to add a new function to an existing namespace/module/class or create a new one?
- How easily could you switch from saving configurations (which usually contain secrets!) from an app server file system to something like HashiCorp Vault or AWS Secrets Manager?
- How much confidence do you have in a given build of your application if you only run the tests that don’t require real connections to other services/databases?
These represent some real-world problems an audience member may have encountered, or at least silently worried about. All we really need is to strike one topic that is top of mind in an audience member to keep their attention. The last one is an enormously important leading indicator on whether the audience member’s team is following a whole host of good CI/CD practices.
Single-Responsibility is not Single-Function
SRP and DI go hand-in-hand, but one of the common criticisms of SRP is the explosion of the number of classes, and probably the naming conundrums that follow. The secret is that adhering to the single-responsibility principle doesn’t mean we have to create a new file and class every time we create a new function. Perhaps a slightly more helpful word than “responsibility” could be “role”.
I struggled to come up with a good analogy for this, but living in NYC, there are these mythical “doorman buildings” that I’ve never lived in. A doorman has many responsibilities and performs many tasks. He or she may greet people, act as a form of security, assist with hailing cabs, holding doors, receiving packages, and the list goes one. It’s probably a really hard job, and likely requires immense patience. But the reason why I mention it is that some smaller buildings have one person doing this job a time, and larger buildings employ a whole team. A building with a couple dozen apartments likely does not have a need for a team of many people to handle the expected day to day operations. On the other hand, the largest buildings divvy up the work amongst many people working each day, many of whom have designated responsibilities, making each role fairly distinct, and dependent on the others to offer a complete level of service to the building.
Coming back to software, which certainly grows faster and less predictably than an apartment building… when our app is young and simple, we have no need for 10–layer deep dependency tree because all we’re doing is writing Hello World (maybe 30 times, if we’ve added a loop). When we have to add new functionality, we can consider things like whether it’s going to end up using the same dependencies as another class that already exists, if it’s going to be used in the same areas of the app that the existing class is already being used, and is conceivably logically coherent with that class’s current role? If so, add the function to the existing class and re-leverage your asset!
If and when we ever need that functionality, or something close to it, in more than one place, that’s the time to move it into its own construct, and where we start discussing emerging architecture. If the domain is relatively unknown, it may even be wise to incur some algorithmic duplication, wait until a third similar requirement enters, and then the existence of three use cases will be a good demonstration of what is truly shared and therefore worthy of being represented in an abstraction. One of the best features of a DI-based application is that each class clearly documents its dependencies… similarly to reading a function’s signature, reading a class’s constructor gives us a manifest of requirements without having to look at any of the implementation.
The valuable part is having a relatively good sense of when and where that decomposition is going to take place and ensuring it’s not haphazardly based on the developer’s mood that day or some anxious desire to future-proof against use cases that never come. Real use cases on hand, coupled with a system that partially meets them, is the best set of guidance you could ask for when adding complexity. And don’t doubt that decomposing and refactoring usually adds complexity before it reduces it to a manageable state.
Yeah, you can still use “new”
Right after the SQL Server to MySQL migration, the other common idiom is something like: “using the new keyword is bad” (or to my Python friends, that directly calling your class’s constructor / __init__ is bad). Surely defining a class is only valuable if you can instantiate it. How could it be that installing a third-party DI container library to call new for me is okay, but I can’t do it myself? What’s good for the goose is good for the gander?
Of course we can write new in our post-DI code, but we should be wary of what we’re instantiating. Is the object responsible for making external service calls or implementing functionality for one of many scenarios of a feature? If so, it should probably be injected. If the object is a pure message class to hold some data, with no behavior, go ahead and run new inside of while (true) all day.
It’s a helpful practice to divide up our class archetypes into two main categories. “Actor classes” — classes with behavior, and no data other than their injected dependency references — and “message classes” — immutable objects that represent input or output at process boundaries and across abstractions. Of course there will be instances where an actor class may track some small amount of state (eg. batching records to send to an external service) and message classes may have small amounts of behavior (eg. defining a property that combines or projects some of the underlying data fields). But, if we stick to this distinction, it becomes really clear which types need to be injected, and which types represent our API contract input/output and can be new-ed up as needed.
Preparedness is the real value
There is likely limited value in practicing disciplined DI, SOLID, and the other things in solving our problems we have on hand today. It’s always going to be easier, and simpler, to naively brute force the new functionality into the system, as if we’re sprinting to the end of a race. If we really do think that it is the end of the race, that this is the last and final feature we’re adding before we shrink wrap the system and let it run on its own forever, then it really doesn’t matter how we structure the solution.
As much as timeboxed sprints and standardized release cadences might encourage a mentality of getting to the end of the race, the reality is that there is almost always another iteration coming up. The value is not to be found today, but to be found over the aggregate of months’ and years’ of sprints — the right balance of not trading off technical quality with immediate functional goals. It’s rare we know what is going to be asked for next week or even tomorrow, or even if we have the right understanding of today’s requests. The value, the problem to solve, is to be able to calmly and coolly react, accepting — even embracing — the changes in features based on new information or feedback.
When we do encounter those inevitable changes, our clean boundaries, like bulkheads on an ocean liner, between components tell us:
- The area(s) that will require modification to achieve the goal
- The adjacent area(s) that will require modification, or at least regression testing, to complement
- The area(s) that we can choose to defer modification of, usually at some cost of reducing availability of new functionality, or adding in some additional logic to adhere to existing interface contracts
- The area(s) that have almost no chance of impact, and we don’t need to give too much thought to beyond ensuring our automated checks pass
The real value is not in achieving some idealized code structure state, but rather achieving an idealized level of human focus on the areas undergoing change.
References
[1] Tim Peters. “PEP 20 — The Zen of Python”. 2004. https://peps.python.org/pep-0020/
[2] Wikimedia Foundation. “Model–view–controller”. https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller
[3] Python Software Foundation. “Black”. https://github.com/psf/black
[4] Microsoft. “Deprecated Database Engine Features in SQL Server 2016”. https://docs.microsoft.com/en-us/sql/database-engine/deprecated-database-engine-features-in-sql-server-2016?view=sql-server-ver15
[5] Hashicorp. https://www.vaultproject.io/
[6] Amazon. https://aws.amazon.com/secrets-manager/
