A Reading List for Software Engineers

IMPORTANT: This post is not in its final desired form, yet. The ratings (probably a finer granularity) and categories still need some work and a lot of descriptions are missing.

In this post I would like to assemble a list of resources that I found helpful for my work as a software engineer. Since my views slowly change over time, I will try to keep this post up to date and continuously add / remove / adapt the content. To make it easier to consume this reading list, I will put the different books into different categories, potentially having a book in multiple categories at the same time. I will also provide for each book one or more ratings out of the categories essential, useful, optional and classic.

As usual, a short disclaimer: Everything I present in this post (and in this blog in general) is my personal view and does not represent the view of my current employer, my former employers or any future employers.

Overview


Code Health

Refactoring: Improving the Design of Existing Code
[Rating: essential]

Design Patterns: Elements of Reusable Object-Oriented Software
[Rating: essential]

Working Effectively with Legacy Code
[Rating: essential]

Test Driven Development: By Example
[Rating: essential]

Clean Code: A Handbook of Agile Software Craftsmanship
[Rating: essential]

Clean Architecture: A Craftsman’s Guide to Software Structure and Design
[Rating: essential]

The Art of Readable Code: Simple and Practical Techniques for Writing Better Code
[Rating: useful]

A Philosophy of Software Design
[Rating: useful]

Understanding Software: Max Kanat-Alexander on simplicity, coding, and how to suck less as a programmer
[Rating: essential]

Algorithms and Data Structures

Introduction to Algorithms
[Rating: essential]

This book is essential for building a solid foundation regarding algorithms, data structures and to some extent also computational complexity.

I first stumbled upon an earlier version of this book during my studies in computer science and really enjoyed reading it. The book itself is quite verbose, covers a large amount of topics and contains a lot of illustrations. Overall, it is a very good introductory book, as well as a valuable reference later on.

On top of that, the book can be very useful for interview preparation.

Algorithms
[Rating: useful]

Not quite as good as Introduction to Algorithms for my taste, but still a very good book.

The focus is more on practical aspects of algorithms, while Introduction to Algorithms covers the practical as well as the theoretical side. For this reason it might actually be the preferred choice for some engineers.

This book can also be very useful for interview preparation.

The Art of Computer Programming
[Rating: classic]

These books are definitely among the most famous books about computer programming of all time. Having started with this project in 1962, Donald Knuth published the first three volumes between 1968 and 1973. Volumes 4 and 5 were published in 2011 and 2015, respectively. Volume 6 is about to be published and volume 7 might conclude this compilation at some point in the future.

I personally bought volumes 1-4 at the beginning of 2017. So far I have just read very few parts. But since I added these books to my personal library my productivity miraculously increased by 2-3%.

Not an easy read, though.

Programming Languages

C++

Effective Modern C++: 42 Specific Ways to Improve Your Use of C++11 and C++14
[Rating: essential]

Effective C++: 55 Specific Ways to Improve Your Programs and Designs
[Rating: useful]

The C++ Standard Library: A Tutorial and Reference
[Rating: useful]

Modern C++ Programming with Test-Driven Development: Code Better, Sleep Better
[Rating: useful]

Python

Fluent Python: Clear, Concise, and Effective Programming
[Rating: essential]

Effective Python: 59 Specific Ways to Write Better Python
[Rating: useful]

Java

Java: The Complete Reference
[Rating: useful]

Go

The Go Programming Language
[Rating: essential]

Tools

Practical Vim: Edit Text at the Speed of Thought
[Rating: essential]

The Linux Command Line: A Complete Introduction
[Rating: useful]

Docker Deep Dive
[Rating: useful]

Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming
[Rating: useful]

Pro Git
[Rating: useful]

Mercurial: The Definitive Guide
[Rating: useful]

Software Craftsmanship

Coders at Work: Reflections on the Craft of Programming
[Rating: useful]

97 Things Every Programmer Should Know: Collective Wisdom from the Experts
[Rating: useful]

97 Things Every Software Architect Should Know: Collective Wisdom from the Experts
[Rating: useful]

The Passionate Programmer: Creating a Remarkable Career in Software Development
[Rating: useful]

Effective Programming: More Than Writing Code by Jeff Atwood
[Rating: useful]

How to Stop Sucking and Be Awesome Instead
[Rating: useful]

A Programmer’s Rantings: On Programming-Language Religions, Code Philosophies, Google Work Culture, and Other Stuff
[Rating: useful]

Software Developer Life: Career, Learning, Coding, Daily Life, Stories
[Rating: useful]

The Pragmatic Programmer: your journey to mastery
[Rating: useful]

Processes

Clean Agile: Back to Basics
[Rating: useful]

The Clean Coder: A Code of Conduct for Professional Programmers
[Rating: useful]

Becoming a Technical Leader: An Organic Problem-Solving Approach
[Rating: useful]

Debugging Teams: Better Productivity through Collaboration
[Rating: useful]

The Dip: A Little Book That Teaches You When to Quit (and When to Stick)
[Rating: useful]

So Good They Can’t Ignore You: Why Skills Trump Passion in the Quest for Work You Love
[Rating: useful]

Crucial Conversations Tools for Talking When Stakes Are High, Second Edition
[Rating: useful]

Crucial Accountability: Tools for Resolving Violated Expectations, Broken Commitments, and Bad Behavior
[Rating: useful]

The Healthy Programmer: Get Fit, Feel Better, and Keep Coding (Pragmatic Programmers)
[Rating: useful]

The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change
[Rating: useful]

Getting Things Done: The Art of Stress-Free Productivity
[Rating: useful]

Great at Work: The Hidden Habits of Top Performers
[Rating: useful]

Classics

Code Complete: A Practical Handbook of Software Construction, Second Edition
[Rating: classic]

The Mythical Man-Month: Essays on Software Engineering
[Rating: classic]

Code Health is not a Trade-off

It is a popular perception in software engineering that there exists a trade-off between code health and delivering software fast. This is a huge misconception fueled by the fact that we very often maneuver ourselves into such a situation, where we suddenly have to decide between improving the quality of our codebase and delivering new features.

Yes, this trade-off is real and I have experienced it myself more often than I would have liked to, but the underlying problem is that we allow this trade-off to materialize in the first place without actually getting anything in return.

In the remainder of this post I will look into some of the reasons that create such trade-off situations, how we could avoid them and what we could do to get out of them.

Before going more into detail, a short disclaimer: Everything I present in this post (and in this blog in general) is my personal view and does not represent the view of my current employer, my former employers or any future employers.

Accumulating Technical Debt

Having a large amount of technical debt is the reason why we experience this trade-off between code health and delivering features fast. So why are we getting into this vortex of technical debt? There are three reasons for that.

First, we very often incorrectly perceive that by going into technical debt we are able to gain short-term advantages which potentially outweigh all the long-term downsides. But in software development this is just not possible. Alright, there are a few notable exceptions here, like the one-off script, the small prototype or the feasibility study. But even in those cases, how often did we need our one-off script more than once? How often did our small prototype grew over time until it made it into production? How often did our feasibility study require time-consuming debugging until it gave correct results? I do not want to claim that those exceptions do not exist, but they are extremely rare. In all other cases, there are no significant short-term benefits from going into technical debt. Technical debt hits us much faster than we expect, even for single-person short-term projects, but more so for multi-developer projects with a long lifetime.

Second, we sometimes just do not know how to do it better. Each developer has a specific set of skills and experiences. Even by doing our best, we might immediately introduce additional technical debt. And this is one of the reasons why I personally do not like the term technical debt. While we go into technical debt consciously, but based on false premises, in some cases (first point), we mostly go into technical debt accidentally (this point and the next one). And this is not how you typically get into other forms of debt.

And third, even in cases where we do know how to implement things perfectly, we get it wrong. Actually, we get it wrong all the time. And not because we are missing skills or experiences, but because we are not able to predict the future. We are implementing something in software (and not hardware), because we expect the implementation to change over time. Even in cases where our current implementation is perfect (whatever that means, but this is a different story), it might be far from ideal in one month from now or in half a year. And that means that we get additional technical debt just because the context (e.g. constraints, requirements, scale, etc.) changes over time. Again, the term debt does not seem appropriate here.

How to avoid Technical Debt?

Looking at the three different ways in which we are adding technical debt, how can we best avoid it? First of all, lets not get lured into consciously adding technical debt by the false premises of short-term gains. In most cases, this is not what we get back in return. So what about the other two cases in which we accidentally add technical debt?

We can reduce the amount of technical debt that is added because we were not able to find better ways of implementing or designing a feature by getting better ourselves. Continuous learning and improvement is something I am very passionate about. It takes time, but there are plenty of resources (books, videos, courses, etc.) available that could help us to avoid repeating the same mistakes we (or other developers) did in the past. Those are techniques like test-driven development, pair programming, code reviews, refactoring, patterns and many more. I always wanted to compile a list of useful resources and might be able to add it in the next days, so you have some starting points.

What about the technical debt that is added because the context of our project has changed? Well, we can easily avoid this by getting better at predicting the future. Just kidding. While we might get better over time about making reasonable predictions of the future, this will never be sufficient to avoid this issue altogether. What we can do is to accept that we will get things wrong and to design the system with that in mind. This is not easy at all and in my opinion one of the most difficult things in software development. But by designing the system for change, by deferring decisions and by keeping the codebase simple and modular, we can mitigate a lot of technical debt as soon as it occurs.

Whatever we do, we will sooner or later accumulate technical debt. Even a team with the most experienced and skilled engineers will have to deal with technical debt. The important thing here is to continuously deal with it and to reduce it as soon as it occurs. As long as we are able to keep technical debt below a critical threshold (and this threshold is not very large), we are not getting into the situation where we have a trade-off between code health and feature development. This is what we should aim for. But what if we are above that threshold?

How to reduce Technical Debt?

Working in a project with a large amount of technical debt is not a pleasant situation to be in. As I said before, this is something we should try to avoid from the very beginning. Don’t get into such a situation.

But what if it is too late or if you just joined a new project which is in an unhealthy state? It can be extremely stressful to work in such an environment. If that is the case for you and if the situation is not improving, I would recommend to look for other opportunities and to not remain in that situation for too long.

In any case, if we are in such a situation, we basically have to deal with the uncomfortable trade-off between code health and feature development. That means there has to be someone in the decision chain which supports reducing technical debt. If that is not the case, I doubt that such an endeavour could be successful and would like to refer to what I said in the previous paragraph. If, on the other hand, reducing technical debt is a priority, it will still be a very difficult task, but there are a few things we can (at least try to) do.

Replacing the whole system is most of the times very risky and something I would generally advice against. Instead, I would try to reduce technical debt more locally. I would start by improving testability and test coverage in order to increase the confidence in the codebase and to make further changes easier. Being able to change code with confidence might allow us to modularize some components and decouple them from the rest of the system. Replacing such components with better implementations does not come with the same risks as doing it for the whole system and is something that worked very well for me in the past. Apart from that, as a general rule we could try to simplify any code we touch for maintenance or for feature development. This allows reducing technical debt while developing new features and might slowly move the whole project into a more healthy state (where other changes might become feasible).

All in all, this is not easy. I am not sure if I mentioned that before, but the whole point of my post is to avoid getting here in the first place. And if you end up here, then good luck.

Conclusions

There is no fundamental trade-off between code health and delivering features fast. We experience this trade-off because of a large amount of technical debt. Being in such a situation is very unpleasant and significantly slows down the progress of projects. In the extreme, it makes projects fail.

And this is why we should be mindful about technical debt, the reasons why we accumulate technical debt, how to reduce technical debt and most importantly, how to avoid adding technical debt in the first place.

This is easier said than done and I hope I will find the time to get into more concrete examples here in this blog. Please let me know if there are any specific topics you would be interested in.

Scrum is fragile, not Agile

As the title suggests, this post is about two different aspects of Scrum. The first part deals with Scrum not being Agile and the second part is about Scrum being fragile.

Before going more into detail, a short disclaimer: Everything I present in this post (and in this blog in general) is my personal view and does not represent the view of my current employer, my former employers and any future employers.

Scrum is not Agile

I guess a typical reaction to this heading would go like “How is this possible? Scrum is not Agile? Isn’t Scrum the number one Agile software development process?”. The short answer is that Scrum claims to be an Agile process, but the sad reality is that Scrum is quite far from being Agile. I will show you why.

Lets have a quick look at the Agile Manifesto. It states that it values “Individuals and interactions over processes and tools”. Lets also have a quick look at the meaning of the word agile. According to the Oxford Dictionary agile means “Able to move quickly and easily”. It is not a coincidence that the term agile has been chosen to represent the high-level ideas within the Agile Manifesto. In fact, one major point behind Agile is that in many software projects it is extremely difficult to move quickly and easily. This is not the case for a completely new project, but over time many projects get into a situation where sustainable development is simply not possible anymore. To prevent this (and other issues), the Agile Manifesto and the Principles behind the Agile Manifesto provide several high-level guidelines. These guidelines are not specific well-defined processes or tools and they allow for many different implementations. I suspect that both of these properties (high-level and allowing different implementations) were fully intended. The overall goal was not to present a silver bullet, but to help peers to avoid many of the pitfalls in software development, which the authors of the Agile Manifesto experienced first-hand and which fall into exactly these categories.

Now lets have a look at the Scrum Guide (written by two of the authors of the Agile Manifesto). In contrast to the Agile Manifesto and the Agile Principles, this guide seems quite lengthy. Surprisingly, the whole guide does not mention Agile a single time. I am not sure if this was historically always the case, but if the authors of the Scrum Guide do not claim that Scrum is Agile, then we would already be done with the first part of this blog post. I assume that this is not the case, so lets move on. The Scrum Guide is about a framework which contains “roles, events, artifacts, and the rules that bind them together”. In other words, it is a very specific and well-defined process. This does not sound agile and it also does not sound Agile (remember: “Individuals and interactions over processes and tools”). This is quite ironic and obvious. And this is where the whole Scrum movement should have stopped. But it did not and instead frustrates an increasing number of software developers all around the world. And whenever a Scrum project fails, it is not because of Scrum’s potential flaws, but because Scrum was not implemented correctly. That sounds like a nice transition into the second part of this post.

Scrum is fragile

This part is very short. I thought that the wordplay (Scrum being agile / fragile) is kind of funny and apart from that it perfectly describes one of the things that really bother me about Scrum: Whenever a Scrum project fails, it is because Scrum was not implemented correctly. And you can read about a vast amount of such projects. What does it mean, if a large number of intelligent software developers are not able to implement Scrum correctly? It means the whole framework is fragile. And this is another major argument against using Scrum. What is a framework good for, if it is so difficult to use?

Well, it seems that with the help of expensive consulting and coaching, as well as training and certificates, Scrum might in fact provide value. But it is not clear if this is value for the companies developing software and the hard-working software developers or for those who offer services in and around the Scrum ecosystem.

Personal View

I would like to finish this post with a bit of my personal view regarding software development, Agile and Scrum. To me it seems that one very important part of high quality software development is to maintain a simple priority queue of tasks. The weight is a combination of the value a task provides for the customer / developers and the estimated effort to implement this task. For some developers this comes naturally. For teams and companies for which this is not the case, Scrum offers a rather expensive and inefficient implementation of a priority queue.

And lets be honest. Software development is a very difficult and complex work. Are we really surprised that so many projects fail? The field is still very young and we need to learn a lot. And this is crucial: We need to learn from past experiences, let it be failures or success stories. And here we collectively fail. We are not using the wrong processes or implementing the right processes in the wrong way. We are simply caught in a rat race and not able to make a short break in order to look at and learn from all the things that happened around us, maybe even before our time. It is our duty to extract the knowledge, the experiences and the wisdom from the many resources that are so easily available to us: The many many books, articles and videos about software development and, last but not least, the Agile Manifesto.