The Hard Part of Software Development

I’ve spent a lot of time the last few weeks looking at some of the new buzz words in software development. Domain specific languages, dynamic languages, TDD, DDD, *DD, etc… Most of these ideas have definite benefits to the work of software development but I think they miss the mark on what is really hard in software.

In most projects I’ve worked on the past twenty some-odd years, the hard part is not how to architect a project, not how to tune a database and not how to eke out every CPU cycle of a function call.  In fact the hardest part has always been in understanding how a business does business. There are many names for this but I like to think of this as domain specific knowledge. It’s the time in the meeting room with the business veterans (stakeholders of one sort or another) that understand how it really works.  Whether that is how freight is shipped across country, how users register with forums or how those scanners at the grocery stores actually work; in all cases these stake holders already know how they do or want to do their job.  It is our work as developers into transitioning that into actual software.

Eric Evans talks about this in his Domain Driven Design book, but I think some readers of his book seem trapped into thinking that the magic of the domain is the top-down design of a system.  In my opinion they are missing the point.  Turning domain knowledge into a system is hard work, no matter what design methodology you use.

It’s a people problem. We engineers talk in terms of code, architecture and data.  We have our private language that helps us communicate.  If you don’t think that’s true, ask your spouse/partner what happens when they hear you talk about work with your friends. The thing is that in most industries this is true. When you talk to stake holders, they have their own language too. Our job is to pull that information from them and translate it into software designs. Often a small misunderstanding can be the cause for large changes in a system. Getting this knowledge transfer right is crucial.  For the most part we do an admirable job of it, but I think it is critical to understand how essential this is. Constant communication with the eventual users of a system is a key to the long-term success of any project.

So, if you’ve stayed with me this far we’re probably in agreement about the importance of domain knowledge. Here’s where I get incensed. So much of development these days is being done by outside developers.  The strategy of business to keep developers around has vanished. This is evidenced by off-shoring, outsourcing and contractor-based development. Companies are attempting to cut costs by using cheaper developers as well as development teams they can cut loose once a project is complete.

The problem with this strategy is that all the domain knowledge is leaving the enterprise. Companies spend a lot of money for a project and some percentage of that money is in teaching the development team about the problem domain and the way that the company does business.  By having short-term development strategies all this knowledge isn’t being recouped by the companies. I don’t think they understand yet what a bad deal this is for them.  But it’s not just bad for companies, it is also bad for developers.

In some sense we developers are part of the problem. Quitting your $75K/yr job to be hired back at $75/hr seems like a good deal, but in fact it is not a good deal for either party.  Your loyalty is to the paycheck and when you leave, the domain knowledge goes with you. This extra knowledge can help if you stay in the same industry but that’s pretty rare.  Usually you go to a new type of company and spend time ‘ramping-up’ on how they do their business. This seems bad for developers because most of the ones I know have passion for technology. They want to be part of a winning team. By being a well-paid mercenary it becomes easy to not care about what you are doing.  At the end of the contract you just move on, forcing you to divest in a personal stake. I miss that part of this business.  I have had more enjoyment about projects that didn’t work than all the mercenary positions I’ve ever held (yes, I am looking at you Gen<X>).

The dirty little secret in my house is that I’d give up my business and go work for a company if I could believe in them and believe that I was part of something special.  In some ways I think most developers are like that. But in this atmosphere of short sightedness, it just doesn’t exist.

So do I have a call for action?  No. I think that domain knowledge is an important idea that both developers and companies need to address, but I don’t have a nice and tidy solution. This is a shift that I think has to happen in software development. Both sides of the table need to look long at the last five to ten years and determine if what we’re doing now is better than before.  Does it only feel better because each line of code is cheaper to produce (mostly a product of better platforms, not better coders). I hope this can change.

UPDATE: Here’s an interesting link that has a different opinion of this subject:

http://thedailywtf.com/Articles/Up-or-Out-Solving-the-IT-Turnover-Crisis.aspx