Posts Tagged ‘Mass production of software’

We have discussed  craft software development in several previous posts. While I personally like software craft production methods, as a software architect I want to understand how to make software development more efficient in mass production environment.

Software mass production has several distinctive characteristics:

  1. Developer working in such environment faces large amounts of code she/he is not familiar with; often it is legacy code written years or decades ago by somebody who is not available anymore. There is no much time to learn the code; the developer must become productive on that code very soon.
  2. One can’t assume that a developer working on the code will have skills level higher than average; having a developer with low skills must not be a risk.
  3. Cost/time to market considerations are of paramount importance.

The first point means that developers often are not familiar with the code they are supposed to work with and have very little time to learn it. This is a big problem, but there are ways to alleviate it.

The code should be broken into relatively small modules with very clearly and explicitly defined boundaries (even at the level of code repository) and contracts between them. My observation is that time necessary to learn unfamiliar code grows at least as square of the code size as the latter increases. Having it broken into well encapsulated modules helps to reduce the amount of code that has to be learned.

A better design also helps to understand the code (see here) as well as other basic techniques.

All necessary elements of the code must be explicitly described by the code. You think it is always the case, do you? Not at all. Think of dynamic languages –  for those languages type definitions that are important parts of the code (code metadata) do exist in developers heads only and never expressed/documented explicitly. Or think of implicit agreements that are used in Perl. You can find much more examples if you look around.  While such things may be not a big problem when a developer knows the code she/he works with well, this is a big problem in software mass production and as such must be eliminated.

The second point means that the code base and the tools must be such that even a beginner developer will produce a code of good quality.

While having code modular and explicit definitely helps, I think this is more about proper tools. Think, for example, about differences in how strings are defined in C++ and Java. While C++ strings caused countless problems, a way strings are defined in Java just eliminated completely the type of issues.

Basically for software mass production we have to choose deliberately tools that do not allow common problems to happen or at least warn about possibility of them happening. We better help the developer to avoid problems than rely on her/him to fix them.

This is not just about software languages. Think, for example, about differences between Maven and Ant. Ant is a sort of free-form tool while Maven is much stricter and prescribes certain ways of doing things. Maven is obviously better suited to software mass production.

The third point is of paramount importance for software mass production while it is of little to no importance to open source and lesser importance for software craft production.

Let’s suppose your company has a problem it wants to solve by developing certain software. There are two options: a technically excellent solution (architecture/technology/design etc) and just satisfactory one. Which one would you choose? The first one? Not so fast – you haven’t yet considered all important inputs for the decisions. For software mass production we have to consider cost and time (this is business after all).

Let’s suppose the first solution will cost $10M and it will take 5 years to implement while the second one will take 9 months and will cost $0.9M. Now advantages of the first solution are not so obvious, right?

Let’s add one more input… Your company has just $1M budget for developing the solution and if it is not available within a year, your company will be out of business. Given this, which solution is better?

Cost/time considerations are important for all choices we make in software mass production, not just big ones. They are exact reasons why developers working in software mass production environments e.g. should use modern IDEs as opposed to just vim or another text editor, should use object-oriented languages as opposed to procedural ones (the procedural ones are 5 times more expensive to develop – see here), etc, etc.

Probably, there are more issues that should be considered if we want to have efficient software mass production. But those three look to me as the critical ones. Disregrad of any of them will likely cause many problems.


Read Full Post »

As we discussed in earlier posts from this series (the first one is here, the previous one is here), while it is possible and practical having small software development teams doing software craft production, it is impossible to have a large software craftsmen team existing for a prolonged period of time. Well, maybe not impossible, but this would be a rare occasion.

By all means IT industry as a whole nowadays have no choice but to produce software while employing average developers. Since developers with average job market skills and expertise are not capable of software craftsmanship, some other ways to produce software of satisfactory quality should be used.

This situation is not unique. Many industries have faced such problem in the past. Cars, airplanes, radio sets, toothbrushes etc, etc  were craft produced initially. However at some point the demand outgrew capacity of craft production and mass production methods were employed. Mass production in a broad sense can be defined as:

Mass production is the name given to the method of producing goods in large quantities at low cost per unit. But mass production, although allowing lower prices, does not have to mean low-quality production. Instead, mass-produced goods are standardized by means of precision-manufactured, interchangeable parts. The mass production process itself is characterized by mechanization to achieve high volume, elaborate organization of materials flow through various stages of manufacturing, careful supervision of quality standards, and minute division of labor.

There is an important difference between  mass production in general and mass production of software.  Normally mass production results in large quantities of near identical products. Mass production of software results in a large number of substantially different products (I’m not talking here about printing identical installation CDs, I mean writing code).  To be precise, I should have used term job production for this type of mass production; but  in my opinion it obscures the meaning and I prefer to use less precise but more explicit mass production term.

IT industry started facing necessity of software mass production at the beginning of 1960s. This was a time when new more affordable computers were introduced (e.g. IBM System 360 and DEC PDP-8 models). Until then computers were rare and software was relatively simple, therefore there were enough folks capable of software craftsmanship to fulfill the demand. But at that point computers became affordable for various businesses  and demand for new software development exploded.

Attempts to apply old craft production methods to software mass production resulted in many unpleasant surprises and failed projects. One of the early analysis of the problem could be found in a classic book The Mythical Man-Month by Fred Brooks. Mr Brooks wrote this book based on his experience as a  development manager in charge of OS/360 – operational system for IBM System 360 computers.

Many different ideas and “panaceas” were introduced over time, many of them all but forgotten (although many have become a part of usual practice). For example, who remembers now hot debates around structured programming  (1960s)and almost religious wars that were waged  over use of GOTO statements? Noticeably, those ideas were introduced first in the same time period (1960s) when IT industry encountered the challenge of software mass production.

Still, I think that situation didn’t improve much until at least 1995-2000 time period. It is very indicative that Tome Love and Richard Wiener wrote in their book Object Lessons written at around 1995: “The successes come from craftsmen, the failures from engineers!” (p.20) which I interpret as they saying that software craftsmen deliver while software mass production fails. My observations support this and I believe that attempts of software mass production were failures rather often until approximately year  1995 – 2000.

One may say that big software products/systems were successfully developed before 1995-2000. True. The point is that failures happened frequently, more frequently than it is acceptable (see Object Lessons for examples).

So what changes happened at around 1995-2000? We’ll discuss this in the next post.

Read Full Post »