What's still unknown
Malcolm Gladwell’s presentation On Spaghetti Sauce was filmed in February 2004 and posted to TED in September 2006. That’s five and three years ago.
(Please note: The following text might contain spoilers, so how about taking the less than 20 minutes and watch the presentation and continue reading then?)
One of the findings he presents is this: “People don’t know what they want!”
The data he talks about comes from the food industry: Spaghetti sauce, mustard, coffee. Asking people he said, is not good enough to find out what they really want. He reports that, if asked, everyone (of the audience) would say that they like a “dark, rich, hearty roast” coffee, even though only about 25-27% would actually prefer this kind of coffee. And he says that no-one would ever say he liked milky weak coffee.
Now, I think there’s a connection to the way software is developed in a large number of places, even today: First of all requirements are gathered, then specifications are written, development coming next and eventually the software is tested (if time and budget permit). This sounds like ‘waterfall’ to me, even though it might be an improved version, be it the V-Model (widespread in Germany) or any other iterative methodology.
Reality (as I have experienced) is this: It’s not working very well, certainly not in large projects. One of the ‘problems’ are changing requirements. And the reason they do change is, I think, the same for coffee and software: You don’t know what you want, until you see something you can try, a (not too large) number of different yet similar kinds of whatever you’re looking for1.
That might well be a way to go in software development: Create a little bit of the system in a way that actually works and let stakeholders try it. Now, this of course is not a particularly new idea: Scrum does it, and other relatively new methods also do it this way.
To me there’s an open question: As you see this working in the food industry, even the automobile industry2, why is it so hard to introduce this way into software projects?
1 This is also the way design is usually created: A customer presents his ideas and requirements and the designers create a few different possible solutions.
2 Gone are the days of Henry Ford, who said “Any customer can have a car painted any colour that he wants so long as it is black”. See the wikipedia article about Henry Ford