How bad is SLOC (source lines of code) as a metric?

code-metrics

16,932

Solution 1

Someone said :

"Using SLOC to measure software progress is like using kg for measuring progress on aircraft manufacturing"

It is totally inappropriate as it encourages bad practices like :

Copy-Paste-Syndrome
discourage refactoring to make things easier
Stuffing with meaningless comments
...

The only use is that it can help you to estimate how much paper to put in the printer when you do a printout of the complete source tree.

Solution 2

The issue with SLOC is that it's an easy metric to game. Being productive does not equate to producing more code. So the way I've explained it to people baring what Skilldrick said is this:

The more lines of code there are the more complicated something gets.
The more complicated something gets, the harder it is to understand it.
Before I add a new feature or fix a bug I need to understand it.
Understanding takes time.
Time costs money.

Smaller code -> easier to understand -> cheaper to add new features

Bean counters can understand that.

Solution 3

Show them the difference between:

for(int i = 0; i < 10; i++) {
    print i;
}

and

print 0;
print 1;
print 2;
...
print 9

And ask them whether 10 SLOC or 3 SLOC is better.

In response to the comments:

It doesn't take long to explain how a for loop works.
After you show them this, say "we now need to print numbers up to 100 - here's how you make that change." and show how much longer it takes to change the non-DRY code.

Solution 4

Solution 5

I disagree on SLOC being a bad metric. It may be moot to go into a years-old question with eleven answers, but I'll still add another.

Most arguments call it a bad metric because it is not suited to directly measure productivity. That is a strange argument; it assumes the metric to be used in an insane way. With this reasoning, one could call the Kelvin a bad unit because it is unsuited to measure distance.

Code length is a viable measure of ballast.

The amount of non-comment code lines correlates with:

undetected errors
maintenance costs
training time for new contributors
migration costs
new feature costs

and many more similar kinds of costs, like the cost of optimization.

Of course SLOC count isn't a precise measure of any of these. Code can be anywhere between very nice and very ugly to manage. But it can be assumed that code length is rarely free, and thus, longer code is often harder to manage.

If I were managing a team of programmers, I would very much want to keep track of the ballast it creates or removes.

View more solutions

16,932

Bob Cross

Computer scientist working at [A Facility] developing [Systems] that [Do Stuff] at [Places].

Updated on June 11, 2020

Comments

Bob Cross about 4 years

We are documenting our software development process. For technical people, this is pretty easy: iterative development with internal milestones every four weeks, external every 3 months.

However, the purpose of this exercise is to expose things for our project management in terms that they can understand. Specifically, these non-technical managers need metrics that they can understand.

I understand our options for metrics well and have proposed a whole set (requirements met and actual costs vs. budgeted costs are two of my favorites). However, we do have some old hands involved and they tend to hang onto metrics like SLOC.

I understand the temptation of SLOC: it seems easy for non-software people to understand and it seems like the closest analog of a physical thing (it's just like counting punched cards back in the old days!).

So here's the question: how can I explain the dangers of SLOC to a non-technical person?

Here's some concrete motivation: we work on a fairly mature deployed system that has years of history behind it. As we add features, SLOC tends to stay approximately level or even decrease (refactoring removes old / dead code, new features are really just adjustments of existing, etc). To a non-programmer manager, a non-increasing SLOC in a development project is perplexing at best....

Clarifying in response to a recent answer below: remember, I'm arguing that SLOC is a bad metric for the purposes of measuring project progress. I'm not arguing that it is a number that's not worth collecting. It requires extensive context to do anything useful with it and most program managers don't have that context.
- relascope about 9 years
  
  if you want to put in a nail somewhere, do you count the hammer strokes?
- Kzqai over 8 years
  
  "Can you tell how good a book is by the number of pages?"
- industryworker3595112 about 8 years
  
  SLOC very good metric!! ...for how bad stuff is. The more SLOC you have, the worse position you are in.
- Vincent Cantin about 7 years
  
  Just tell them that "1 line of code" is worst "2 lines\n of code".
- Oloff Biermann almost 4 years
  
  I know this is an old question but I think it is still highly relevant! The more abstraction we get with frameworks, the more pointless SLOC becomes for measuring productivity.
Bob Cross almost 14 years

I totally agree that test cases and coverage are better. That said, I actively refuse to allow measurement of individual people for project management people. We're a team and that sort of thing stays inside the team.
mkoistinen almost 14 years

I would agree with you that the first case is better, but the 'non-technical' person might agree that the second case is. And, they might be right, the second case will compile/run faster and require less memory =)
Bob Cross almost 14 years

I agree that you and I both look at this example (or similar on Wikipedia) and say, yes, of course. However, non-programmers see "gibberish" and "more gibberish". It's sort of like using a word in its own definition....
Bob Cross almost 14 years

I had heard that comment before but don't know the citation. Do you have a reference for it?
Bob Cross almost 14 years

in a modern language, I feel that SLOC is too fine-grained. Example: Findbugs reports bugs per Java class and package. THAT is more useful - we can look at a list of classes and say that "this is the worst class / package" and we should organize a focused effort. Bugs per line of code is hard to take action on - what should we do?
Bob Cross almost 14 years

sure, it's easy to see Dilbert in everything if you try to. That said, we can't just say "all metrics are dumb." We have to suggest something better. That said, SLOC definitely IS dumb.
Bob Cross almost 14 years

okay, that 5 step process is actually quite compelling. Thanks.
MaddTheSane almost 14 years

“Measuring programming progress by lines of code is like measuring aircraft building progress by weight.” — Bill Gates
Hammerite almost 14 years

You could edit the strip to remove panels 3, 4 and 5. Then the message would become less "metrics are dumb" and more "ill-considered metrics are dumb", and the analogy between SLOC and PHB's "number-of-words" metric would remain clear.
MaddTheSane almost 14 years

We'll have to subtract panels you delete.
Ken Henderson almost 14 years

Whitespace (extra lines) and/or comments are not counted as part of SLOC.
Peter Tillemans almost 14 years

@Bob According to an article I found online it was Bill Gates : dev102.com/2008/09/09/…
Jörg W Mittag almost 14 years

@Bob Cross: I think it is usually attributed to Bill Gates. (Which is kind of ironic, considering what bloated monsters Windows and Office are. Windows is around 50 million SLOC, Office around 270 million. That's over 300 million SLOC, using all the tools 2010 has to offer, to achieve pretty much the same thing Alan Kay and his group did in 1976 with 60 thousand SLOC.)
Eugene Mayevski 'Callback almost 14 years

this is not something non-technical staff needs to know.
Bob Cross almost 14 years

I agree that you and I would understand that length != better. However, for non-technical people that are used to projects that need to produce very long cables (for example), length is a measure that they understand. The breakdown is that software == knowledge. It doesn't have a physical analog.
Chuck almost 14 years

@mkoistinen: The compiler can unroll the loop for you if it's actually a good idea. And bloated binaries with hand-unrolled loops like this often perform worse even though intuitively it seems like the computer should be doing less "work".
Sidharth Panwar almost 14 years

@Bob If you have better metrics then there's no point in sticking to SLOC. You can explain this to the managers. Why they are still struck with it is because the industry still holds on to it. Tell them that from QA point of view it's better to have findbug type of reports. Try and understand "their" focus, if it's internal monitoring they'll agree on findbugs, if it's external then they'll still need SLOC because that's what the client understands.
Bob Cross almost 14 years

@Sidarth, in case it isn't clear, some of the people involved ARE the clients. They aren't savvy enough to understand a Findbugs (especially since they're looking for a "size" metric in this category) - they aren't technical in our area.
Ira Baxter over 13 years

And like Bill's comment about "Who'd need more than 640K?" he was totally off the mark. If you talk to people that design airplanes, they fight to remove every last ounce of weight. This is the entire excuse for the Boeing Dreamliner. People should do that for code, too. (Its good to be an icon, people will quote you no matter how dumb your remark).
Robert P almost 12 years

As a metric by itself it's bad. As a value to derive other metrics it can be useful, such as predicting defect density in your software.
Paul Crowley over 11 years

Bill Gates never said "640K ought to be enough for anybody". wired.com/politics/law/news/1997/01/1484
Chuu over 9 years

Is it bad that my gut instinct is to agree that everything is measurable if you try hard enough? SLOC is an absolutely terrible metric, but that doesn't mean that there aren't relevant ones.
Bob Cross over 9 years

If you read the last paragraph of my question (and several of the answers), you'll see agreement with your point. Measuring SLOC is a good idea (and can be handled automatically): e.g., when measuring test coverage. What I continue to argue (and so do most of the answers) is that "new SLOC written" is not a good metric for project progress. If you can implement more requirements near zero SLOC increase, your value per line of code increases.
Vandroiy over 9 years

@BobCross I guess we all agree in general. It just might be better to focus on what SLOC does mean instead of what it does not mean. That is more constructive; the managers get their metric, which is indeed useful, and learn how to interpret it. The question whether it directly measures productivity barely needs extra mentioning.
Bob Cross over 9 years

Thanks for your suggestion. If you reread the original question, I suggest two metrics that communicate progress and value to the project management better than SLOC. The original question was focused on the problem of lazy metrics: the equivalent of counting punch cards and calling that progress.
Kzqai over 8 years

This is a great argument for SLOC as a metric. Once a project has matured, we should be trying to plateau the SLOC count, and not allow it to increase wildly.
Yunfei Chen almost 4 years

@mkoistinen Second example is better?? Are you sure, what if you do a million, ten million are you going to spend all day typing out code that takes someone else 10 seconds to type out?? Are you going to spend 10 million more $$ on someone for the 10 million version which does the same thing as the three line version, not to mention the 10 million line version cannot be scaled while the three line version can....
Yunfei Chen almost 4 years

@Vandroiy More constructive, writing redundant repeated code and introducing harder to debug code is more constructive???
Yunfei Chen almost 4 years

I disagree with your first point there, Do NOT write 2 lines of code when 1 will do, unless the 2-line version makes the code 2 times easier to maintain.
Yunfei Chen almost 4 years

By that logic Selection sort is the best algorithm for sorting because it is the shortest, QuickSort, HeapSort and MergeSort are unnecessary....
Yunfei Chen almost 4 years

Even I do not know what FindBugs is , And I am a C++ programmer in software Developing, Is it a new tool for finding bugs in the code or an IDE or something??
mkoistinen almost 4 years

@YunfeiChen yea, you're probably right... 10 years ago, I was wrong.
Yunfei Chen almost 4 years

@mkoistinen I do not get the 10 years part, unless it is a joke about how it took you 10 years to write out those print statements :).....
Yunfei Chen almost 4 years

@JoshLee We will have to substract bugs you delete... I think that should be more programming related...
mkoistinen almost 4 years

@YunfeiChen you were responding to a comment I posted 10 years ago.
DylanYoung almost 4 years

I don't think the comic suggests "all metrics are dumb", but rather that it's dumb to try to measure certain things in certain ways. Pretty sure Alice understands that the cartesian plane under a Euclidean metric is super useful.
DylanYoung almost 4 years

Quicksort is much more than 2 times faster than Selection sort for large n. n^2 versus nlogn.