Disadvantages of Test Driven Development?

Question

In my code I have declared, within the custom control ValidatingTextBox, the following dependency property:

public DependencyProperty visibleText = DependencyProperty.RegisterAttached("visText", typeof(String), typeof(ValidatingTextBox));
public String visText
{
    get { return theBox.Text; }
    set { theBox.Text = value; }
}

But when I try to use the xaml

<local:ValidatingTextBox>
    <ValidatingTextBox.visibleText>

    </ValidatingTextBox.visibleText>
</local:ValidatingTextBox>

it says that such a dependencyproperty doesn't exist within ValidatingTextBox. What am I doing wrong? Is there a better way to interact with the child text box of my custom control?

Solution 1

Several downsides (and I'm not claiming there are no benefits - especially when writing the foundation of a project - it'd save a lot of time at the end):

• Big time investment. For the simple case you lose about 20% of the actual implementation, but for complicated cases you lose much more.
• Additional Complexity. For complex cases your test cases are harder to calculate, I'd suggest in cases like that to try and use automatic reference code that will run in parallel in the debug version / test run, instead of the unit test of simplest cases.
• Design Impacts. Sometimes the design is not clear at the start and evolves as you go along - this will force you to redo your test which will generate a big time lose. I would suggest postponing unit tests in this case until you have some grasp of the design in mind.
• Continuous Tweaking. For data structures and black box algorithms unit tests would be perfect, but for algorithms that tend to be changed, tweaked or fine tuned, this can cause a big time investment that one might claim is not justified. So use it when you think it actually fits the system and don't force the design to fit to TDD.

Solution 2

If you want to do "real" TDD (read: test first with the red, green, refactor steps) then you also have to start using mocks/stubs, when you want to test integration points.

When you start using mocks, after a while, you will want to start using Dependency Injection (DI) and a Inversion of Control (IoC) container. To do that you need to use interfaces for everything (which have a lot of pitfalls themselves).

At the end of the day, you have to write a lot more code, than if you just do it the "plain old way". Instead of just a customer class, you also need to write an interface, a mock class, some IoC configuration and a few tests.

And remember that the test code should also be maintained and cared for. Tests should be as readable as everything else and it takes time to write good code.

Many developers don't quite understand how to do all these "the right way". But because everybody tells them that TDD is the only true way to develop software, they just try the best they can.

It is much harder than one might think. Often projects done with TDD end up with a lot of code that nobody really understands. The unit tests often test the wrong thing, the wrong way. And nobody agrees how a good test should look like, not even the so called gurus.

All those tests make it a lot harder to "change" (opposite to refactoring) the behavior of your system and simple changes just becomes too hard and time consuming.

If you read the TDD literature, there are always some very good examples, but often in real life applications, you must have a user interface and a database. This is where TDD gets really hard, and most sources don't offer good answers. And if they do, it always involves more abstractions: mock objects, programming to an interface, MVC/MVP patterns etc., which again require a lot of knowledge, and... you have to write even more code.

So be careful... if you don't have an enthusiastic team and at least one experienced developer who knows how to write good tests and also knows a few things about good architecture, you really have to think twice before going down the TDD road.

Solution 3

When you get to the point where you have a large number of tests, changing the system might require re-writing some or all of your tests, depending on which ones got invalidated by the changes. This could turn a relatively quick modification into a very time-consuming one.

Also, you might start making design decisions based more on TDD than on actually good design prinicipals. Whereas you may have had a very simple, easy solution that is impossible to test the way TDD demands, you now have a much more complex system that is actually more prone to mistakes.

Solution 4

I think the biggest problem for me is the HUGE loss of time it takes "getting in to it". I am still very much at the beginning of my journey with TDD (See my blog for updates my testing adventures if you are interested) and I have literally spent hours getting started.

It takes a long time to get your brain into "testing mode" and writing "testable code" is a skill in itself.

TBH, I respectfully disagree with Jason Cohen's comments on making private methods public, that's not what it is about. I have made no more public methods in my new way of working than before. It does, however involve architectural changes and allowing for you to "hot plug" modules of code to make everything else easier to test. You should not be making the internals of your code more accessible to do this. Otherwise we are back to square one with everything being public, where is the encapsulation in that?

So, (IMO) in a nutshell:

• The amount of time taken to think (i.e. actually grok'ing testing).
• The new knowledge required of knowing how to write testable code.
• Understanding the architectural changes required to make code testable.
• Increasing your skill of "TDD-Coder" while trying to improve all the other skills required for our glorious programming craft :)
• Organising your code base to include test code without screwing your production code.

  ---

PS: If you would like links to positives, I have asked and answered several questions on it, check out my profile.

Solution 5

In the few years that I've been practicing Test Driven Development, I'd have to say the biggest downsides are:

Selling it to management

TDD is best done in pairs. For one, it's tough to resist the urge to just write the implementation when you KNOW how to write an if/else statement. But a pair will keep you on task because you keep him on task. Sadly, many companies/managers don't think that this is a good use of resources. Why pay for two people to write one feature, when I have two features that need to be done at the same time?

Selling it to other developers

Some people just don't have the patience for writing unit tests. Some are very proud of their work. Or, some just like seeing convoluted methods/functions bleed off the end of the screen. TDD isn't for everyone, but I really wish it were. It would make maintaining stuff so much easier for those poor souls who inherit code.

Maintaining the test code along with your production code

Ideally, your tests will only break when you make a bad code decision. That is, you thought the system worked one way, and it turns out it didn't. By breaking a test, or a (small) set of tests, this is actually good news. You know exactly how your new code will affect the system. However, if your tests are poorly written, tightly coupled or, worse yet, generated (cough VS Test), then maintaining your tests can become a choir quickly. And, after enough tests start to cause more work that the perceived value they are creating, then the tests will be the first thing to be deleted when schedules become compressed (eg. it gets to crunch time)

Writing tests so that you cover everything (100% code coverage)

Ideally, again, if you adhere to the methodology, your code will be 100% tested by default. Typically, thought, I end up with code coverage upwards of 90%. This usually happens when I have some template style architecture, and the base is tested, and I try to cut corners and not test the template customizations. Also, I have found that when I encounter a new barrier I hadn't previously encountered, I have a learning curve in testing it. I will admit to writing some lines of code the old skool way, but I really like to have that 100%. (I guess I was an over achiever in school, er skool).

However, with that I'd say that the benefits of TDD far outweigh the negatives for the simple idea that if you can achieve a good set of tests that cover your application but aren't so fragile that one change breaks them all, you will be able to keep adding new features on day 300 of your project as you did on day 1. This doesn't happen with all those who try TDD thinking it's a magic bullet to all their bug-ridden code, and so they think it can't work, period.

Personally I have found that with TDD, I write simpler code, I spend less time debating if a particular code solution will work or not, and that I have no fear to change any line of code that doesn't meet the criteria set forth by the team.

TDD is a tough discipline to master, and I've been at it for a few years, and I still learn new testing techniques all the time. It is a huge time investment up front, but, over the long term, your sustainability will be much greater than if you had no automated unit tests. Now, if only my bosses could figure this out.

View more solutions

Comments

[Brendan Weinstein]

I added an answer stating that BDD can alleviate some of these negatives. I urge you to consider this factor when collecting your negative input, since some of it can be eliminated and no longer considered negative.

[IanL]

For clarifications, I am not against or for. I am trying to make an informed decision on the matter, but most people who advocate TDD do not understand, or will not admit the negatives.

[Jim Hurne]

The title mentions "Test Driven Development", but the body of the question mentions "Test Driven Design". Which of the two is this question about? There are important, but subtle differences between the two. Test driven design is about letting the tests drive the software's design. Test driven development is usually associated with writing tests before production code (but not necessarily letting tests influence the design).

[Lewis]

TDD is a cage which detains developer's creativity.

[Casper Leon Nielsen]

please stop closing important questions, djesus

[Rob Cooper]

This is a common problem. I tend to mock these with hard coded objects, then test the database seperately. Your business layer should then only work with static data, you DAL will then be tested in a controlled environment (where you can script the data into it etc.)

[IanL]

Is this really a negative, or sly way of stating a positive.

[Rob Cooper]

Definately can be a problem, however I find I am seeing a noticable difference in how much I am affected by this. All comes down to "writing testable code" I guess.

[Garry Shutler]

All private methods should be tested through the public methods that would exist anyway.

[Allain Lalonde]

Doesn't a unit test by definition only test the smallest unit of code possible. A function, method, etc.. Since refactorings are supposed to be incremental in a TDD approach, the total % of your test codebase that breaks should be small for any given refactoring. Right?

[Jason Cohen]

This is not possible with all classes. Sometimes you don't want to mock up all dependencies etc. and you just wish to test a utility method.

[user9991]

Absolutely. You have to consider BDD when evaluating TDD.

[Wedge]

@Allain, that's the theory, in practice things are always more complicated, especially when working with a team. There's nothing magical about test code that automatically makes it better written or better designed. It's a classic "who watches the watchers" problem.

[Tom Martin]

A lot of "graphic effects" will be based on geometry which can easily be applied to testing. Otherwise it is based on a library that should be well tested

[IanL]

Graphic Effects or UI elements?

[raju]

I completely agree that the more tests you have the more potential test refactor you'll have to do when making system changes. I'm confused about the statement regarding making design decisions based on TDD? That's one of the major benefits of TDD, right? It drives out design by nature?

[casademora]

While you should have an idea of what the architecture of your system will look like, you don't have to know a whole lot ahead of time when doing TDD. TDD means that the tests DRIVE the design, so it will change as you implement more test scenarios

[Adi]

The main point (4) is - any system which is not well defined and is likely to keep changing to match an evolving visual behavior, different AI spec, behavioral algorithms, etc.. will cause big time investment in repeated test definitions since we keep on changing the desired test results.

[hayalci]

It seems BDD = behaviour-driven development

[mparaz]

I agree with the vacuum. The original tutorials of TDD where you'll write the test without ANY code - and get a compile error - is crazy.

[Dafydd Rees]

Depends on your definition of "deliver a solution on time" - is that "any old, partially broken solution on time" or "deliver working solutions on time". You certainly loose the ability to deliver partially broken solutions on time. As far as dev speed goes - I like the metric "elapsed time between start of dev and a week of faultless live deployment". If you measure it fairly, it's hard to even stop the clock at all on a copmlex piece of non-TDD work.

[erikkallen]

-1, this is exactly the thing the OP said he didn't want.

[erikkallen]

+1, so true. Add to this the requirement to sometimes add getters/setters to private fields just to be able to set up and read state correctly for a unit test, even though the state should be private to the class.

[Ankit Roy]

Many true statements, but: what erikkallen said. -1.

[sleske]

True, but wouldn't that be the same without TDD? Without it, you'd have to do more manual testing, which would suffer the same problem.

[oligan]

What was the rest of the sentence ending with "(cough VS Test), then main"?

[Dan]

@ j_random_hacker says hacker... LOL

[Robert Koritnik]

Isn't the "Big time investment" going to save you time later on while developing your solution? Especially with a complex one? I guess it should save you time. Not to think about maintenance phase where small changes can easily break the system. ( or maybe I'm just being naïve about unit + regression tests preventing future bugs )

[Robert Koritnik]

Using tools like Pex & Moles you can quite easily avoid writing interfaces for every small damn thing. Moles will help you with that tremendously.

[András Szepesházi]

+1 "you might start making design decisions based more on TDD than on actually good design prinicipals" - the biggest pitfall of TDD IMHO.

[Esko Luontola]

+1 for the selling problem. :) I'm right now at a new company and thinking of how to create a culture which would allow skills to spread freely.

[Daniel C. Sobral]

Sadly, the first reasonable answer I saw...

[plmaheu]

Seems like a critism of unit testing and object oriented programming, not TDD.

[Snowcrash]

Aha - but that's where TDTDD comes in. Test Driven Test Driven Development.

[ha9u63ar]

Quite practical and simple answer - +1 for the "Mind setting" part

[Yuriy Nakonechnyy]

@ScottSaad the problem IMO is that design should be outlined first and then it should be validated by writing tests and corrected if needed. I've seen numerous cases when people were jeopardizing good design just to be able to write test. As a result - most of the system was covered by tests but the design was really ugly. I think that this happens because TDD is pushed to the masses as very simple methodology with following misconception: if part of the system is covered by tests and they pass, then everything is fine (including design).

[HorseloverFat]

I still occasionally find bugs in my test tests. So now I practise TDTDTDD.

[TheMorph]

Actually correct **unit testing**–not only TDD–requires mocks / stubs. And programming against an interface is often a good idea, same goes for patterns. If you mix UI and logic you will have a bad time. If you have to test DB interaction you can still mock your DAO for the unit tests and use the real thing for an integration test.

[Adi]

Sergio / Robert, I am very much in favor of having unit testing for generic systems and definitely for components which represents the very base for systems. Having said that, I would add that one need to differentiate between these cases and over simplification of the real life by trying to claim that every system can be treated this way. Not all systems can be generalized and simplified for unit testing and if you try to force the nature of unit testing on such systems you can easily end up spending much more time correcting you unit tests than actually testing real results.

[Nils von Barth]

A good example of TDD preventing a simple solution are shell scripts, where even a 5-line script may require a very complex test; see: Bash and Test-Driven Development esp. answer Actually complex scripts can be written in Python etc., where TDD is easier, but for simple tasks TDD rules out shell scripting.

[Scott Nimrod]

Consider writing your tests as if it's a living requirements document. Then you would see the light. Also read XUnit Test Patterns.

[David Sykes]

Random tests can fail intermittently, and make repeating them difficult

[Mike Dunlavey]

@DavidSykes: Whenever you do a random test, you record the parameters, so that if it fails you can repeat it, or you can repeat it later even if it didn't fail. The point is it doesn't depend on you to think up the test cases. If you're like me you instinctively gravitate toward safe test cases.

[BlueLettuce16]

Think about it, you are likely to end up designing to specific test cases, so you won't get creative and start thinking "it would be cool if the user could do X, Y, and Z". - In my opinion that is exactly the opposite. If you write unit tests you wonder about different business cases and that means that you are creative and it makes possible to foresee something unforeseen. All that creativity is not important however if your implementation has bugs.

[BlueLettuce16]

@Adi: I think that you are wrong. In my opinion every system can be tested that way, it's only the matter of self-discipline.

[BlueLettuce16]

@Yura: It is interesting what you say that people were jeopardizing good design just to be able to write tests. In my opinion if there were a good design there wouldn't be necessary to jeopardize it. I have seen once such a project and the codebase was a nightmare, but people thought the same - that the design is great. I agree only with the part that TDD is pushed to the masses as very simple methodology, however it's the exact opposite. In my opinion when the code is designed good then when you make one small change then there is no chance to brake all tests or large amount of them.

[Yuriy Nakonechnyy]

@BlueLettuce16 totally agree with you - TDD is quite complex methodology and recently I've found very decent explanation of it: TDD is design tool, not testing tool which means that it should be used 80% for designing and verifying the design and maybe 20% for testing. The problem is newbies use it 100% for testing and that's why it fails so often

[Yuriy Nakonechnyy]

As outlined in the following article TDD is About Design, Not Testing: drdobbs.com/tdd-is-about-design-not-testing/229218691

[mike rodent]

I don't believe in no sanity clause since I growed up.

[Joshua Grosso Reinstate CMs]

@SnowCrash +1 I was looking around Google to see about how much time people spend testing their tests, and then I saw this answer. I officially found this because I was wondering about TDTDTDD.

[Joshua Grosso Reinstate CMs]

But, TDD makes it so that you can refactor and feel warm and fuzzy. I think you may have answered the negation of the question, but then you answer the right question in the next sentence; I am thoroughly confused. Care to shed some light?

[wax_lyrical]

Perfectly put. I couldn't agree more. For me, tests certainly don't help describe real-world, higher-level problem definitions. Good documentation has proven its worth time and time again. As the tech. industry ages, time-tested notions should be dispensed with using ever greater caution. Self-documenting code is a ridiculous notion. I do believe in prototyping, refactoring, and the agility borne of not ever-defining a problem at the start. However, ironically, not over-defining the problem at the start makes stubbing for TDD a minefield.

[sara]

This doesn't deal at all with TDD. If someone else in another department is writing your test cases, you are not doing TDD. If you have manual test cases you are not doing TDD. If your library code breaks and your tests fail because of changes to GUI you are most probably not doing TDD either. This seems more like arguments against big ineffective enterprise QA departments.

[sara]

I think this is unfair. Good TDD practices condemn magic numbers and obscure tests. Tests should be simple and as, or preferably more readable than your production code itself. your tests ARE the documentation. make sure they look like it. this answer feels a bit like saying "documentation is evil because sometimes people write really bad documentation" or "classes are bad because I've seen some god classes that were hard to deal with".

[lmiguelvargasf]

As you I consider that some consultant companies are taking advantage of pair programming and TDD just to get more money from their clients. It is pretty disappointing how these clients pay for ideas that seem reasonable at first glance such as two people think much better than 2 or that TDD assures that every line of code is tested, but in the end they are just excuses for making a client pay more for something just one person can do.

[Hitesh Sahu]

I agree upon fact that one mist have knowledge of design and testing before jumping into tdd. This is critical in projects with new hire as they are new to both.

[mike rodent]

I believe the future is (TD)^∞TDD. I have one file so far: it contains the letter "x".

[Zuko]

I agree with @Tim. Convincing members to adopt it is the hardest part.

[sabsab]

vote up for the Wisdom

[user2288580]

I've been developing for 36 yeas now this post may give you some good advice: stackoverflow.com/questions/738539/tdd-how/45971814#45971814

[Fabio]

the example I usually give is a SqlDataSource embedded in an ASPX page - this approach tightly coupling UI and database layers - which can be considered as bad design.

[Erik Reppen]

Yes! Let's test our tests! And then test the tests that test those tests! And then... Wait a minute. No. You can't actually test everything. And why would you even want to try to? Anybody who desires 100% test coverage is not giving me any confidence that they have real confidence in regards to their code.

[drognisep]

100% coverage is not always a realistic goal for real world projects. I don't think that using TDD requires 100% coverage, because some things are simply outside of the scope of a unit test (integration tests, functional tests, etc). I wrote a post that covers the scope of unit testing a little while back, check it out!

[YEH]

only third statement is legit "learn through debugging is lost"

[KolA]

Large tests codebase can easily discourage bold refactoring moves. Too many large projects stuck with obsolete domain design just because dissecting it means remodeling or deleting hundreds or thousands of tests and there is no political will to do so. Unit tests code is a client to your production code and will be coupled with it now matter how hard you try. Unit tests by their very definition verify implementation details, so if implementation needs to change drastically then it will inevitably render the tests invalid.

[John Henckel]

I don't know if it's because I'm worst or best.. but TDD rubs me the wrong way. It is because it forces me into dual maintenance mode too early. Every time I change the design of a class, now I have to also change the test cases. I expect and accept that from a mature class, but not from a class that I just wrote last week! Also can I just say that DI and TDD are not well supported by languages such as Java and C#. Someone really needs to create a new language so that the cost of TDD and DI is literally zero. Then we won't be having this conversation anymore.

[Roman-Stop RU aggression in UA]

It is a wrong assumption that you can write the tests once and do not change them. They are code and every code requires eventual refactoring after you do changes. Tests is not an exception. Refactoring tests is a must if you want to keep them maintainable.