About a week and a half ago I finally started breaking ground (that is, created a Visual Studio soluton and started writing code, with the intention of it being used in production) on a big composite pet project I've been considering for a very long time. I won't get into what the project is going to be about (at least not now), but I will say parts of it are multi-faceted software foundation / framework. So, once I'm done, I'll be able to take a bunch of the code I wrote here and re-apply it on a completely different project. This is important to me as I am seriously in need of gaining some inertia in pet project development and deployments.
So right away I started writing tests to "prove out" each little piece of code as I wrote it. I'd heard the TDD gospel and wanted to convert, but I don't have a lot of TDD practice under my belt. But right away I had about 75% code coverage, which I felt was pretty good compared to 0% of projects in the past. Every time I wrote a little bit of code, I hit Ctrl+R, A to be sure nothing I just did failed. This worked fine, at the beginning, while working with simple, lightweight objects.
Pretty soon I found myself implementing abstract provider/service classes, and then create at least one or two implementations of them. And then I wrote this big dependency object that gets passed around back and forth. By the time I had some prototype code written, I realized that my tests would not break because I hadn't been writing code around my tests, I had been writing my tests around the code I had been writing. And at this point I was clueless as to whether the code I had just written would work, because this was all plumbing code. I wanted to at least try something and see if the basics were working, so I created a console application and wrote some code with Console.Write()'s to get a visual on the output. But at this point things were getting messy. My dependency object I had created was created for the one implementation class of my abstract provider/service object. I now had a throw-away console app in my solution that didn't belong. And my test coverage was down to something like 5%.
That's when I realized I was going about this all wrong. For months I had a hard time understanding why TDD proponents argued that writing code before their tests is "putting the cart before the horse", when writing tests before code seemed so backwards to me. But now it started to click. For many years, I've been designing code in my head, then implementing those designs in code, and if it works, great, if not, I fix the bugs. I'm starting to see now how I need to retrain myself. It's okay to design code in your head. You just need to document those designs in tests. At first, they won't even compile. Fine. But at least document your designs in tests.
Funny, I haven't ever heard anyone make the connection, but writing tests with mocks of the way you want your code to work is exactly the same to a programmer as wireframes are to web design. One should not design a web page by throwing some HTML into a web site, trying to make it look nice and complete, and then showing it to the customer for the first time and saying, "Here you go, it's done. Now tell me what's wrong with it." Rather, a professional web production team would first collect the requirements (in workflow verbiage) of the web site, throw together some wireframe mock-ups, create Photoshop design comps in parallel, and then go to implementation, all the while following up with the customer for feedback during each and every step along the way. TDD is exactly the same but for the programmer. The tests are wireframes and business rules, just like a web application spec has bullet-point business rule text that serves as manual tests for all parties involved to both define and validate the implementation.
Even if there is no third party customer who would approve your "wireframes" and you're working solo, the analogy of "wireframing a web design" should still apply. A web designer/engineer can, but shouldn't, create a web site before wireframing it. He should start with a napkin, and go from there. Likewise, one can, but shouldn't, write code before writing tests for that code, because tests are not just validations that assumed functionality works but are also definitions of assumed functionality, and without definitions of assumed functionality there are really no coding objectives.
The hardest part for me to do TDD so far in this new experience is realizing I wasn't doing TDD in the first place, and then going back and scrapping all the code I had just written, whether it was good or not. I knew it had a little bit of 'bad' mixed in with the good, and that should not be acceptable. Awareness of "a little bit of bad code buried in there" typically means mostly bad code in the long run, and a lot of wasted time, because bad code cascades and affects everything else, like yeast.
One other thing about these incidents is that I've also been re-learning the theory of YAGNI (you ain't gonna need it!). I was getting too comfortable with the idea of introducing extra method signatures on objects that were not getting properly tested to begin with, and for which my use cases were purely theoretical, and not truely intentional. I've argued here in this blog in the past that YAGNI is extremely naive and not the appropriate attitude when writing software in smaller shops (like one or two man bands) because the multi-role developer often knows exacty what he needs and should not be withheld from adding what he needs in order to do his job. That's great, ignore YAGNI, so long as you're taking an "undisciplined", less-formal approach to software or are writing software that is not intended to be reused such as a framework. However, in my case, I'm writing framework bits, and I must balance ease of use by way of high versatility versus ease of use by way of simplicity. Other programmers, including myself at some later point in time, should not be confused with options. Programmers are software users, just like their end-users are software users. So the end-users use the mouse and click simple things, whereas programmers use framework code to write code, fine, either way, they're all users. The Apple and Google approaches are to keep it simple, stupid (KISS). Don't overwhelm the user with options, especially if either of two options reach the same outcome. I should define one "best practices" path and only introduce the other path when it is needed, not when it is slightly and occasionally convenient.
Part of the reason why I write like such is to drive it into my head. I still settle for less than what I preach sometimes.
There's one other thing; tests are a part of the TDD strategy, but I'm also beginning to think that once I have made some headway into these framework bits to be able to actually write some code that passes tests, I might also start writing some application/implementation project code. Call it a manual test if you like. The tests themselves should not lose priority. But I think it makes sense, to an extent, to see code written in action as soon as it's feasible in order to prove out code design prototypes. Writing application code should NOT actually be additional tests but rather take advantage of what's alreay passing and help guide the tests as the tests are being written. In this light, the only problem I have with TDD is that TDD is spoken of too often in terms of unit and integration tests, whereas the whole lifecycle of designing with tests and of testing must be much broader than unit and intgration tests. TDD starts with unit tests, but unit tests, by definition, only test one tiny unit of functionality. In order to really drive your design and development through tests, you need to be able to test workflows, too. Starting from Point A, given n parameters, we should ultimately arrive at B, and this might involve several steps.
Oh, I still need to watch Rob Conery's MVC Storefront videos where he, too, learned TDD. I got about ten or so videos in until I caught up, and then stopped watching while he kept streaming them out.