Life, Teams, and Software Engineering

Month: September 2010

AAA vs BDD Structuring in Unit Tests

UPDATE: I updated this to include updates to the BDD example’s test function name.  I’m starting to dislike having the called interface in the test name.  It’s inflexible and unnecessary and ultimately doesn’t help the reader all that much.

It’s always good when there are people on your team whom you can both learn from and teach things to. Such is the case with my current team. A couple team members have never done unit testing as it’s known in the industry today. Mostly just “pound at the interface until I’m comfortable and throw it over to testing to deal with” unit testing.

During our first code review there were a lot of issues with unit tests. I actually prefer to let it pan out this way; this way people actually get in there and try it out, to later see what works, what didn’t, and (hopefully) get some ideas from the rest of the team about how to improve it. One recurring theme was that there was a lot of redundant code throughout test suites. Setup and teardown were outright missing! That’s good, because now they’ve seen the problem, and part of the solution is to use those. Another thing was how the test cases themselves were structured. I’ve come across two widely accepted ways of structuring unit tests:

Arrange – Act – Assert
Given – When – Then

I’ve personally used both in the same codebase when it makes sense, but I’m wondering if there’s more to it than just semantics and readability. With AAA you are more likely to interact with the class under test directly inside your test function:

  1. void interface_context_somethingHappens()
  2. {
  3.     //arrange
  4.     mock1->setCallShouldSucceed(false);
  5.     mock2->addFakeValue(“Value”);

  6.     //act
  7.     out->interface();

  8.     //assert
  9.     CPPUNIT_ASSERT(somethingHappened);
  10. }

However, I’ve started to notice that if you read your tests, I mean really read them, then using the Behavior-Driven-Development (BDD) Given-When-Then structuring will actually nudge you towards factoring the real test preparation and calls to the class under test out of your test case:

  1. void SomethingShouldHappenInSomeContext()
  2. {
  3.     givenSomeContext();

  4.     whenActionPerformed();
  5.     thenSomethingShouldHappen();
  6. }

  7. void givenSomeContext()
  8. {
  9.     //configure context
  10.     mock1->setCallShouldSucceed(false);
  11.     mock2->addFakeValue(“Value”);
  12. }
  13. void whenActionPerformed()
  14. {
  15.     //execute action
  16.     out->interface();
  17. }
  18. void thenSomethingShouldHappen()
  19. {
  20.     //check that what should happen happened
  21.     CPPUNIT_ASSERT(didSomething);
  22. }

Yes, it’s more code, and yes, it may just be semantics, but I see something more. The naming alone has suggested that maybe I should remove the details from the test. Not only does this produce well-factored code but, but it pulls communication with the class under test to the boundary of my test suite and away from my test cases. Now, if the usage of this class changes for some reason, I only have to update it in one or two places in my test suite, rather than in every single test case. Obviously this approach may not be suited for every single situation, but like I said, I use both where it feels right. Granted, this is considered a good practice when using AAA as well, but you’ve got to name those newly extracted functions something don’t you?

I’d love to get some feedback on this. What convention does your team follow? Are my observations valid or can it be chalked up to something else?

Autotest for Compiled Languages (C#, C++) using Watchr

When I was learning Rails I set up Autotest on Ubuntu with Growl notifications, which I thought was a pretty slick idea. On Ruby this whole technique is super easy and efficient because Ruby is an interpreted language; there’s no compile time to slow you down, and no other files to pollute your directory tree. Compiled languages don’t have that advantage, but I think we deserve some continuous feedback too. Here I’ll describe how to configure Watchr, a generic Autotest utility, to run compiled tests whenever a source file in the path is updated. This tutorial will use a C# example, but it’s trivial to have it trigger on different file types.

Getting Started

First, we’ll need to install Ruby and Watchr.  Because I’m using Windows I just downloaded RubyInstaller.  Make sure you put the Ruby/bin directory in your PATH.

Next, download Watchr from Github, extract the archive and navigate to that directory.  Or you can just download the gem directly, but some people might want to run the tests locally first. The following command will install the gem from the local directory:

C:\mynyml-watchr-17fa9bf\>gem install Watchr

Configuring Watchr

Now that we have all the dependencies installed, we need to configure Watchr. This process is easiest if you already have a single point of entry for your continuous build process, but if you don’t it’s not that bad and you’ll probably want one anyway. Now, at the same level as the directory(ies) containing your source code, create a text file. I usually call this autotest.watchr, but you could call it autotest.unit or autotest.integration if you’re into that sort of thing. For now, just put in the following line in:

  1. watch(‘./.*/(.*)\.cs$’) {system “cd build && buildAndRunTests.bat && cd ..\\}


Yes, it’s that easy. What this is doing is telling Watchr to monitor any files that match the regular expression (in this case a recursive directory search for .cs files) inside the watch() call, and then execute the command on the right. I also have it configured to return to the same directory when it’s finished, but I don’t know if that’s actually necessary. The watch() pattern is what you would modify for different environments. For example, you could use watch('./.*/(.*)\.[h|cpp|hpp|c]$') for a Mixed C/C++ system, or watch('./.*/(.*)\.[cs|vb|cpp|h]$') for a .NET project with components built in different languages. An important thing to note is the $ at the end of the regex. Because it’s likely that there will be a lot of intermediary files generated during the build process, we don’t want a file which happens to match this pattern that’s generated at build time to trigger an infinite loop of build & test (like happened to me). The heavy lifting is done here, but the stuff specific to your project happens in build/buildAndRunTests.bat. Let’s take a look at that:

  1. pushd ..\
  2. echo Building tests
  3. “C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com” Tests.Unit\Tests.Unit.csproj /rebuild Release
  4. popd
  5. pushd ..\Tests.Unit\bin\Release
  6. echo Running tests through nunit-console
  7. nunit-console.exe Tests.Unit.dll /run=Tests.Unit
  8. popd


You’ll obviously want to customize this to the specifics of your project, but right now it’s hard-coded to call Visual Studio 2008’s devenv.com (on a 64-bit OS) and build a project called Tests.Unit. For brevity it also assumes that nunit-console.exe is available on the PATH. Not terribly interesting, but that’s the rest of the work.

Now to have all the magic happen. Run the following command in a new console window from your project directory:

C:\Projects\MyProject>Watchr autotest.watchr

That’s it! Watchr is now monitoring for changes to files that match your pattern. Simply modify any file matching the pattern and watch the whole process set off. Once it finishes, you can hopefully see the results and it will wait for the next change.

Now there’s one less thing you have to do during your heavy refactoring sessions, or just with day-to-day development.

SA_OAuthTwitterEngine Search Error 400

It may be deliberate that this function doesn’t work but that doesn’t make much sense.  I’m trying to use SA_OAuthTwitterEngine as the sole Twitter engine in my app but I’m having problems with the getSearchResultsForQuery: method (yes, I’ve hooked up the YAJL library).  Every single time I send a request using the getSearchResultsForQuery: selector on my SA_OAuthTwitterEngine I’m get the following error:

"The operation couldn’t be completed. (HTTP error 400.)"

MGTwitterEngine produces no such error.  You wouldn’t think calling this selector would be a problem since it inherits directly from MGTwitterClient and therefore provides the interface, but actually it is.  The problem lies in SA_OAuthTwitterEngine’s implementation of

  1. - (NSString *)_sendRequestWithMethod:(NSString *)method
  2.                                 path:(NSString *)path
  3.                      queryParameters:(NSDictionary *)params
  4.                                 body:(NSString *)body
  5.                          requestType:(MGTwitterRequestType)requestType
  6.                         responseType:(MGTwitterResponseType)responseType;

which overrides the default in the base MGTwitterEngine.  It’s described quite nicely in a comment but does nothing to explain the consequences of doing it this way.

  1. // ——————————————————————————-
  2. // mdificatfrom the base clase
  3. // the base class appends parameters here
  4. // ——————————————————————————-
  5. //    if (params) {
  6. //        fullPath = [self _queryStringWithBase:fullPath parameters:params prefixed:YES];
  7. //    }
  8. // ——————————————————————————-

The problem is that it doesn’t consider which type of MGTwitterRequestType is being used before making the decision to construct an OAuth request.  Certain actions simply don’t require an OAuth-formatted request, and in this case it just plain doesn’t work.  MGTwitterSearchRequest and MGTwitterSearchCurrentTrendsRequest are two types that I don’t believe require authentication, and therefore have no use for OAuth requests.  To fix the problem for this particular request type add the following code to the beginning of SA_OAuthTwitterEngine’s implementation of _sendRequestWithMethod:

  1. //Defer work to super because these requests don’t support OAuth
  2. if (requestType == MGTwitterSearchRequest || requestType == MGTwitterSearchCurrentTrendsRequest) {
  3.     return [super _sendRequestWithMethod:method path:path queryParameters<:params body:body requestType:requestType responseType:responseType];
  4. }

This certainly beats lugging around an instance of each class, one for reading and one for writing.  Hopefully this helps someone out, it was driving me nuts.

After using GeSHi I have now decided that I need a new blogger theme….but one thing at a time.  Back to XCode.

OCMock iPhone 4.1 SDK

I imagine that this is a common occurrence when a new version of the iPhone SDK comes out but I figured I’d put it out there for the droves of development teams who will take the plunge with the update this coming week.

If your iOS project relies on OCMock and you just upgraded to the iOS 4.1 SDK (and you upped the Base SDK on the target to 4.1) then chances are your unit test projects aren’t building.  To get it working again:

  1. Checkout the OCMock trunk: http://svn.mulle-kybernetik.com/OCMock/trunk
  2. Open the OCMock project in XCode
  3. Change the Base SDK on the OCMockPhoneSim target to iOS Simulator 4.1 (see Base SDK vs. Active SDK for why).
  4. Build the target.
  5. Copy libOCMock.a and Headers from the build/$(CONFIG)-iphonesimulator folder to wherever you were keeping them before.
That should allow you to link and run but if your experience is anything like mine you’ll be getting a bunch of unexplained errors like “An internal error occurred when handling command output: -[XCBuildLogCommandInvocationSection testName]: unrecognized selector sent to instance”.  Highly annoying but ultimately it will still report “X out of X tests passed” although with all those errors I’m less inclined to trust it.
Thanks to Vincent Daubry for pointing me in the right direction when he experienced the problem during the last major update.

Agile Design: Ask, is it still valuable?

Today I got some feedback from my lead on an early draft HLD document. Being new to the team, I asked whether or not they intended to keep the overall UML static class diagram which there was a placeholder for. He said that he wanted to keep it. I agreed because it could be a good first exposure for someone trying to ramp up on how the system works. What we disagreed on was how and when that diagram is created.

I explained that I have no issue with the diagram as long as it is generated from the code, as opposed to it being created before any code is written. He responded with “up-front design is a software engineering best practice”. Right away I could see this was a philosophical difference. I responded that while I agree in principle, I have never seen an end system look like the UML diagram that came before it. There’s nothing wrong with that, the team just learns more about the system as it’s implemented and takes action to improve the design. They let it evolve. We can spend all day diagramming, but I’d rather not spend time documenting and diagramming details that have a high probability of not reflecting reality when it’s all said and done. It just makes the document misleading, and what’s more, we’ll end up having to reverse-engineer the diagrams from source to sync them up anyway.

In his defense, I’m sure he’s not really talking about the same big design up front that agilists abhor, but I still believe that up-front design all the way down to the static class diagram level is silly. If it’s a thought-exercise, fine, it’s usually helpful to at least identify your high level components in order to develop your metaphor, but don’t begin the process with the expectation that it will reflect the final system. You’ll be disappointed.

Is all design analysis wasteful? Heck. No. My stance would have been different if there were a lot of unknowns with technology or various functional behavior. The additional analysis can be used to further mitigate the risk of a costly failed implementation. Most of all, it allows the team to learn more before diving in.  However, even in this scenario I would recommended a different approach, such as Spiking the feature or developing some examples to drive TDD. I prefer an approach that provides real feedback that I’m on the right track. Diagrams aren’t very good at this.

If the team is learning more about the system, that’s the value in performing some up front design, but only when the time isn’t spent on something they would have easily discovered on their own if they had just gone ahead. When faced with a choice of more up-front design analysis, ask yourself, “Is this still valuable?” If your team will gain something from it then go for it without remorse (there’s nothing un-agile about being responsible), but once mostly everyone is comfortable, move on and get to the real (and fun) work.

Copyright © 2017 Life, Teams, and Software Engineering

Theme by Anders NorenUp ↑