Book Recommendations: Testing

I got an email from one of my teammates yesterday asking me for some book recommendations on testing.  I'm glad that he didn't just ask for books on unit testing, hopefully that means I've done a good job at emphasizing that it's not all about unit testing.  Anyway, here's my list of required reading on testing:

The Art of Unit Testing, Roy Osherove - At the time this was the only book to talk about the structure of unit tests and what made good tests versus bad tests.  To my knowledge this is still true and it's #1 on my required reading list for developers.  The author sought to fill the gap between starting from scratch and the various works on Test-Driven development that were already out there.  This book doesn't focus on TDD, it focuses on unit testing.  What makes a unit test, what makes one good or bad, and how to build suites of good ones.

Agile Testing: A Practical Guide for Testers and Agile Teams, Lisa Crispin - I'm a strong believer in breaking down the silos between the "development team" and the "testing team", to the point where I hate hearing those designations.  For any team to be successful the entire team must be involved with everything from the beginning and this includes testing.  Nothing is done until it is tested.  This book shows that no matter what your designation is on the team that you have value to add to testing, and that testing early and often is necessary for long term success.

Growing Object-Oriented Software, Guided by Tests, Steve Freeman and Nat Pryce - This is the best book on Test-Driven Development I have read to date.  The examples and explanations of the whys and hows of test-first development, as well as the authors' wisdom to focus on test-first at all levels (i.e. Acceptance Test-Driven Development combined with Unit Test-Driven Development) make a fantastic book to learn how to apply the test-first approach.  The authors focus on showing you that incrementally building your test suite while you add the functionality doesn't have to be difficult or overwhelming.  It's about repeating simple cycles of developing an acceptance test at the outer loop, watching it fail, then moving to the inner loop and developing a set of unit tests while implementing the actual functionality while working towards passing that acceptance test (and all the unit tests of course).

xUnit Test Patterns: Refactoring Test Code, Gerard Meszaros - I've gone through a great deal of this book, and while it has a wealth of information I honestly had a hard time finding places where some of the patterns could be applied to my test code.  That doesn't make it a bad book by any stretch, however, and if you're the kind of person that patterns speak to, this will help you find structure in what can sometimes feel like a daunting learning curve.  I find this as more of a helpful reference than required reading, but it's still nice to have around.

Advanced Software Testing - Vol. 1: Guide to the ISTQB Advanced Certification as an Advanced Test Analyst, Rex Black - Despite the fact that this is a certification book, there is a lot of useful information in here about test analysis techniques.  Understanding these techniques and both where and how to apply them is necessary to be able to develop test suites that are "lean" in that they cover the necessary parts of your system without wasting effort through duplication or ineffective test cases.  The layout is a little chaotic, but if you can cut through the noise you'll find the techniques indispensable.

Practice - Seriously.  This is the best way to learn the ins and outs, limitations and strengths of various tools in various language, as well as how to generally unit test various situations.  CUnit or Unity?  CppUnit or gtest?  NUnit or MBUnit?  CMock or mock-by-hand?  GMock or mock-by-hand?  RhinoMocks or Moq?  You can't know to what situation each of these tools is best suited unless you've tried them (ideally on a real project).  But don't try to master the tools, try to master the practice.  It will make you more valuable if you understand the foundations since you'll be able to move between environments quickly with little ramp-up time.

There are no shortcuts to mastery, and as much as one would like to believe they're special in the beginning (I'm sure I've been guilty of this), the 10000 hours can't be faked.  Practice and you'll get better at it, you'll find the patterns, and you'll be able to pick up most any tool like this and use it effectively.  But you need to develop the foundation to understand why you're unit testing before it can evolve from just another mundane task to perform before something is "done" to being engrained in the very way you do your work.

Read More

Overtime as Failure

First, let me make myself clear.  This post isn't focusing on voluntary overtime (though you should still consider the impact to your team, but that's another post), it's focusing on forced overtime in order to meet some deadline.  I consider this to be one of the worst kinds of failure.

Teams forced into this situation are typically faced with the worst of all possible choices.  Do we take shortcuts to get out on time and risk the quality issues that can come from that, or work long days and weekends, not have a life, and still likely introduce problems due to burnout?

It's failure if you have to even ask this question, but let's look at the choices anyway.

Keep Working 8 Hours

Hopefully this is the direction your management is leaning, but it comes with a cost.  In order to continue working at the same pace you must sacrifice scope or let the schedule slip.  Yes, personnel can change too, but I'm assuming that by the point you realize you need to make this decision that adding new people won't get you to the finish line any faster.  Everyone must either accept the delay with the understanding that this is what is necessary to achieve the expected level of scope and quality, or strip out a few unfinished/untested features and move them to the next release (yes they should move to the top of your backlog).  This will let you release a working product of known quality without cutting corners.

Burn the Midnight Oil

Some people might not think this is so bad.  I think it's one of the worst things you can do to your team.  What message are you sending when you're forced into this situation?  To management?  To your team?  By deciding to work long hours, weekends, or even burning the midnight oil to put out the occasional fire (many of which were a direct result of those long hours and weekends) you're sending the message to everyone involved that this is the new normal.

This sets a precedent and creates the expectation that all future situations like this will be addressed the same way.  Then the first time it doesn't go this way (because you tried to do it the right way) you'll catch flak about why your team is no longer dedicated to their job.  Do you really want to knowingly commit to 7-10s every few months?  Think of the children.

Evaluate Why

After you've made it across the finish line, you absolutely must evaluate how you got there.  How did you get into that situation to begin with?  Did the team commit to too much?  Did certain features take longer to implement or test than the team estimated?   Did the team not consider testing costs?  Were there a lot of unknown unknowns that crept in?  Maybe some other part of the process caused the schedule to drag?  You need to answer these questions in order to avoid this type of situation in the future.  

Don't Go There

This is one of those cases where the best answer is to avoid asking the question in the first place.  Pull things that would normally be done at the end of a project earlier into the schedule.  Build your installer or deployment pipeline after the first couple weeks, produce system-level tests for each new feature you add (and make sure they pass before moving on!), do code reviews on a per-feature basis rather than monumental reviews at the end, etc.  The more frequently we do these things the better we get at them and it also has the welcome side effect of enabling us to be releasing almost all the time.

Remember, no matter what anyone tries to force you to do Quality Is Not An Option.  Be a professional and stand your ground.  This is a tough spot to be in, but burning your team out or torching the codebase with sketchy implementations will do more harm than good in the long run.

Comments are open.  I'm sure everyone has their own stories or advice to lend to this situation.  Let's hear it!

Read More

My Personal Manifesto

I actually started this post back in August of 2010 under the title "What Values Drive Me?" but for some reason I never finished.  Perhaps this type of self-reflection would have been too much?  Either way, here is my meager attempt at explaining what drives me and the type of professional I try to be.  An instruction manual if you will.

The general format of these sections will define what type of X I strive to be.  I think the best way to discover this about myself is to flip it on its head and ask how I expect someone to deliver if I'm on the receiving end.  This will be a living post that I'll come back and update every now and then I'll note when I change something.

What type of professional do I want to be?

If I were paying someone else to do what I'm doing, how would I expect it to be done?

• Take care - no shortcuts, no matter what the external pressures, even if I'm a bear about it.
• To be kept in the loop - I'd like to be able to have a constant pulse on progress.  I expect to be able to get this feedback on demand, even if I have to bother the deliverer(s) to get it.
• To be consulted when questions arise - I don't want someone to get it wrong just because they didn't ask.  Ask the question, the customer may not even have considered it.
• To be told the truth, no matter how inconvenient - Bad news stinks, hidden bad news is worse, but it is what it is.  If you believe you're not going to meet some date or some scope, tell the customer so they can evaluate how best to move forward.  Don't deliberately under-promise in order to over-deliver.  It destroys your client's ability to forecast future resources and projects.
• To see progress not just on a chart or table, but really working - Charts are good, but they're only part of the story.  If I can't see some feature or functionality actually working did you really do anything?  This isn't an unreasonable request.  Expect demos to be requested and be prepared to execute them.  Done means done, including being demonstrable.
• If you say "done" or "complete" I expect it's really done - "Code Complete" means nothing to me.  Is it usable?  Could I deploy it now?  Can someone who is not you pick it up and work on it?  Now?  6 months from now?  Get on the same page about what done really means, but if there is no agreed-upon definition, ask what you would expect if you were sitting where your client is.

What type of leader do I want to be?

How do I prefer to be led?

• Lead by example, not command and control - If I expect other people to adopt practices or do things a certain way, I need to do it first and let them see the benefits.  Telling them to do it a particular way does not encourage them to understand the Why behind doing it that way.  "Just do it" isn't encouraging, nothing can replace a good light bulb moment.
• Expect greatness - Make sure people know they're accountable for their work.  Encourage them to explore and grow within and beyond their skillsets.
• Coach - make sure you can help people pick up good practices by coaching them when they need it.
• Allow failure - failure is learning, learning is growth, just keep it in check and as short-lived as possible.
• Encourage exploration - you don't know everything, so let other team members discover new things to teach to you.
• A Sense of Purpose - I need to believe in what I'm doing to stay motivated, so I want to give my team a sense of purpose to help drive them to do their best work.

What type of coach do I want to be?

If I were a team member who needed help with something, how would I expect to be coached?

• One on one - This is really what coaching is all about.  If I'm to learn something well it should be one on one with someone who knows how to do it.
• My hands on - If I were getting instruction I expect to be driving.  Let the other person drive when you're teaching them and talk them through it.  Let them reason it out, don't just do it for them or whip up an example and expect them to be able to follow what you're doing.
• Let others fix their issues - This is similar to the last one, but important enough to highlight on its own.  As a team lead it's really tempting to just dive in and fix everything for everyone else.  Don't do this, it's a trap!  If you're reviewing code for another team member (especially a more junior team member), don't just go into the code and fix it for them.  Present the comments and let them go on to fix them.  They'll grow more this way, both technically and confidence-wise, and you'll find that in the future you'll have less and less to fix with their work.  If you just do it for them you'll always be doing it and you're ruining a growth opportunity for them.  Tools providing asynchronous code reviews like Crucible work very well for this.

What type of team member do I want to be?  

What do I expect from my teammates?

• Keep things clean - Make sure you're following best practices as established by the team.  This means tested, documented, well-factored.
• Keep things working -  Keep the system building and the tests passing.  If you break it, fix it right away before doing anything else.
• Keep others updated - Make sure people know what you're working on.  This will keep the team from duplicating work and will keep things moving forward instead of sideways.
• Admit when you need help - We all want to prove we can solve this problem, but sometimes you just need a different perspective.  It's OK to ask for help, remember that we're working as a team and not as a bunch of individuals on disparate parts of the same system. 
• Step up, but don't overdo it - Don't burn yourself out and don't work a ton of hours that your team isn't working.  You're worth more well rested, and working 12 hour days skews what the rest of the team appears to be delivering on.  If you work a bunch of extra hours and then stop, the team's velocity will decline and your customer/client will wonder why delivery has slowed.  Normal means normal, don't force the rest of your team to define normal as 10-12 hour days.

How am I best managed?

• Expect me to keep you apprised of progress somehow.  Forcing me to provide this keeps it in my face and makes me ask whether or not what I'm working on is on track.
• Tell me what that progress report should contain.  I'm a big fan of task boards and burndown charts since you can get as much or as little information as you want, but if there is some measure or detail of progress you need that I'm not providing then tell me.  I'll figure out how to get it to you, but I can't know you need it unless you tell me.
• Get me access to the people I need to get questions answered, when they arise.  I don't like making assumptions and I try to catch myself making them whenever I can, but I can't remove assumptions unless I can get questions answered.  I don't like stacking assumptions or questions up to ask at a weekly or monthly meeting with the customer/client since by then I've likely stacked other things in the system over top that assumption, so having access to them sooner (ideally immediately) is always better.
• Give me leeway to explore new options.  I don't want my relevancy or my ability to deliver quality to my clients to stagnate, so I don't want my toolset to stagnate either.  Just because an organization has historically used one tool for some purpose for so long doesn't mean that it applies equally well to all situations.
• Keep me challenged.  I get bored, and when I get bored I start looking for other opportunities.  I don't like to be idle with my career, my client's money, or my organization's resources.  If I'm not growing I'm wasting time.

Read More

First Real Test for Net Neutrality?

First, read this.

Is this the first instance of an access provider giving preferential treatment to data based solely on where it originates?  Is removing the limitation of an arbitrary bandwidth cap truly "preferential" or a legitimate business decision?

Now I admit that Comcast claims this is on their "internal networks" and so this is why it won't count towards bandwidth limits, but I'm willing to bet that it costs the same to maintain the equipment it travels on no matter where the bits are coming or going.  I'm also willing to bet that the route the data follows is 99% the same as data coming from a service like Netflix on the "real internet" (granted with less "distance" to travel).

I'm sure their argument could still be made, however, but at the end of the day it's a fine line.  What's to stop them from standing up their own version of another bandwidth intensive service like Spotify, iTunes, Pinterest or Backblaze and giving preferential treatment to their version?  Granted, these hardly consume the bandwidth video streaming services like Netflix consume (as of today), but at what point does such a practice become anti-competitive?

I suppose since they're not actively targeting traffic coming from a service like Netflix or Backblaze (e.g. through throttling or reduced quality of service) that they're not really doing anything wrong?  But again, where is the line?

Either way, bad news for Netflix and possibly any other number of media streaming or bandwidth-heavy services.  Maybe Netflix should get with Verizon to open a similar door on their networks.

Read More