If you are not sure where to begin, or need assistance implementing unit tests in your development process, we are here to help!

Recommended resource for automated unit tests:
‘The Art of Unit Testing’ by Roy Osherove | Buy on Amazon

Transcript

Rightley: We get asked all the time, how do we avoid the 80/20 rule, where 20% of the work seems to take up 80% of the time when you’re writing code. That usually comes down to chasing down bugs and dragging out a project, a schedule, exceeding the timeline and budget. How do we go about avoiding that? I’d say, one of the biggest keys in the way that we do that is by implementing automated unit tests throughout the development. So, the end result is you get code that’s well thought out, it’s reliable, and it remains stable as you build more and more functionality on top of the code you just wrote.

So today we wanted to take a little bit of time to talk about the importance of unit tests in your product development and the total cost of ownership. I have Dave Abramson here, our Sr. Systems Architect to discuss that with us. So, what are unit tests?

Dave: Unit tests are pieces of code that exercise other small, isolated bits of code in your system.

Rightley: Ok, and why are they important?

Dave: Like you said, they increase the reliability of your code, they give you a level of comfort and protect against change. I don’t know if a lot of people may have heard the term whack-a-mole, when you change one piece of the code here and another problem over here happens. If you don’t have automated unit tests it could be days or weeks before that other problem is detected. Whereas if you have unit tests that’s constantly testing this code over here, a change over here, you run all the unit tests and you see a failing test over here.

Rightley: For those that aren’t super familiar with it, how often are unit tests run or how are they set up?

Dave: Unit test should be run by the developer at least every time before they check code into whatever repository they’re using. At the very least. Preferably they’re running the unit tests as they’re writing code, so they’ll be writing a test that fails, then they’ll be writing the code that makes the test pass.

Rightley: Is that all also called test-driven development?

Dave: Yes, that’s the basics of test-driven development.

Rightley: Ok so if you’ve heard that term that’s a lot of what that’s referring to. Are unit tests required?

Dave: Not all the time, but they should be all the time in my opinion. If you’re on my project they are. I think that any project that adheres to SIL level 2

Rightley: Like industrial project, industrial work?

Dave: Or any Class II or III medical device, software safety Class B or C is required to have unit tests

Rightley: So that’s something that regulatory bodies are looking for when you submit a package to be marketed

Dave: Yes

Rightley: How do you implement a good unit test?

Dave: It should be small. It should test an isolated piece of code. And what I mean by isolated is that you’re not going in at the highest level of code and pulling in all these difference dependencies to run the test because that just causes more points of failure. What you’re trying to do is isolate code and test it, one thing

Rightley: Like one function?

Dave: It could be one function, or it could just be one logical unit of work that may take a few function calls

Rightley: Give me an example, like parsing in a message, or is it more or less granular than that?

Dave: Sure, so let’s take the example maybe of receiving a message over ethernet and then parsing that message out.

Rightley: Where would you unit test that? At what level?

Dave: Ok so you would want to unit test the parsing for sure. You may or may not need to test something on the ethernet level, the thing is, with unit testing you want to eliminate dependencies. So you’re certainly not going to test at receiving the ethernet and then going all the way down to parse. What you’re going to do is have a buffer that you essentially assume was received over the socket. And then task that in your parsing logic.

Rightley: So if building a buffer was something that all should be tested, that should be done separately.

Dave: Right

Rightley: Got it. So breaking up into small testable parts where if one thing fails you know what failed and it’s easy to go find and fix.

Dave: Yes and I also think that having a good unit test has a clear name. And in the name it should say what’s being tested, the scenario or the conditions under which it’s being tested, and the expected result. This way when you’re looking at test explorers or lists of results of tests, you can easily filter them, search them, in groups of tests.

Rightley: Great tip. How do you think unit testing effects the overall project schedule/budget?

Dave: Well, I think you often end up with 25-30% more time in development. That’s not to say that it takes 30% more time to develop the unit tests because you can actually reduce your time developing the product code if you’re writing good unit tests as you go.

Rightley: Because you’re thinking about what the function needs to do when you write the unit tests so the thought part is already done, so it’s a different approach.

Dave: That’s right.

Rightley: Is it safe to say then that’s how we really help to escape the whack-a-mole situation at the end where yes, we’re investing that extra 25% up front but we’re not doing 25% bug fixing at the end?

Dave: Yeah, whack-a-mole can take a lot more than 25% of your development

Rightley: Not to mention, you get better quality code at the end too, right?

Dave: Absolutely, better quality, it’s more reusable units of code, more loosely coupled, and it’s testable. The code is testable, if you’re writing unit tests as you write code, you’re naturally going to write code that can be tested. If you’re using unit tests, gone are the days of writing your code, starting up the whole system and stimulating it manually in such a way that you hit the code that you just wrote and then set your debugger or breakpoints. You’ll still do a little bit of that, but largely you just write your tests, write your code, you run the test, it passes, and you move on.

Rightley: So you don’t have to think about it anymore.

Dave: And you don’t have to think about it anymore until the test fails.

Rightley: So I’m thinking about too what you’re saying, it also sounds like it probably enforces better coding habits across your team.

Dave: Absolutely

Rightley: So you get those added advantages of everybody is writing things in a similar way and makes everybody’s code work together better.

Dave: Yes, and you can run tests overnight as part of automated builds and you can stop problems early before they become bigger problems.

Rightley: Is it safe to say too, when we’re implementing unit tests you end up spending the time there, but then that ad hoc and regression testing then you’re also getting a savings there, not to mention the whack-a-mole at the end.

Dave: A huge savings

Rightley: But your integration tests and your system level tests are probably simpler too.

Dave: Absolutely, and the testing is repeatable.

Rightley: Yeah, and it’s done for you automatically.

Dave: If you’re doing it manually, ad hoc, you might be doing different things.

Rightley: It’s hard for a human to act as a machine and be consistent in the way that things are tested, time and time again.

Dave: Right, and you might do something differently and something passes when it shouldn’t have. And the other nice thing about unit tests is if you find a bug, you can write a unit test that exposes that bug and you then just change the code to make that test pass, and then as long as that test continues to pass, the bug is gone.

Rightley: It sounds like there’s a lot of hidden benefits that if you’re implementing this, and it’s a virtuous cycle across the code.

Dave: They’re only hidden if you’re not doing it. If you’re not unit testing that’s the only time they’re hidden. Once you start doing unit testing they become painfully obvious, those benefits. All the old ways you were doing things, you’ll kick yourself.

Rightley: So what does someone need to know when they’re starting a project, about how they actually set this up and start doing it?

Dave: Well, they’ll need to decide what testing framework they’re going to use. There’s a lot of testing frameworks out there. Any language you’re writing in these days has a testing framework.

Rightley: What does it cost?

Dave: It shouldn’t cost anything. You can pay if you want but I’ve never paid and they’re fine.

Rightley: So there’s a lot of tool chains, a lot of test frameworks that are out there that are freely available.

Dave: A lot of test frameworks freely available and toolchains which are something different.

Rightley: Thank you. Once you’ve determined what toolchain to use by doing a little Googling, a little research, what are your next steps? How do you go about mocking up and stubbing out and setting it up?

Dave: That’s actually something I forgot. In unit testing, there’s a concept of mocking or stubbing objects which is basically setting up code that allows the code you’re testing to run without a concrete dependency. So for instance, go back to that parsing. Say I’m inputting a stream into my parse function, typically maybe that’s some kind of socket stream into the parsing function. I could change that and instead use a in MemoryStream to pass into the parsing function.

Rightley: It’s a buffer that you’re controlling as opposed to the actual…

Dave: Right. When you’re unit testing you’re going to end up writing code that depends on interfaces, not concrete objects.

Rightley: Which is also kind of another good concept

Dave: Right, that’s how you end up with the loosely coupled code, more reusable, like that.

 Rightley: So you touched on naming conventions, is there anymore you could expound upon to help folks out tactically when they’re actually setting up a test?

Dave: Can I go back to mocking and stubbing? There’s mocking and stubbing frameworks that you can get that are also free. So, if you’re using gtest. A lot of people use gtest for C and C++.

Rightley: Which is a lot of the work we do.

Dave: Yes. It’s freely available, it runs on Windows, it runs on Linux, you can run it on embedded platforms, we’ve run it on a STM32.

Rightley: So even microprocessors, you can run.

Dave: Yeah, we run unit tests on micros.

Rightley: Which is something that I think a lot of people don’t think that you can do.

Dave: You can absolutely do it. It doesn’t even have to be fancy. Worse case, if you don’t have an OS or if you’re just bare metal, you just have a compile switch or a different way of building it that instead of initializes all your hardware and goes into your executive loop, just calls a bunch of tests and puts some stuff and writes to the output. So you get feedback that way. There’s really no reason not to test. But, the mocking framework, gtest has gmock, which allows you to create basically objects that you make when your code that you’re testing is going to call one of those interfaces, you can make it just do something like return 5, if that’s what you need it to do.

Rightley: So I love that we’re getting down into the weeds and a little bit of the tactics of this. This would be really good for engineers that have maybe dipped a toe in or think they’re ready to get into it. I see you’ve brought along a reference here. Is this something you’d recommend.

Dave: Yeah, I would recommend this book. It’s ‘The Art of Unit Testing’, by Roy Osherove. It’s a really good book.

Rightley: Non-paid endorsement, but this is actually something that we use around here.

Dave: I think every intern that comes in here says how do I write unit tests? And I just kind of say read this and ask me questions.

Rightley: If we have any other questions about unit tests, you can see we got kind of technical on this, but we’re happy to answer any other questions that might come up. Also, we’ll put a link to the book in the comments if you’re interested in it. It’s a great reference and as far as programming books go it looks a little heavy, but it’s actually not too bad. I think this is one that we can wholeheartedly recommend.

Dave: There’s a good chapter in here on changing your company culture.

Rightley: So there’s some actual application about how people can use it.

Dave: Yeah, how if you’re in a company that doesn’t do unit testing.

Rightley: How you can begin to introduce the idea.

Dave: Yes, because you’ll find resistance at first.

Rightley: So, earlier Dave, you mentioned writing the units tests while you write the code, what’s the advantage of doing that as opposed to writing the unit tests after you’ve written the code?

Dave: Well, if you write them after you’ve written the code, it’s likely that you haven’t set up your code to work off of interfaces and all your dependencies are going to be concrete, classes are going to depend on other classes, you’re going to have to initialize a bunch of code just to test one little piece.

Rightley: That’s when the time to do it could really stack up. You start losing the advantages.

Dave: Right, you lose a lot of the advantages if you wait until after the code’s been written because you won’t be writing code that’s testable because you’re not in that frame of mind.

Rightley: If you’re ever faced with the prospect of having to go back maybe for some regulatory reason and writing it, it’s something that you can do but something that you should try to avoid at all costs if you’re doing fresh development. Thank you all very much for tuning in.

Thank you all very much for tuning in, thank you Dave, very much, for getting very tactical with us and also sharing a little bit about unit testing. We hope this is useful. Again, if there’s anything else that we can share with you, please put it in the comments and we’ll response, we’re happy to create another video like this one. Thank you very much.

PARTNER WITH PSI

Whether you’re creating a Class II or Class III medical device, or perhaps a SIL Level 2 or SIL Level 3 industrial automation or process control system, we can help you build automated unit tests into your project.
Tell us about your next project!