Software Engineering

Tuesday, August 01, 2006

A Mockery of Mock Objects

Mock objects are emulations of real objects except without any implementation. These emulations have their outputs already defined before they are used. Typically unit tests that utilize mock objects will use them to mock various contingencies such as: emulating a specific interaction with the tested object, and setting up test return values on initialization objects. Mock objects seem appealing and catchy at first, but if you really think about it they aren't very useful. Don't waste your time learning Mocking Frameworks.

A lot of people would argue that one must mock an object because the real object is too difucult to initialize and setup for a unit test. Well, really if it is that difficult to set up, then one should refactor the real object so that it is simple to use. After all, simplicity is what we are all after when designing an object model anways, isn't it?

Another problem I have seen from very intelligent software engineers is that they get over enthusiastic about mocking objects. I have been on a team where the software engineers literally forced a test to pass by mocking too many objects. They mocked too many objects within the unit test forcing it to pass. It was like a teacher handing you a test with all the answers already filled in for you! The test did not test anything; it was set up to pass before it even ran! Needless to say the test passed, but there where numerous errors in the code.

Mocking object interactions is actually the root of the problem. When one mocks an interaction, then regression testing is no longer possible. For example, lets say you have an object A that object B implements internally. Object B calls a method of object A. In Object B's unit test, object A is mocked and the method called is set to return a value X. Object A's method implementation is now changed to return a new value Y. Since object B has mocked object A and its return value has been mocked, then object B's unit test still passes with flying colors! Object A's unit test will also pass with flying colors. But, the real applicaiton will fail. This could have been prevented if no mock objects were used.

Additionally, for those that feel like they "need" to mock objects for a particular situation, I can show you how to reproduce that situation without mock objects. Whenever you feel like you need a mock object you are developing code that is not testable or you are just being lazy.

The problem with mocking objects is that issues between object interaction passes by the "Daily Build and Smoke Test", or continuous integration, undetected. Because issues between object interactions aren't detected during continuous integration due to mocking interactions, then that means these issues won't expose themselves until functional/acceptance testing. By this point it is too late in the game! It is too late because typically acceptance tests are ran at the end of the iteration which could be thirty business days! Now, you have thirty business days worth of code to sift through in order to find the bug. If your lucky the bug will be detected at runtime during acceptance testing, but again too late in the game. Waiting for acceptance tests to catch problems with object interactions is just too late and unacceptable!

For more information and discussions about avoiding mock objects, visit these related posts (I will add more links to related articles as I find them. If you have any comments or links, then please feel free to add them to the comment section):
http://agiletesting.blogspot.com/2006/03/should-acceptance-tests-talk-to.html

3 Comments:

  • Hi Bruce,

    I think you may have missed the point. Or are much cleverer and observant than I! Which is entirely possible. Maybe if I offer an explanation of how I use mocks and integration testing it will counter your post and maybe make you less hostile to mock objects. Maybe you’ll turn round and tell me to eff off! Alternatively, you might offer me a counter explanation and change my mind.

    Firstly, mock objects are typically used to emulate objects that have a system boundary. Or at least that is how I use them. So, here is an example. My team is working on communicating to some old legacy machine, we have to look at running processes, format them, and present them in a web-page. The formatting of lists, merging and subsequent sorting is very, very important. Thus, with the aim of nailing the requirements we create some Fit tables, then some JUnit tests complete with mocks, and we start capturing the business rules in Java. What we mock is the communication to the back end via telnet. Right now, I don’t give a monkey’s about that. We also create a service layer for the front controller to use. This also returns mock data. This allows us to evolve the page design with the customer and allows us to start testing well before the infrastructure ‘glue’ is written.

    During this process at no point do we mock objects that exist. We only mock objects that wrap an external dependency. Mocking frameworks such as Jmock and EasyMock really help with this. Thus, your call to the community to not “waste your time learning Mocking Frameworks” is a bold one!

    The use of mocking really helps you to move forward with refining the requirements and also when you have a non-optimal team size, i.e. > 8, it enables communication. Sometimes, God forbid, one part of the team may be working on a different class. If they can provide the interface the rest of the team can work in parallel, in a test-driven manner. Now, as long as the behavior of the interface is well understood, development should go smoothly. However, if the interface is poorly understood the use of mocks will help the explorative design of the interfaces.

    Thus your claim “A lot of people would argue that one must mock an object because the real object is too difficult to initialize and setup for a unit test” is wrong. Most of us mock objects because the real object is either 1) impossible to set up, because it is a web-service or a database or 2) we don’t know what we are mocking and are happily exploring the design space. You are also a test-driven-developer, so you also know the value of evolving code in this manner.

    With your next point:

    “They mocked too many objects within the unit test forcing it to pass. It was like a teacher handing you a test with all the answers already filled in for you! The test did not test anything; it was set up to pass before it even ran!”

    You are completely right! Fools with techniques or fanatical developers will always miss the point. I have seen this too.

    In regards to your point about “ Waiting for acceptance tests to catch problems with object interactions is just too late and unacceptable!”. You are implying a couple of things:

    1) That all object interactions are mocked.
    2) That these will only be tested late in an iteration.

    In response to point 1. The way I use mocks, which I find valuable, is to mock external dependencies. Thus, interactions between the internal objects are not fake at all and thus are tested with unit tests and continuous integration tests. Also, my team creates a real smoke test using Cactus, if we are in container, or JUnit if not, that will test the interactions with ‘real’ objects in a real environment. When executed with the continuous build this is powerful.

    Quality assurance should be holistic and mock objects as a technique is a value adding practice. In response to the claim that only acceptance testing will spot damaged interactions I hope I succeeded in countering that point. Also, acceptance testing is not usually done at the end of an iteration but every day by an embedded tester, the customer team, and the CI box. Therefore, I think that your argument “It is too late because typically acceptance tests are ran at the end of the iteration” is leading the jury somewhat.

    I guess if you worded your blog entry “A mockery of mock objects used incorrectly with a development process that does acceptance testing after the code is developed and your development process is a bit broken” you would have had a point. And a very good one. And I have no doubt that some teams do use mocking like you described and that will produce complete rubbish.

    In conclusion, I am a little bit obsessed with productivity and I don’t carry out a single non-value adding thing. And yet I do use mock objects. With this in mind, I hope you may re-consider the use of mocks. I am going to read more of your blog entries tomorrow.

    All the best, James Dobson.

    PS – mocking helps inexperienced programmers with their design too. Also very powerful and necessary to create testable code.

    By Anonymous Jamie Dobson, at 2:00 AM  

  • I agree with the spirit of the blog post. I also agree with Jamie's comments which correct some of the flaws in the blog post.

    By Blogger Shannon -jj Behrens, at 2:17 PM  

  • "They mocked too many objects within the unit test forcing it to pass. It was like a teacher handing you a test with all the answers already filled in for you! The test did not test anything; it was set up to pass before it even ran!"

    This is just poor system design then. The class under test is trying to do too much and has too many dependencies.


    Mocking helps us to test the functionality of just a small unit of code. By not mocking, you are increasing the number of places the test could fail, making errors more difficult to pinpoint.

    Remember, unit testing (with mocking) is used alongside other types of test. It is the other types which should test the interaction between the objects.

    Writing acceptance tests (or system tests) to catch all edge-cases would be prohibitively expensive in terms of maintenance and running time for a system of any size. That is where the unit tests come in. Mocking keeps these quick and simple (if the class design is good.)

    By Anonymous jcoda, at 8:25 AM  

Post a Comment

<< Home