Difference Between Unit and Integrated Tests

As you can probably tell by the topics I’ve created, I’m attempting to find out as much as possible about testing in a Continuous Delivery environment.

Today I’ve had a discussion with developers around their unit tests, especially around APIs. Our developers mock/stub the API in unit tests and force an error (for example, a Bad Request) to occur.

We have tests which also generate a Bad Request by using a POST and using an invalid body - but we do this against a deployed service.

Our test will check how 2-3 areas of code interact with each other - but when our unit tests test each of these in isolation, is it still worth having an additional test around these?

I’m not sure if this is the best example - but if our developers are covering all code paths in unit tests, where does the value of integration tests (as part of a build process) lie? Should we be using them sparingly?

1 Like

A common pitfall is to have a range of test phases that do the same tests… because requirements. I usually insist that each phase have a different focus or a different environment (with external integrations etc).

What I’m hearing is that you have 2-3 modules you can test both individually and as a whole. In both cases on API and service layer. so you could model this in 4 areas: Module / whole - Service / API. And then place test activities in each field.

A similar model:

Integration tests look for bugs around interfaces and mistaken expectations around which function/piece is supposed to do what – they absolutely are useful even if your team has super awesometastic unit tests in place. I wouldn’t worry about duplicating coverage unless you’re duplicating both the area of coverage AND the nature of the coverage. Of course when prioritizing, you may very well want to worry about getting some coverage in place for areas of the code that have no coverage at all first and then get to worrying about coverage at different levels (unit / integration / end-to-end).

1 Like

How about renaming them to fit your context
http://www.codingthearchitecture.com/2015/06/12/unit_and_integration_are_ambiguous_names_for_tests.html

The way I look at this is one test should, ideally, find out problem. If you have a test which can fail for multiple reasons then you have to spend time figuring out which reason (or reasons) the test fail.

A unit test should be set up so that there is one reason it can fail and there is one assertion to catch that failure. When you see a failure you know exactly what failed.

There might be three reasons an integration test might fail. If you are testing the integration of code A and code B could fail because there is a bug in code A. Or there is a bug in code B. Or there could be a bug in the integration. If we set up the Continuous Integration (CI)/Continuous Delivery (CD) so that if unit tests fail, we don’t even bother running the integration tests then when an integration test between code A and code B fails, we know that all the unit tests passed. Therefore the integration test failed because code A does not integrate with code B.

Again, there were three reason the integration test might have failed but because we run unit tests first, there is really only one reason the integration test failed.

Finally, if an integration test finds ONLY the same defects as a group of unit tests, then it is a redundant test and does not need to be run. You should google “mike cohn test pyramid”. You’ll see examples of the layers of testing (unit, contract, integration, etc.). Tests on the pyramid should be pushed down to simpler tests if possible. If 5 unit tests can catch the same thing as 1 integration test then write 5 unit tests and remove the integration test. If 5 unit tests can catch 5 of 6 defects but the integration test will find the 6th defect as well, write the 5 unit tests and keep the integration test.

Darrell

P.S. Testing too much is better than not enough testing, IF you are trying to reduce defects at all costs. However, too much testing might be costly and the business might decide they’d rather miss a few defects but ship for less money.

1 Like

A unit is the smallest testable part of a software. Unit Testing is majorly performed by software developers themselves or their peers. In rare cases, it may also be performed by independent software testers. In Unit testing, it is advised not to create the test cases rather focus on the behavior testing whereas Integration could be performed by Developer or Testers as per organization policy. As per Top Software testing companies, the difference between these two are mentioned below:

  1. Unit testing is to verify that the single module of code is working fine or not whereas Integration testing verify the multiple modules.

  2. Unit testing is not divided into different categories whereas Integration testing is classified into following categories are
    a. Top-down Integration.
    b. Bottom-up Integration.
    c. Big Bang.
    d. Sandwich etc.

  3. Unit testing is to verify the single component of the software whereas Integration testing is to check the whole behavior of the software.

  4. Module specification is required for Unit Testing and Interface specification is required for Integration testing.

  5. Unit testing lies in White box testing and Integration testing falls under both White box as well as Black box testing.

Besides all these differences, Unit testing and Integration testing are very essential testing techniques for developing error free software.

Hope this information is helpful for you.

3 Likes

We recently completed an API project. I worked with the tech lead to establish good unit tests written by developers, and the test engineers would create some automation. We called the tests written by test engineers behavioral tests.
At first, we had some duplication between the unit tests and the behavioral tests. Since the behavioral tests were meant to exercise the API as a user would, some unit tests were discarded where they duplicated that kind of test. As the API grew, the difference between the unit tests and behavioral tests was easier to determine. Both sets of tests used mocks to isolate the tests from environmental errors.
When we deploy to a testing environment where we can execute integration tests, we execute a single smoke test. The objective of the smoke test is to evaluate connectivity (can the API connect to a database or other servers?), security (are the IDs and roles established in the environment so that the API can operate?), and configuration (are the configurations correct for the environment?). We rarely executed any other tests because we had confidence that API behavior was sufficiently exercised with the unit tests and behavioral tests.

I think there is always a need and there is a huge value for having integration tests even if unit tests exists. The reason is, the way system communicate with each other is a totally different ball game, compared to just the individual components. This is all the more important in the current era of Internet of Things.

For example - How would unit tests test different scenarios related to fitness trackers like Fitbit? We need to simulate real human movements and mock services, data needs to be synced seamlessly across your mobile website, mobile app and fitness tracker. There is GPS, Camera and other sorts of things involved. Just by doing a unit test we wont know whether these interactions are taking place correctly as there are lot of factors involved in it.

Also, some teams do not write unit tests and if there are unit tests they do not cover different failure scenarios of the system.

So, there always needs to be some sort of integration tests performed based on the context of the application.

Hello @epsilon11!

Good to chat with you again, Sir!

I’d like to explore some of the great points you made. While I believe unit tests can be valuable in the scenario you describe, I may not have the context or familiarity with a Fitbit to speak expertly about them.

I agree that integration tests can provide information about the Fitbit behavior while in use by a human. The primary purpose of integration tests are to explore and evaluate connectivity, configurations, and security. What I might consider is isolating human movements, services, and data synchronization. In my opinion, these items resolve themselves to just data.

Human movements generate data in the Fitbit and that data can be modeled for use in unit tests. Scenarios such as walking, running, sitting, typing, and other movements certainly have data signatures which can be used to explore Fitbit code behaviors.
With respect to a mobile application, I believe behaviors can be isolated and explored through unit tests. Isolation is a key testability component in supporting tests at that level. The same can be said for the website.
At an integration level, I start small. Perhaps we integrate the mobile application and the web site while providing mock data we used in unit testing. Or perhaps we integrate the mobile application and the Fitbit. My intent is to minimize the number of components so I can grow confidence before evaluating the mobile application, the web site, and the Fitbit together.

Not having a Fitbit, I’m not familiar with how it uses GPS and image information. These are sensors and sensors generate data. Might that data be useful in unit testing? I agree that the context would drive that conversation.

We see this all too often. I consider it a cultural artifact that should be addressed by Testing leadership.

Agreed. I also encourage testers to explore the boundaries of unit testing. I believe it is possible to have unit tests explore products a little more deeply given the tools and technologies available.

Thoughts?

Joe

(Surely there are lots of good things on the thread already, but I don’t have time to read right now :smiley: - I will shoot some of my views)

Our test will check how 2-3 areas of code interact with each other

I prefer to call this integration tests. Integrated tests would be something slightly different (and usually a scam). Integrated tests are tests where things are simply integrated and checked somehow.

Taking a simple web app, made out of a frontend app and a backend service, these two components would be the units.

An integrated would consist of bring them both up and accessing the frontend through a browser and investigating things from there. Check if the database can save a field with 2 million chars would be one of these scam tests, if done through the frontend app.

On the other hand an integration would be something like a Contract Test, where you provide to each unit some expectations of the other and check if the behavior will be as expected when the integration comes to happen.

Hi Joe, I agree with you. There are ways unit tests can test different components to a certain level but when it comes down to actually using the entire system like an end user, integration tests help in simulating those relevant scenarios.

Historically, what I have done is, I sit with the developer and try to understand what unit tests cover. Those aspects not covered by unit tests are good candidates for me as a tester to cover with my testing approaches/strategies. This way we don’t repeat the same thing the unit test does and also get decent test coverage.

Unit testing is helpful in testing individual modules of a software application in isolation (without any interaction with dependencies) to confirm that the code is doing things right. Integration testing will check if different modules are working fine when combined together as a group.

The purpose of unit testing is to exercise different parts of the module code to detect the coding errors such as interface, local data structures, boundary conditions, independent paths, and error handling paths. Whereas the goal of integration testing is to detect design errors while focusing on testing the interconnection between modules.

Let us suppose that there is an ecommerce mobile app for testing. During unit testing, the sign-up form might work well in terms of validating and registering a new user successfully. But in case of integration testing, if the sign-up form fits well within the frame of the mobile device or not. If the form is user-friendly or not. So, integration testing takes care of all these issues.