> The goal is to test how your code behaves and not what the external tools are doing. In my testing I capture the expected response from the external system and replay it in the test. My interest is in what the application is doing with a given input.
Fair enough. But according to your definition, this still does not test correctness, since if you feed your actual SGBD/LDAP/etc with invalid data, your program will still produce an error.
> I'm not sure what this setup is exactly.
Consider that you have, say, a web application. You have a form, and you change the email normalization rules.
With integration tests, you simulate a POST request with a the form data (all 56 fields of them), you capture the Location header in the result, because you need to know what ID you have generated, and you do a SELECT in your in-memory database to find out if your user email is correct.
With a unit test, you simply build a user, apply the normalization function, and validate that the email looks as expected. Which is one of these scenarios is likely to be less code?
> Unit tests are also a huge overhead in maintenance. Any time you make a change in code, you have to go and update many tests. As you yourself point out, since you're testing many scenarios your tests often have more code than your actual application.
How do integration tests help with that? You're going to either end up with the same number of tests (but with more code) or you won't code the tests, and you may end up with a new issue in the bugtracker next time you release.
> I don't think unit tests are a valid way to ensure correctness either. Your individual pieces of code might be correct, but how they interact might be wrong. The only way to capture the actual use cases is by running end to end tests.
I'm actually not advocating unit tests as an alternative to integration tests, but as something complementary. It is an effective way to test the effectiveness of a unit of code, though.
Fair enough. But according to your definition, this still does not test correctness, since if you feed your actual SGBD/LDAP/etc with invalid data, your program will still produce an error.
> I'm not sure what this setup is exactly.
Consider that you have, say, a web application. You have a form, and you change the email normalization rules.
With integration tests, you simulate a POST request with a the form data (all 56 fields of them), you capture the Location header in the result, because you need to know what ID you have generated, and you do a SELECT in your in-memory database to find out if your user email is correct.
With a unit test, you simply build a user, apply the normalization function, and validate that the email looks as expected. Which is one of these scenarios is likely to be less code?
> Unit tests are also a huge overhead in maintenance. Any time you make a change in code, you have to go and update many tests. As you yourself point out, since you're testing many scenarios your tests often have more code than your actual application.
How do integration tests help with that? You're going to either end up with the same number of tests (but with more code) or you won't code the tests, and you may end up with a new issue in the bugtracker next time you release.
> I don't think unit tests are a valid way to ensure correctness either. Your individual pieces of code might be correct, but how they interact might be wrong. The only way to capture the actual use cases is by running end to end tests.
I'm actually not advocating unit tests as an alternative to integration tests, but as something complementary. It is an effective way to test the effectiveness of a unit of code, though.