End-to-end tests and a horse in vacuum

There's a subsystem (a console app that receives tasks and sends reports via WCF). About a month ago I, influenced by GOOS book, wrote a couple of end-to-end tests for the app just before making another refactoring (yes, the app is rather messy) and adding a new feature to it. I liked how it worked.

Yesterday I did a quite big refactoring of the subsystem doing TDD. Things went really smoothly and a bunch of small, focused, loosely coupled classes were the result. At the end, when I was sure that everything is ready to launch the app, I run the end-to-end tests and (uppps) one of them was red! Good! It turned out that I added a class implementing a role interface that'd been already implemented by another one and pass the former as a dependency. The class has default fake implementation (it threw a NotEmplementedException) and cause the app to shut down in a rather rare case (which luckily was covered by the test).

What I realized from this story: we must write end-to-end tests if we want to be sure that all those nice classes really work together. Without them we're doing such a thing that Russians call "a horse in vacuum", i.e. a thing that doesn't has any practical use.

Comments

Post a Comment

Popular posts from this blog

Haskell: performance

It turned out that it's not Haskell who was slow in my previous Fibonacci test. It was my lack of Haskell knowledge. Haskell has two integral types - Int, which has bounds -2147483648..2147483647 and Integer, which is unbounded. Both types implement Integral type class, so out fib function can be defined in terms of this type class: OK, now we can test performance of the function parametrizing it with the two concrete types. Integer first: We get our previous result ~15 seconds which is rather slow. Now test it with Int type: Whoa! The Int type is ~6 times faster than Integer! And with result of 2.8 seconds Haskell's took the third position in our small rating :) Current list (in seconds): C# - 1.26 F# - 1.38 Nemerle - 1.45 Haskell - 2.8 Clojure - 9 Erlang - 17 Ruby - 60 Python - 120
Read more

Regular expressions: Rust vs F# vs Scala

Image
Let's implement the following task: read first 10M lines from a text file of the following format: then find all lines containing Microsoft namespace in them, and format the type names the usual way, like "Microsoft.Win32.IAssemblyEnum". First, F#: Now Rust: After several launches the file was cached by the OS and both implementations became non IO-bound. F# one took 29 seconds and 31MB of RAM at peak; Rust - 11 seconds and 18MB. The Rust code is as twice as long as F# one, but it's handling all possible errors explicitly - no surprises at runtime at all. The F# code may throw some exceptions (who knows what kind of them? Nobody). It's possible to wrap all calls to .NET framework with `Choice.attempt (fun _ -> ...)`, then define custom Error types for regex related code, for IO one and a top-level one, and the code'd be even longer then Rust's, hard to read and it would still give no guarantee that we catch all possible exceptions. Up
Read more

STM: F# vs Haskell

STM is a very nice parallel programming model used intensively in Haskell and Clojure. There's a F# implementation which can be found in FSharpx library. Today I'm going to test performance of both the Haskell and the F# STMs. The test is very simple - read a couple TVars, check their equality, then write them back incremented by 1, repeat a million times. First, the Haskell code: So, it took about 170 ms. OK, now F#: It took about 1,6 seconds which is an order of magnitude slower than the Haskell result. It's rather frustrating.
Read more