Actually I wanted to use Kotlin for a while now, but there were only minor tasks within the Java part of the project. That finally changed and we can focus to improve the Java backend drastically.

Part of this process was a library update. We decided to upgrade JUnit from 4 to 5. A big pain in the ass. I don’t think, I would do it again. JUnit 5 was also part of a bigger problem, even if it was actually PEBKAC.

Kotest and MockK features

If you already know about Kotest or want to know more about the problem I had, just skip to the next headline. The Kotest introduction is a little bit longer.

As I dived more and more into Kotlin, I stumbled over Kotest. A really neat testing framework for Kotlin. There’s nothing wrong with JUnit, but Kotest gives you way more awesome ways to structure your tests.

A little example:

package io.webcodr.demo

import io.webcodr.demo
import io.kotest.assertions.throwables.shouldThrow
import io.kotest.core.spec.style.FunSpec
import io.mockk.*

class UserServiceTest : FunSpec() {
    private val userRepository = mockk<UserRepository>()
    private val service = UserService(userRepository)
    private lateinit var user: User

    init {
        beforeTest {
            user = User(1, "Jane", "Doe", "jane@doe.com")
        }

        afterTest {
            clearAllMocks()
        }

        context("getUser()") {
            fun verifyRepoCalls() {
                verify {
                    userRepository.findById(1)
                }

                confirmVerified(userRepository)
            }

            test("should succeed") {
                every {
                    userRepository.findById(1)
                } returns user

                service.getUser(1).shouldBe(user)

                verifyRepoCalls()
            }

            test("should fail") {
                every {
                    userRepository.findById(1)
                } throws UserNotFoundException()

                shouldThrow<UserNotFoundException> {
                    service.getUser(1)
                }

                verifyRepoCalls()
            }
        }
    }
}

Kotest offers several different styles to write tests. I chose the FunSpec style for this example. You could also use a BDD-like or Jasmine-like style, if you want to.

It’s much more intuitive to nest tests with Kotest. To be fair, JUnit 5 allows you to use @Nested with an inner class to accomplish nesting as well, but it’s not as intuitive and harder to read than trailing lambdas.

Assertions are also a little easier to write. Kotest has over 100 different matchers. You can use them as extension functions, like in the example above or alternatively as infix functions, for example service.getUser(1) shouldBe user. It’s also quite simple to write custom matchers.

There are many more features like soft assertions, tagging, easy temporary file creation or handling for non-deterministic test cases.

For mocking we decided to use MockK, since it’s way more intuitive to use than Mockito with Kotlin. Don’t get me wrong, Mockito is a great library, but it has one flaw: the when method. when is a keyword in Kotlin and to use it with Mockito, you need to write it in backticks. That’s quite ugly and not intuitive at all.

So, in summary, Kotest offers a bunch of pretty neat features and is very intuitive to use. Of course, JUnit can achieve much of this as well, it’s just not that shiny and little harder to read.

The actual problem or: why the frak is there IntelliJ in the title?

If you migrate an old codebase to Kotlin and want to use Kotest, you will have no choice and have to use JUnit and Kotest in coexistence.

That shouldn’t be a problem, since Kotest uses the JUnit 5 Jupiter engine under the hood. But …

As I wrote a new service in Kotlin and some tests with Kotest, I could not start the JUnit tests anymore. As soon as the maven dependency of Kotest was present, IntelliJ didn’t recognize JUnit tests and used the Kotest files only. With mvn test (Maven Surefire) everything worked fine.

I tried several things and was ready to give up. Search engines didn’t find anything about this problem. Nothing on GitHub, nothing on Stack Overflow.

I hate to give up, so I decided to create a small demo project to open a GitHub issue. Well, that didn’t work out as intended, since the discovery of JUnit tests in the demo project worked fine. The IntelliJ JUnit runner did what it was supposed to: run JUnit and Kotest.

Well, frak. There must be some kind of configuration problem with my real project. I already looked at the IntelliJ runner config, Maven files etc. — nothing worked.

I compared the Maven files from both codebases and there was one difference: my real project did not include the JUnit Jupiter Engine dependency. Bingo. I added it to the Maven file and guess what? It worked like a charm.

What an embarrassment. As we upgraded from JUnit 4 to 5, we forgot to add the new dependency for the engine. I don’t know why the tests worked at all, but it seems the engine is not really necessary for all cases. But it can screw up test discovery quite well, if you forget it.

The dependency configuration in the POM file should look like this:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>${junit.version}</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>${junit.version}</version>
    <scope>test</scope>
</dependency>

Well, that was a long post, but IMO it was necessary to show how this problem came to be and even if you have no trouble at all, perhaps you’ll consider using Kotest. It’s awesome!

Gradle

I’m not sure, but this issue could happen with Gradle as well, when you are migrating to JUnit 5.

The creators of Jest (Facebook) have found a better way: Snapshot tests!

How does it work?

Take a look at the following assertion:

it('should create a foo bar object', () => {
  const result = foo.bar()
  expect(result).toMatchSnapshot()
})

toMatchSnapshot() takes whatever you give to expect(), serializes it and saves it into a file. The next test run will compare the expected value to the stored snapshot and will fail if they don’t match. Jest shows a nicely formatted error message and diff view on failed tests.

This is really useful with generated HTML and/or testing UI behaviour. Just call the method and let it compare to the snapshot.

Updating snapshots

You added something to your code and the snapshot has to be updated? No problem:

jest --updateSnapshot

If you’re using the Jest watcher it’s even simpler. Just press u to update all snapshots or press i to update the snapshots interactively.

What about objects with generated values?

Here’s an example with a randomized id:

it('should fail every time', () => {
  const ship = {
    id: Math.floor(Math.random() * 20),
    name: 'USS Defiant'
  }

  expect(ship).toMatchSnapshot()
})

The id will change on every test run, so this test will fail every time. Well, shit? Nope. Jest got you covered:

it('should create a ship', () => {
  const ship = {
    id: Math.floor(Math.random() * 20),
    name: 'USS Defiant'
  }

  expect(ship).toMatchSnapshot({
    id: expect.any(Number)
  })
})

Jest will now only compare the type of the id and the test will pass.

For certain objects like a date, there is another possibility:

Date.now = jest.fn(() => 1528902424828)

A call of Date.now() will call the mock method and always return the same value.

Some advice

1. Always commit your snapshots! If they are missing, CI systems will always create new snapshots and the tests will become useless.

2. Snapshot tests are an awesome tool, but don’t be too lazy. They are no replacement for other assertion types, especially if you’re working test-driven. Rather use them alongside your other tests.

3. Write meaningful test names. Well, you heard that one before, didn’t you? Really, it helps a lot when tests fail or you have to look inside a snapshot file. Jest takes a test name as an id inside a snapshot file. That’s why you have to update a snapshot after changing the name.

Set-up Jest

Vue CLI

You are using Vue CLI? Consider yourself lucky, the set-up of Jest could not be simpler:

yarn add --dev jest @vue/cli-plugin-unit-jest
vue invoke unit-jest

Vue CLI will do the rest and also create an example spec for the HelloWorld component.

DIY

Install all necessary dependencies:

yarn add --dev @vue/test-utils babel-jest jest jest-serializer-vue vue-jest

Create jest.config.js in your project root directory:

module.exports = {
  moduleFileExtensions: ['js', 'json', 'vue'],
  transform: {
    '^.+\\.vue$': 'vue-jest',
    '^.+\\.js?$': 'babel-jest'
  },
  moduleNameMapper: {
    '^@/(.*)$': '<rootDir>/src/$1'
  },
  snapshotSerializers: ['jest-serializer-vue'],
  testMatch: ['<rootDir>/src/tests/**/*.spec.js']
}

Please adjust the paths in moduleNameMapper and testMatch to your project.

You should also add a modern JavaScript preset to your .babelrc file:

{
  "presets": ["es2015"]
}

Optional step

Add the following line to your .gitignore file:

/coverage

The set-up is now complete. Let’s write a test file!

Writing tests

Here is a little example Vue component:

<template>
  <ul class="demo">
    <li class="demo__item" v-for="value in values" :key="value">{{ value }}</li>
  </ul>
</template>

<script>
export default {
  name: 'demo',
  data () {
    return {
      values: []
    }
  },
  created() {
    this.fetchValues()
  },
  methods: {
    async fetchValues() {
      const response = await fetch('/api/demo/values')
      this.values = await response.json()
    }
  }
}
</script>

The component Demo will fetch some values after its creation and display them in an unordered list. This example is quite simple, but testing is a little more complex due to the usage of fetch, async and await.

Of course, there are some tools to help us:

yarn add --dev fetch-mock flush-promises

Now, let’s write a test:

import { shallow } from '@vue/test-utils'
import fetchMock from 'fetch-mock'
import flushPromises from 'flush-promises'
import Demo from '@/components/Demo.vue'

const values = [
  'foo',
  'bar'
]

describe('Demo.vue', () => {
  beforeEach(() => {
    fetchMock.get('/api/demo/values', values)
  })

  it('renders component', async () => {
    const wrapper = shallow(Demo)
    await flushPromises()

    expect(wrapper.vm.values).toEqual(values)
  })

  afterEach(() => {
    fetchMock.restore()
  })
})

What’s going on?

1. shallow from Vue Test Utils creates a wrapper of the rendered and mounted component, any child components will be stubs. If you need child components in your test, please use mount instead of shallow.

2. fetchMock will create a mocked version of the Fetch API. In this case it will return the defined values for a GET request to /api/demo/values. If you send a request that’s not defined in fetchMock, it will throw an exception and break your tests.

3. The test itself is defined as async to use await for flushPromises(). It will wait until the mocked request is finished and the values are stored in the component’s data.

4. You can now access the data property values and compare the content to the response of the mocked HTTP request.

Conclusion

Setting up Jest for Vue is easy, even if you have to do it manually.

A little warning: setting up Jest for an existing app can be tedious. The current AngularJS app of our customer can’t be tested with Jest, at least for now. The AngularJS HTTP mock does not work and I haven’t figured out the problem yet.

But enough of Angular: the real deal comes with the testing itself. Async/await is a nice and simple way for testing asynchronous behaviour. I don’t think this could be easier and it’s a reliable method with the power of modern JavaScript. Try to imagine what the demo test would look like in ES5 …