Coding · ·

Advanced testing

This guide is for writing unit tests for your NEA. Unit tests can help you score marks, but aren't required. If you don’t use them, make sure you include manual testing with test plans/tables. Often, you'll use a mix of both. Whatever you do, justify your choices of tests and test data.

We've covered the basics of modules and testing before. Here we'll tackle more complex scenarios, including functions that:


Alternatives to expect().toBe()

When writing tests, you'll probably start with expect().toBe() or expect.toEqual(). But there are lots of other functions you can use too, such as expect().toContain().


Testing pure functions

Pure functions are easiest to test because:

Given this web page as an example:

<!-- index.html -->
<!doctype html>
<title>Testing 123</title>
<script>

console.log(wordsToArray('This is really great'))

function wordsToArray (str) {
  return str.split(' ')
}

</script>

We want to test the wordsToArray() function. It's relatively easy because it's pure:

To make our pure function testable we need to:

We end up with these 3 files instead of 1 file:

<!-- index.html -->
<!doctype html>
<title>Testing 123</title>
<script src="script.js" type="module"></script>
// script.js
import { wordsToArray } from './words.js'
console.log(wordsToArray('This is really great'))
// words.js
export function wordsToArray (str) {
  return str.split(' ')
}

Open it in a browser and it should still work. However, now we also have words.js in a way in which we can import it in Bun for testing. We add a test file:

// words.test.js
import { expect, test } from 'bun:test'
import { wordsToArray } from './words.js'

test('wordsToArray()', () => {
  expect(wordsToArray('Hi there')).toEqual(['Hi', 'there'])
  // ... more tests
})

And run it with:

bun --watch test

Ideally, we've actually written the tests before we've written the file we want to test. In which case, we'll see the tests failing. Then, we code the function wordsToArray() and once it's finished hopefully we see all the tests passing.

Under // ... more tests you should have tests that cover all bases. In particular:

For the above function:

Often, it's a good idea to have multiple calls to test() for each function you're testing, each with several expect()s inside.


Testing functions that read global state

Given this web page:

<!-- index.html -->
<!doctype html>
<title>Testing 123</title>
<script>
let state = {
  nums: [5, 7, 3],
  paused: true
}

console.log(state.nums)
console.log(addNums())

function addNums () {
  return state.nums.reduce((acc, v) => acc + v)
}
</script>

We want to test the function addNums(). However, rather than working on parameters alone, this function is no longer pure as it reads a global variable state.nums.

Firstly, it might be better to pass state.nums as a parameter instead. But we'll assume there's a good reason not to do that.

We're going to split the HTML file into 3 files as before, plus one additional state.js file to deal with the global state:

<!-- index.html -->
<!doctype html>
<title>Testing 123</title>
<script type="module" src="script.js"></script>
// script.js
import { addNums } from './calculator.js'
import { state } from './state.js'

console.log(state.nums)
console.log(addNums())
// state.js
export let state = {
  nums: [5, 7, 3],
  paused: true
}
// calculator.js
import { state } from './state.js'

export function addNums () {
  return state.nums.reduce((acc, v) => acc + v)
}

Notice how we define our 'global' variables in a separate file (state.js here), and import that into any file that needs it (both script.js and calculator.js here). The import'ed file is only run once, and both files have a variable called state that points to the same object.

We'll add tests:

// calculator.test.js
import { expect, test } from 'bun:test'
import { addNums } from './calculator.js'

test('addNums()', () => {
  expect(addNums()).toBe(15)
  // ... more tests
})

... so this works, but just based on [5, 7, 3] being added, which isn't very useful by itself!

Let's try to modify state.nums to test different numbers:

// calculator.test.js
import { expect, test } from 'bun:test'
import { addNums } from './calculator.js'

test('addNums()', () => {
  expect(addNums()).toBe(15)
  state.nums = [3, 6] // won't work 
  expect(addNums()).toBe(9)
})

Sadly, we get the error ReferenceError: state is not defined. That's because state isn't truly global, so it isn't accessible here. One solution is to make it truly global with globalThis.state = { ... }, but we're going to try to avoid doing that.

The solution is to import our state inside the test file:

// calculator.test.js
import { expect, test } from 'bun:test'
import { addNums } from './calculator.js'
import { state } from './state.js' // importing here

test('addNums()', () => {
  expect(addNums()).toBe(15)
  state.nums = [3, 6]
  expect(addNums()).toBe(9)
  state.nums = [4, 10, 20]
  expect(addNums()).toBe(34)
})

... now it works! Within the test we can mutate state and because it points to the same object that can be seen in calculator.js, the change is reflected in how the object is seen in that file.

Note that this works because in JavaScript identifiers for objects, arrays and functions are just pointers to some bit of memory. If instead our global state was just export let state = 5 then each file would get its own copy of that state. This is a good reason to put your global state stuff in an object, rather than lots of separate variables that would get passed around by value instead of reference.


Testing functions that modify global state

This time we have a function that uses a parameter, but also modifies global state by pushing a value into state.nums:

// calculator.js
import { state } from './state.js'

export function addNumber (num) {
  state.nums.push(num)
}

Similar to before, we just need to ensure state is import'ed in our test file:

// calculator.test.js
import { expect, test } from 'bun:test'
import { addNumber } from './calculator.js'
import { state } from './state.js' // need this

test('addNumber()', () => {
  state.nums = []
  expect(state.nums.length).toBe(0)
  addNumber(5)
  expect(state.nums.length).toBe(1)
  expect(state.nums).toEqual([5])
  addNumber(9)
  expect(state.nums).toEqual([5, 9])
})

Testing functions that have side-effects

WARNING: This is advanced. It may be better to refactor your code and only test the functions which don't have side-effects.

Ultimately your code will create a GUI, and that means at some point you need to manipulate the DOM in some way, draw with a <canvas>'s ctx, alert() a message to the user, etc.

Bun doesn't run in a browser. That means it has no window, document or <canvas> elements.

Here's our page we want to test:

<!-- index.html -->
<!doctype html>
<title>Hi</title>
<div id="score"></div>
<canvas id="canvas" width="500" height="500"></canvas>
<script>
let ctx = canvas.getContext('2d')
console.log(ctx.fillStyle)
doThing()

function doThing (bad = false) {
  ctx.fillRect(30, 50, 60, 70)
  score.innerHTML = 5
  if (bad) alert('Oops')
  return true
}
</script>

The function doThing():

Before we think about testing this function, do consider that it might be better to:

We treat ctx similar to how we made state global before, so it can be import'ed anywhere:

<!-- index.html -->
<!doctype html>
<title>Hi</title>
<div id="score"></div>
<canvas id="canvas" width="500" height="500"></canvas>
<script src="script.js" type="module"></script>
// script.js
import { ctx } from './canvas.js'
import { doThing } from './things.js'

console.log(ctx.fillStyle)
doThing()
// canvas.js
export let ctx = canvas.getContext('2d')
// things.js
import { ctx } from './canvas.js'

export function doThing (bad = false) {
  ctx.fillRect(30, 50, 60, 70)
  score.innerHTML = 5
  if (bad) alert('Oops')
  return true
}

... notice how canvas.js export's ctx, which is import'ed from other files. Each file that imports it sees the same ctx.

Let's try to test it:

// things.test.js
import { expect, test } from 'bun:test'
import { doThing } from './things.js'

test('doThing()', () => {
  expect(doThing()).toBe(true)
})

... oops, we get ReferenceError: canvas is not defined where we try to canvas.getContext('2d').

Bun doesn't know there's an HTML element with id="canvas", and we can't use document.querySelector('#canvas') either as there's no document in Bun.

Mocks to the rescue! We're going to mock the canvas.js file/module, making our own fun ctx thing that we craft ourselves:

// things.test.js
import { expect, test, mock } from 'bun:test'

// we mock the canvas.js module
// instead returning an object with a property called ctx
// ...which has a method `fillRect()` that... does nothing.
mock.module('./canvas.js', () => {
  return {
    ctx: {
      fillRect: () => {}
    }
  }
})

// when using the mock we have to
// import the file we're testing after we've added the mock
// and with different syntax, called 'dynamic imports'
let { doThing } = await import('./things.js')

test('doThing()', async () => {
  expect(doThing()).toBe(true)
})

That's the pesky ctx dealt with. Now we're going to deal with score.innerHTML = 5 which is in the file we're testing. This made use of the quirk where as we had an element with id="score" it's available to JS as a variable. Instead, let's do it the correct way:

// things.js
import { ctx } from './canvas.js'

export function doThing (bad = false) {
  ctx.fillRect(30, 50, 60, 70)
  let score = document.querySelector('#score')
  // ... or import 'score' from a file like state.js
  // ... or inject 'score' as a parameter
  score.innerHTML = 5
  if (bad) alert('Oops')
  return true
}

... notice the inline comments above which suggest 2 alternative (and likely better) ways to do this instead. The import option would have involved a file with export let score = document.querySelector('#score').

When doThing() runs, document.querySelector('#score') will fail as document isn't a thing outside the browser. Let's fix that by making it a thing:

// things.test.js
import { expect, test, mock } from 'bun:test'

mock.module('./canvas.js', () => {
  return {
    ctx: {
      fillRect: () => {}
    }
  }
})

let { doThing } = await import('./things.js')

// create our own 'document' that has just what we need in it
// making it global by attaching it to globalThis
globalThis.document = {
  querySelector: () => ({ innerHTML: '' })
}

test('doThing()', async () => {
  expect(doThing()).toBe(true)
})

This will work. A better method would have involved exporting score from eg state.js, or taking it as a parameter to doThing().

Back to the ctx mock that we made, we may want to use the same mock for lots of tests across lots of test files. One way to do that is to make a kind of shadow canvas.js called eg canvas.mock.js which we use to replace canvas.js for the purpose of our tests:

// canvas.mock.js
export let ctx = {
  fillRect () {}
  // ... other methods and properties that ctx usually has
}
// things.test.js
import { expect, test, mock } from 'bun:test'

// note the line below
mock.module('./canvas.js', () => import('./canvas.mock.js'))
let { doThing } = await import('./things.js')

globalThis.document = {
  querySelector: () => ({ innerHTML: '' })
}

test('doThing()', async () => {
  expect(doThing()).toBe(true)
})

Finally, we're going to deal with the alert('Oops') that's in the function we're testing.

We replace alert() with an empty function as we don't want to actually alert() on the console when running the test:

// things.test.js
import { expect, test, mock } from 'bun:test'

mock.module('./canvas.js', () => import('./canvas.mock.js'))
let { doThing } = await import('./things.js')

globalThis.document = {
  querySelector: () => ({ innerHTML: '' })
}

globalThis.alert = () => {} // this is the new line

test('doThing()', async () => {
  expect(doThing()).toBe(true)
  expect(doThing(true)).toBe(true) // alert() would have been called in here
})

We can even wrap that empty function in mock(), which gives us the ability to inspect it to see how many times it was called, and what parameters it was sent when it was called:

// ...things.test.js
globalThis.alert = mock(() => {}) // wrapped in mock()

test('doThing()', async () => {
  expect(doThing()).toBe(true)
  expect(doThing(true)).toBe(true)
  expect(alert).toHaveBeenCalled()
  expect(alert).toHaveBeenCalledWith('Oops')
})

It's worth pointing out that using alert() usually isn't a great idea in a function you want to test. Instead, probably return a value and let the code that calls the function do an alert() if needed, and don't bother testing that, just the return value. More generally, try to separate out your functions that do just logical things vs those that have side-effects, and focus on tests for the former.

If you want to properly test DOM related stuff, consider reading this page on DOM testing in the Bun docs.


Testing non-deterministic functions

Given this code:

console.log(randomItem(['a', 'b', 'c'])) // eg 'b'

function randomItem (arr) {
  return arr[Math.floor(Math.random() * arr.length)]
}

... we notice it makes use of Math.random() so is non-deterministic. That is, given a particular input, we don't know exactly what the return value will always be.

But we could test:

  1. is its output always one of the available options?
  2. if we run it lots of times, is it statistically giving the right output?
  3. given a biased/fixed Math.random(), what should it output?

Here's how we can test for 1.:

// always returns an item from the list
let items = ['a', 'b', 'c']
for (let i = 0; i < 10; i++) {
  expect(items).toContain(randomItem(items))
}

... we should make sure we have a loop that runs it enough times. 10 feels about right to me.

Here's how we can test for 2.:

// check all items are approximately as likely as each other
let items = ['a', 'b', 'c']
let counts = {}
for (let item of items) counts[item] = 0
for (let i = 0; i < 9_000; i++) {
  let item = randomItem(items)
  counts[item]++
}
for (let key in counts) {
  expect(counts[key]).toBeGreaterThan(2700)
  expect(counts[key]).toBeLessThan(3300)
}

And how we can test for 3.:

// deterministic by fixing Math.random
let tmp = Math.random
Math.random = () => 0
expect(randomItem(['a', 'b', 'c'])).toBe('a')
Math.random = () => 0.999
expect(randomItem(['a', 'b', 'c'])).toBe('c')
Math.random = tmp // reset it back to normal

... this last example works because Math.random() is already a global. When twiddling with Math.random(), be sure to reset it to its original functionality after, and remember it returns between 0 and 1, including 0 but not including 1.


Want to know more about tests?