.. index:: BATS ************************************ BATS - Bash Automated Testing System ************************************ The CoreOS distribution supports writing tests using shell syntax by providing the `bats` command. If you want to use `bats`, you will need the following CoreOS packages: - bats - bats-file - bats-assert Overview of BATS ================ A BATS test can be as simple as a single .bats file. For example: .. code-block:: bash #!/usr/bin/env bats bats_load_library bats-support bats_load_library bats-assert @test "can output to stdout" { run echo hello assert_output 'hello' } You can run it using the command `bats .bats` This will give you the following output: .. code-block:: bash sam@SAVE:~/Projects/tests$ bats .bats .bats ✓ can output to stdout 1 test, 0 failures The run command ================ In shell tests, you often need to run commands and capture their output, exit status, and error messages. The run command provided by `bats` allows you to execute commands within your test cases and collect this information for later assertion and validation. The run command will make the following variables available: - `${status}`: exit code of the command run by `run` - `${output}`: combined content of `stdout` and `stderr` - `${lines[@]}`: array of lines of the output - `${BATS_RUN_COMMAND}`: command run by the `run` command .. code-block:: bash @test "invoking foo with a nonexistent file prints an error" { run foo nonexistent_filename [ "$status" -eq 1 ] [ "$output" = "foo: no such file 'nonexistent_filename'" ] [ "$BATS_RUN_COMMAND" = "foo nonexistent_filename" ] } The `run` command accepts some parameters: - `-N`: Expect N as exit status and fail otherwise - `-!`: Expect non-zero exit status and fail if the command succeeds. - `--keep-empty-lines`: don't remove empty lines from `${lines}` - `--separate-stderr`: Use separate variables for stderr `${stderr}` and `${stderr_lines[@]}` .. code-block:: bash @test "invoking foo without arguments prints usage" { run -1 foo [ "${lines[0]}" = "usage: foo " ] } The bats-assert helper ====================== The `bats-assert` helper provides some functions to create more readable tests. These assertions use the variables created by the `run` command and can be used as follows: .. code-block:: bash @test 'assert_output()' { run echo 'have' assert_output 'want' } The following functions are provided: - `assert` and `refute`: Assert that a given expression evaluates to true or false. - `assert_equal`: Assert that two parameters are equal. - `assert_not_equal`: Assert that two parameters are not equal. - `assert_success` and `assert_failure`: Assert that the exit status is 0 or 1. - `assert_output` and `refute_output`: Assert that the output does (or does not) contain the given content. - `assert_line` and `refute_line`: Assert that a specific line of the output does (or does not) contain the given content. - `assert_regex` and `refute_regex`: Assert that a parameter matches (or does not match) the given pattern. The bats-file helper ==================== The `bats-file` helper provides functions to help work with files in tests: **Test File Types:** - `assert_exists` and `assert_not_exists`: Check if a file or directory exists. - `assert_file_exists` and `assert_file_not_exists`: Check if a file exists. - `assert_dir_exists` and `assert_dir_not_exists`: Check if a directory exists. - `assert_link_exists` and `assert_link_not_exists`: Check if a link exists. - `assert_block_exists` and `assert_block_not_exists`: Check if a block special file exists. - `assert_character_exists` and `assert_character_not_exists`: Check if a character special file exists. - `assert_socket_exists` and `assert_socket_not_exists`: Check if a socket exists. - `assert_fifo_exists` and `assert_fifo_not_exists`: Check if a fifo special file exists. **Test File Attributes:** - `assert_file_executable` and `assert_file_not_executable` - `assert_file_owner` and `assert_file_not_owner` - `assert_file_permission` and `assert_not_file_permission` - `assert_file_size_equals` - `assert_size_zero` and `assert_size_not_zero` - `assert_file_group_id_set` and `assert_file_not_group_id_set` - `assert_file_user_id_set` and `assert_file_not_user_id_set` - `assert_sticky_bit` and `assert_no_sticky_bit` **Test File Content:** - `assert_file_empty` and `assert_file_not_empty` - `assert_file_contains` and `assert_file_not_contains` - `assert_symlink_to` and `assert_not_symlink_to` **Working with a temporary directory:** - `temp_make` and `temp_del` Pre- and Post-test case hooks ============================== In some cases, it's useful to have a function that runs before or after each test case in a bats file. A function named `setup` will run before each test case, and a function named `teardown` will run after each test case. This example creates a directory in the setup function but lacks a teardown that removes the directory. The second time the setup function is run, the setup will fail as the directory already exists: .. code-block:: bash #!/usr/bin/env bats bats_load_library bats-support bats_load_library bats-assert bats_load_library bats-file setup() { mkdir tmp echo 'a' >> ./tmp/test } @test "test contains a single a I" { assert_file_contains ./tmp/test '^a$' } @test "test contains a single a II" { assert_file_contains ./tmp/test '^a$' } .. code-block:: bash sam@SAVE:~/Projects/tests$ bats test.bats test.bats ✓ test contains a single a I ✗ test contains a single a II (from function `setup' in test file test.bats, line 8) `mkdir tmp' failed mkdir: cannot create directory ‘tmp’: File exists 2 tests, 1 failure This can be easily fixed by adding a teardown function: .. code-block:: bash #!/usr/bin/env bats bats_load_library bats-support bats_load_library bats-assert bats_load_library bats-file setup() { mkdir tmp echo 'a' >> ./tmp/test } teardown() { rm -rf ./tmp } @test "test contains a single a I" { assert_file_contains ./tmp/test '^a$' } @test "test contains a single a II" { assert_file_contains ./tmp/test '^a$' } .. code-block:: bash sam@SAVE:~/Projects/tests$ bats test.bats test.bats ✓ test contains a single a I ✓ test contains a single a II 2 tests, 0 failures Pre- and Post-test file hooks ============================= To run some code before executing a test file or after executing it, the functions `setup_file` and `teardown_file` can be used. The last example could be refactored to only create the tmp directory once: .. code-block:: bash #!/usr/bin/env bats bats_load_library bats-support bats_load_library bats-assert bats_load_library bats-file setup_file() { export DIR="./tmp" export FILE="${DIR}/test" mkdir "${DIR}" } teardown_file() { rm -rf "${DIR}" } setup() { echo 'a' >> "${FILE}" } teardown() { rm "${FILE}" } @test "test contains a single a I" { assert_file_contains "${FILE}" '^a$' } @test "test contains a single a II" { assert_file_contains "${FILE}" '^a$' } Multiple files ============== With `bats`, a file is a test suite. If you have multiple `bats` files in a directory and you provide the directory in the `bats` command line, `bats` will execute all the test suites. Example: `bats .` .. code-block:: bash sam@SAVE:~/Projects/tests$ bats . ./first.bats ✓ can run our script ✗ second test (in test file ./first.bats, line 27) `false' failed ./second.bats ✓ multi file ./test.bats ✓ test contains a single a I ✓ test contains a single a II 5 tests, 1 failure Pre- and Post-suite hooks ========================= If you want to execute the same function before each test suite or after each test suite, create a file named `setup_suite.bash`. In this file, create a function named `setup_suite()` and another named `teardown_suite()`. Exporting the test results ========================== Test results can be exported using the JUnit XML format. This can then be used in other tools and merged with other JUnit XML formats to generate a final test report. Example: .. code-block:: bash sam@SAVE:~/Projects/tests$ bats . -F junit This will produce the following XML content on stdout: .. code-block:: xml (in test file ./first.bats, line 27) `false' failed Going further ============= `bats` scripts can be checked with shellcheck for common mistakes. The `bats-assert` add-on provides many helper functions to perform assertions with a more readable syntax than the shell's built-in syntax. See https://github.com/bats-core/bats-assert The `bats-file` add-on provides helper functions to check for files. See https://github.com/bats-core/bats-file/ You can find a list of projects using `bats` on this page: https://github.com/bats-core/bats-core/wiki/Projects-Using-Bats