1 The testing framework used is [PHPUnit](https://phpunit.de/); it can be installed with [Composer](https://getcomposer.org/), which is a dependency management tool.
3 ## Setup a testing environment
9 - a system-wide version, e.g. installed through your distro's package manager (eg. `sudo apt install composer`)
10 - a local version, downloadable [here](https://getcomposer.org/download/). To update a local composer installation, run `php composer.phar self-update`
13 ### Install Shaarli development dependencies
23 Xdebug must be installed and enable for PHPUnit to generate coverage reports. See http://xdebug.org/docs/install.
26 # for Debian-based distributions
27 $ aptitude install php-xdebug
33 Then add the following line to `/etc/php/<PHP_VERSION>/cli/php.ini`:
36 zend_extension=xdebug.so
41 Run `make test` and ensure tests return `OK`. If tests return failures, refer to PHPUnit messages and fix your code/tests accordingly.
43 By default, PHPUnit will run all suitable tests found under the `tests` directory. Each test has 3 possible outcomes:
46 - `F` - failure: the test was run but its results are invalid
47 - the code does not behave as expected
48 - dependencies to external elements: globals, session, cache...
49 - `E` - error: something went wrong and the tested code has crashed
50 - typos in the code, or in the test code
51 - dependencies to missing external elements
53 If Xdebug has been installed and activated, two coverage reports will be generated:
55 - a summary in the console
56 - a detailed HTML report with metrics for tested code
57 - to open it in a web browser: `firefox coverage/index.html &`
59 ### Executing specific tests
61 Add a [`@group`](https://phpunit.de/manual/current/en/appendixes.annotations.html#appendixes.annotations.group) annotation in a test class or method comment:
65 * Netscape bookmark import
68 class BookmarkImportTest extends PHPUnit_Framework_TestCase
74 To run all tests annotated with `@group WIP`:
76 $ vendor/bin/phpunit --group WIP tests/
79 ### Running tests inside Docker containers
81 Test Dockerfiles are located under `tests/docker/<distribution>/Dockerfile`,
82 and can be used to build Docker images to run Shaarli test suites under common
85 Dockerfiles are provided for the following environments:
87 - `alpine36` - [Alpine 3.6](https://www.alpinelinux.org/downloads/)
88 - `debian8` - [Debian 8 Jessie](https://www.debian.org/DebianJessie) (oldstable)
89 - `debian9` - [Debian 9 Stretch](https://wiki.debian.org/DebianStretch) (stable)
90 - `ubuntu16` - [Ubuntu 16.04 Xenial Xerus](http://releases.ubuntu.com/16.04/) (LTS)
92 What's behind the curtains:
94 - each image provides:
96 - Shaarli PHP dependencies (OS packages)
97 - test PHP dependencies (OS packages)
99 - the local workspace is mapped to the container's `/shaarli/` directory,
100 - the files are rsync'd so tests are run using a standard Linux user account
101 (running tests as `root` would bypass permission checks and may hide issues)
102 - the tests are run inside the container.
104 To run tests inside a Docker container:
107 # build the Debian 9 Docker image for unit tests
108 $ cd /path/to/shaarli
109 $ cd tests/docker/debian9
110 $ docker build -t shaarli-test:debian9 .
112 # install/update 3rd-party test dependencies
113 $ composer install --prefer-dist
115 # run tests using the freshly built image
116 $ docker run -v $PWD:/shaarli shaarli-test:debian9 docker_test
118 # run the full test campaign
119 $ docker run -v $PWD:/shaarli shaarli-test:debian9 docker_all_tests