Writing tests, discovering and running them with Grok.testing¶
| Author: | ulif |
|---|
Prerequisites¶
You need Grok 0.13 or newer installed. If you are developing on an older version of Grok, have a look at z3c.testsetup How-To instead.
To follow along with this How-To, you can install a blank new project called Sample.:
$ cd /tmp
$ grokproject Sample
$ cd Sample
Step by step¶
You don’t have to download anything else, because Grok already includes all the packages you need.
Test that the project’s testrunner works when you run ./bin/test. You should see something like this:
$ ./bin/test
Running tests at level 1
Total: 0 tests, 0 failures, 0 errors in 0.000 seconds.
Now you’re ready to start adding your tests. There are several different kinds of tests and we’ll scratch the surface of each one. The tests are either doctest or python and the difference is that you either write them in pure python or as text that you embedd in the doc strings of your classes or in separate plain-text (.txt) files. grok.testing is about discovering tests, not writing them or running them.
Let’s write a very simple unit test for the app.py that has been created.:
$ cd src/sample
$ mkdir app_tests; cd app_tests
$ touch __init__.py
$ emacs test_app.py
Notice the importance of creating an __init__.py file in that new directory to make it a valid Python package. Here’s some example code that is really silly but at least proves that the test is found during runs of testrunner:
"""
Do a Python test on the app.
:Test-Layer: python
"""
import unittest
from sample.app import Sample
class SimpleSampleTest(unittest.TestCase):
"Test the Sample application"
def test1(self):
"Test that something works"
grokapp = Sample()
self.assertEqual(list(grokapp.keys()), [])
The next thing is to tell grok.testing where to find this test. That’s achieved by creating a file called tests.py in the ‘src/sample’ directory with the following content:
import grok
test_suite = grok.testing.register_all_tests('sample')
The ‘sample’ string here denotes our sample package, which we want
to be scanned for test files. We could also pass any other package
name, which is available at runtime, in ‘dotted name’ notation.
That’s it! How cool is that? I love the Just Works’ism of writing “register_all_tests(‘sample’)” and it does it.
Let’s now crack on with a “doc test”. One way to get started on a doc test is to run grok in debug mode (typing ‘./bin/zopectl debug’) and when you’re done, copy and paste what you’ve written in the interactive prompt. Here’s a really simple doctest copy and paste after having run ./bin/zopectl debug. I call this file doctest.txt and I place it in the app_tests directory too:
Do a simple doctest test on the app.
************************************
:Test-Layer: unit
When you create an instance there are no objects in it::
>>> from sample.app import Sample
>>> grokapp = Sample()
>>> list(grokapp.keys())
[]
Make sure it is found by the testrunner:
$ ./bin/test
Running tests at level 1
Running unit tests:
Running:
..
Ran 2 tests with 0 failures and 0 errors in 0.007 seconds.
These tests are just making sure that you’re code is working but we haven’t yet tried to run any tests in a full blown Grok environment. That’s called functional testing (or integration testing). The first test we’ll make is a functional test in python. The setup of the complete Grok environment is called the “functional layer” Fortunately grokproject sets one up for you automatically which you can use in your functional test cases. The magic you need is the instance object FunctionalLayer which you’ll find in the file testing.py. Again pay attention to the marker in the doc string is set to python and also bare in mind that the test is extremely simple. Save this as functional.py in the app_tests/ directory.:
"""
Do a functional test on the app.
:Test-Layer: python
"""
from sample.app import Sample
from sample.testing import FunctionalLayer
from zope.app.testing.functional import FunctionalTestCase
class SampleFunctionalTest(FunctionalTestCase):
layer = FunctionalLayer
class SimpleSampleFunctionalTest(SampleFunctionalTest):
""" This the app in ZODB. """
def test_simple(self):
""" test creating a Sample instance into Zope """
root = self.getRootFolder()
root['instance'] = Sample()
self.assertEqual(root.get('instance').__class__, Sample)
Apologies for the stupidity of the test but at least it’s picked up the next time we run bin/test:
$ ./bin/test
Running tests at level 1
Running unit tests:
Running:
..
Ran 2 tests with 0 failures and 0 errors in 0.021 seconds.
Running sample.testing.FunctionalLayer tests:
Set up sample.testing.FunctionalLayer in 1.756 seconds.
Running:
.
Ran 1 tests with 0 failures and 0 errors in 0.022 seconds.
Tearing down left over layers:
Tear down sample.testing.FunctionalLayer ... not supported
Total: 3 tests, 0 failures, 0 errors in 1.956 seconds.
As you can see, running the functional test is a lot slower. The first two tests took 0.007 seconds and now all three tests took 1.96 seconds. This is why developers sometimes refer to unit tests as “fast tests”.
Choosing to write unit tests versus functional tests depends upon what you want to achieve with your testing. Unit tests verify the correct behaviour of code in isolation, useful for ensuring that your code is loosely coupled. They are a quicker form of checking for simple errors than clicking around in a web browser, and they can help find bugs by exposing unseen edge cases. Functional testing ensures that the parts of your application work together as a whole, and are useful for ensuring that your application behaves as desired. If you are new to testing, try both forms of testing, as you will often find yourself mixing and matching from both test approaches depending upon what you want your tests to do.
The final type of test is a functional doc test which is really sexy in its simplicity. Create a file in app_tests/ called functional.txt and let it have the following content:
Do a functional doctest test on the app.
****************************************
:Test-Layer: functional
Test creating a Sample instance into Grok::
>>> from sample.app import Sample
>>> root = getRootFolder()
>>> root['instance'] = Sample()
>>> root.get('instance').__class__.__name__
'Sample'
We now have one python unit test, a doc test, a python functional test and a function doc test. Let’s check that they all run:
$ ./bin/test
Running tests at level 1
Running unit tests:
Running:
..
Ran 2 tests with 0 failures and 0 errors in 0.005 seconds.
Running sample.FunctionalLayer tests:
Set up sample.FunctionalLayer in 1.755 seconds.
Running:
.
Ran 1 tests with 0 failures and 0 errors in 0.005 seconds.
Running sample.testing.FunctionalLayer tests:
Tear down sample.FunctionalLayer ... not supported
Running in a subprocess.
Set up sample.testing.FunctionalLayer in 1.771 seconds.
Running:
.
Ran 1 tests with 0 failures and 0 errors in 0.004 seconds.
Tear down sample.testing.FunctionalLayer ... not supported
Total: 4 tests, 0 failures, 0 errors in 4.896 seconds.
Further information¶
This how-to is meant to be an introduction to get you started on writing tests, how those tests are discovered, and how you can run them. To help making this How-to as short as possible, I’ve skipped...
- The discussing the various options for z3c.testsetup which is the base of grok.testing.
- The options on the test runner (./bin/test –help).
- Doctests embedded as docstrings inside the Grok classes.
- Browser testing with requests (example here)
Hopefully this how-to can improve over time to make things even simpler and dummy-proof but looking back you’ll have to admit that we managed to get a lot done with very little configuration work.
