1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
|
=============
Test Design
=============
Our tests are based on the unittest framework. All tests of modules
in a directory usually reside in a subdirectory **test**. There are
basically two types of unit tests:
- **Interpreter Level tests**. They run at the same level as PyPy's
interpreter.
- **Application Level tests**. They run at application level which means
that they look like straight python code but they are interpreted by PyPy.
Both types of tests need an objectspace they can run with (the interpreter
dispatches operations on objects to an objectspace). If you run a test you
can usually give the '-S' or '-T' switch for using the Standard or the Trivial
ObjectSpace respectively.
Writing a test
--------------
The best reference is to go to some test files and look how they are done.
Almost all tests start with the following lines::
import autopath
from pypy.tool import test
of which the first line determines the package path automatically by searching
for **pypy** among the parents of the test-file's directory. The second
line imports PyPy's test-hook of which ''test.main()'' is the most important
and is usually the last line of the testfile. Further below you will find
some preliminary information about a new test module that was recently
developed at the Amsterdam Sprint (Dec. 2003) and still is under development.
Note that we are using a unittest2 framework (but not for long)
Command line tool test_all
--------------------------
You can run almost all of PyPy's tests by invoking::
python test_all.py
which will run all tests against the Standard Object Space. If you want
to test against the TrivialObjectSpace then issue::
python test_all.py -T
For more switches invoke "python test_all.py -h".
This used to be reversed -- the TrivialObjectSpace was the default, and
you had to specify the StandardObjectSpace. You may find some documentation
that has not been changed to reflect the new reality.
-------------------------------------------------------------------------------
.. _mail: http://codespeak.net/pipermail/pypy-dev/2003q2/000789.html
New unit testing framework (not ready yet)
------------------------------------------
Current implementation
~~~~~~~~~~~~~~~~~~~~~~
The following picture is an UML class diagram of the framework. This
diagram is taken from the source code, which can currently be found
at src/pypy/tool/newtest.py as of 2003-12-19. (Most probably, the name
of the file will change, however.)
::
+------------+ 1 1 +----------+
| TestResult |----------------------------------------| TestItem |
| (abstract) | +----------+
+------------+ | *
A |
- |
| | 1
+----------------------------+-----------+ +-----------+
| | | | TestSuite |
+-------------------------+ +---------+ +---------+ +-----------+
| TestResultWithTraceback | | Success | | Skipped | A
| (abstract) | +---------+ +---------+ |
+-------------------------+ | loaded by
A A |
- - | *
| | +------------+
| | | TestCase |
+-------+ +---------+ | (abstract) |
| Error | | Failure | +------------+
+-------+ +---------+ A
-
|
|
concrete test
case classes
Like the unittest framework of Python, our framework implements
tests as test methods in TestCase classes. Custom test case classes
derive from the shown TestCase class, defined in this module. As in
Python's unittest, a test case class can contain setUp and tearDown
methods. Additionally, it contains a method 'skip' which can be
called to stop a test prematurely. This won't be counted as a failure
or error.
Test cases are loaded by a TestSuite class via the method init_from_dir.
This method will read all test modules in and below a specified
directory, inspect them for classes derived from TestCase (i. e. *our*
TestCase), and in turn inspect them for test methods (like in
unittest, all methods which start with "test").
For every found method, TestSuite will store its module, class and
unbound method objects in a TestItem object. There are also other
properties stored, e. g. the source code for each method and its
docstring.
When the TestSuite's method 'run' is called, all collected TestItems
are run and, according to the outcome of the test, a TestResult object
is generated which holds a reference to "its" TestItem object.
The TestResult classes Success and Skipped denote a passed test. A
skipped test means that not all test code has been run (and no error
or failure occurred before the skip method was called). If a test
fails, resulting in a Failure object, a test, e. g. tested with
assertEqual, has failed. An Error object is generated if something
else causes an unforeseen exception to be raised.
Outlook
~~~~~~~
To be directly usable for the PyPy project, the selection of specific
object spaces has to be implemented. Moreover, the existing test
modules have to be converted. Because the TestCase class interface
remains compatible, this should only involve changes which can be
performed automatically, that is, changing the import statement for
this test module and the base class of test cases.
|
GitWeb