diff options
Diffstat (limited to 'guide/pytest.html')
-rw-r--r-- | guide/pytest.html | 93 |
1 files changed, 50 insertions, 43 deletions
diff --git a/guide/pytest.html b/guide/pytest.html index eaf96b1..c1446d6 100644 --- a/guide/pytest.html +++ b/guide/pytest.html @@ -104,6 +104,55 @@ tests. If this is the case, the dependency must be stripped.</p> </pre></div> </div> </section> +<section id="disabling-plugin-autoloading"> +<span id="index-1"></span><span id="index-0"></span><h2>Disabling plugin autoloading<a class="headerlink" href="#disabling-plugin-autoloading" title="Link to this heading">¶</a></h2> +<p>Normally, when running a test suite pytest loads all plugins installed +on the system. This is often convenient for upstreams, as it makes it +possible to use the features provided by the plugins (such as <code class="docutils literal notranslate"><span class="pre">async</span></code> +test function support, or fixtures) without the necessity to explicitly +enable them. However, there are also cases when additional plugins +could make the test suite fail or become very slow (especially if pytest +is called recursively).</p> +<p>The modern recommendation for these cases is to disable plugin +autoloading via setting the <code class="docutils literal notranslate"><span class="pre">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span></code> +environment variable, and then explicitly enable specific plugins +if necessary.</p> +<div class="admonition note"> +<p class="admonition-title">Note</p> +<p>Previously we used to recommend explicitly disabling problematic +plugins via <code class="docutils literal notranslate"><span class="pre">-p</span> <span class="pre">no:<plugin></span></code>. However, it is rarely obvious +which plugin is causing the problems, and it is entirely possible +that another plugin will cause issues in the future, so an opt-in +approach is usually faster and more reliable.</p> +</div> +<p>The easier approach to enabling plugins is to use the <code class="docutils literal notranslate"><span class="pre">-p</span></code> option, +listing specific plugins. The option can be passed multiple times, +and accepts a plugin name as specified in the package’s +<code class="docutils literal notranslate"><span class="pre">entry_points.txt</span></code> file:</p> +<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span> +<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span><span class="o">=</span><span class="m">1</span> +<span class="w"> </span>epytest<span class="w"> </span>-p<span class="w"> </span>asyncio<span class="w"> </span>-p<span class="w"> </span>tornado +<span class="o">}</span> +</pre></div> +</div> +<p>However, this approach does not work when the test suite calls pytest +recursively (e.g. you are testing a pytest plugin). In this case, +the <code class="docutils literal notranslate"><span class="pre">PYTEST_PLUGINS</span></code> environment variable can be used instead. It +takes a comma-separated list of plugin <em>module names</em>:</p> +<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span> +<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span><span class="o">=</span><span class="m">1</span> +<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_PLUGINS</span><span class="o">=</span>xdist.plugin,xdist.looponfail,pytest_forked + +<span class="w"> </span>epytest +<span class="o">}</span> +</pre></div> +</div> +<p>Please note that failing to enable all the required plugins may cause +some of the tests to be skipped implicitly (especially if the test suite +is using <code class="docutils literal notranslate"><span class="pre">async</span></code> functions and no async plugin is loaded). Please +look at skip messages and warnings to make sure everything works +as intended.</p> +</section> <section id="using-pytest-xdist-to-run-tests-in-parallel"> <h2>Using pytest-xdist to run tests in parallel<a class="headerlink" href="#using-pytest-xdist-to-run-tests-in-parallel" title="Link to this heading">¶</a></h2> <p><a class="reference external" href="https://pypi.org/project/pytest-xdist/">pytest-xdist</a> is a plugin that makes it possible to run multiple tests @@ -191,47 +240,6 @@ to strip options enabling them from <code class="docutils literal notranslate">< </pre></div> </div> </section> -<section id="explicitly-disabling-automatic-pytest-plugins"> -<h2>Explicitly disabling automatic pytest plugins<a class="headerlink" href="#explicitly-disabling-automatic-pytest-plugins" title="Link to this heading">¶</a></h2> -<p>Besides plugins explicitly used by the package, there are a few pytest -plugins that enable themselves automatically for all test suites -when installed. In some cases, their presence causes tests of packages -that do not expect them, to fail.</p> -<p>An example of such package used to be <code class="docutils literal notranslate"><span class="pre">dev-python/pytest-relaxed</span></code>. -To resolve problems due to the plugin, it was necessary to disable -it explicitly:</p> -<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span> -<span class="w"> </span><span class="c1"># pytest-relaxed plugin makes our tests fail</span> -<span class="w"> </span>epytest<span class="w"> </span>-p<span class="w"> </span>no:relaxed -<span class="o">}</span> -</pre></div> -</div> -</section> -<section id="expert-disabling-plugin-autoloading-entirely"> -<h2>Expert: disabling plugin autoloading entirely<a class="headerlink" href="#expert-disabling-plugin-autoloading-entirely" title="Link to this heading">¶</a></h2> -<p>If a test suite invokes pytest recursively (this is particularly -the case when packaging other pytest plugins), the <code class="docutils literal notranslate"><span class="pre">-p</span></code> option -can be insufficient to disable problematic plugins, as it does not -get passed to the nested pytest invocations. For these packages, -the next best thing is to use environment variables.</p> -<p>Unfortunately, environment variables can only be used to disable -autoloading entirely. When doing that, you need to explicitly force -loading plugins that the test suite really needs.</p> -<p>This is done using two envvars: <code class="docutils literal notranslate"><span class="pre">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span></code> -to disable autoloading plugins, and <code class="docutils literal notranslate"><span class="pre">PYTEST_PLUGINS</span></code> to specify -plugins to load. The latter takes a comma-separated list of entry point -modules. To find the correct module names, look into -the <code class="docutils literal notranslate"><span class="pre">entry_points.txt</span></code> inside the package’s <code class="docutils literal notranslate"><span class="pre">.egg-info</span></code> directory.</p> -<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>python_test<span class="o">()</span><span class="w"> </span><span class="o">{</span> -<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_DISABLE_PLUGIN_AUTOLOAD</span><span class="o">=</span><span class="m">1</span> -<span class="w"> </span><span class="nb">local</span><span class="w"> </span>-x<span class="w"> </span><span class="nv">PYTEST_PLUGINS</span><span class="o">=</span>xdist.plugin,xdist.looponfail,pytest_forked - -<span class="w"> </span>distutils_install_for_testing -<span class="w"> </span>epytest -<span class="o">}</span> -</pre></div> -</div> -</section> <section id="typeerror-make-test-flaky-got-an-unexpected-keyword-argument-reruns"> <h2>TypeError: _make_test_flaky() got an unexpected keyword argument ‘reruns’<a class="headerlink" href="#typeerror-make-test-flaky-got-an-unexpected-keyword-argument-reruns" title="Link to this heading">¶</a></h2> <p>If you see a test error resembling the following:</p> @@ -405,11 +413,10 @@ setting ignores <code class="docutils literal notranslate"><span class="pre">Dep <li class="toctree-l2"><a class="reference internal" href="#skipping-tests-based-on-markers">Skipping tests based on markers</a></li> <li class="toctree-l2"><a class="reference internal" href="#skipping-tests-based-on-paths-names">Skipping tests based on paths/names</a></li> <li class="toctree-l2"><a class="reference internal" href="#avoiding-the-dependency-on-pytest-runner">Avoiding the dependency on pytest-runner</a></li> +<li class="toctree-l2"><a class="reference internal" href="#disabling-plugin-autoloading">Disabling plugin autoloading</a></li> <li class="toctree-l2"><a class="reference internal" href="#using-pytest-xdist-to-run-tests-in-parallel">Using pytest-xdist to run tests in parallel</a></li> <li class="toctree-l2"><a class="reference internal" href="#dealing-with-flaky-tests">Dealing with flaky tests</a></li> <li class="toctree-l2"><a class="reference internal" href="#avoiding-dependencies-on-other-pytest-plugins">Avoiding dependencies on other pytest plugins</a></li> -<li class="toctree-l2"><a class="reference internal" href="#explicitly-disabling-automatic-pytest-plugins">Explicitly disabling automatic pytest plugins</a></li> -<li class="toctree-l2"><a class="reference internal" href="#expert-disabling-plugin-autoloading-entirely">Expert: disabling plugin autoloading entirely</a></li> <li class="toctree-l2"><a class="reference internal" href="#typeerror-make-test-flaky-got-an-unexpected-keyword-argument-reruns">TypeError: _make_test_flaky() got an unexpected keyword argument ‘reruns’</a></li> <li class="toctree-l2"><a class="reference internal" href="#importpathmismatcherror">ImportPathMismatchError</a></li> <li class="toctree-l2"><a class="reference internal" href="#fixture-not-found">fixture ‘…’ not found</a></li> |