The defining feature of this eclass is that it defines a pkg_setup
phase. It normally calls python_setup function in order to find
a suitable Python interpreter, and set the global environment
@@ -77,7 +78,7 @@ the three highlighted lines are specific to Python eclasses, plus
the implicitly exported pkg_setup phase.
When depending on other Python packages, USE dependencies need to be
declared in order to ensure that the dependencies would be built against
the Python implementation used for the package. When Python
@@ -158,7 +159,7 @@ called at most once. In particular, it is invalid to split the above
example into multiple python_gen_any_dep calls.
In some packages, Python is only necessary with specific USE flag
combinations. This is particularly common when Python is used for
the test suite. In that case, the dependencies and pkg_setup call
@@ -203,7 +204,7 @@ need to be wrapped in appropriate USE conditions:
Another possible case is that Python is required unconditionally
but some dependencies are required only conditionally to USE flags.
The simplest way to achieve that is to use ${PYTHON_DEPS} globally
@@ -243,7 +244,7 @@ a similar condition in
-
The hardest case for this eclass is to declare multiple Python
dependencies conditional to different USE flags. While there are
multiple possible ways of doing that, the least error-prone is to move
@@ -359,7 +360,7 @@ USE conditional blocks inside
-
+
@@ -376,7 +377,7 @@ USE conditional blocks inside Sphinx 4.5.0
+ Powered by Sphinx 5.0.1
& Alabaster 0.7.12
|
diff --git a/guide/basic.html b/guide/basic.html
index 53970e8..bd31abd 100644
--- a/guide/basic.html
+++ b/guide/basic.html
@@ -1,10 +1,10 @@
-
+
-
+
Common basics — Gentoo Python Guide documentation
@@ -12,6 +12,7 @@
+
@@ -34,13 +35,13 @@
The various eclasses in python-r1 try to follow a single design. You
will probably use more than one of them, so it is worthwhile to shortly
explain the common bits used by all of them, as well as the non-obvious
differences between them.
The PYTHON_COMPAT variable is used by all Python eclasses, and must
be declared in all ebuilds before they are inherited. It specifies
the list of Python implementations supported by the package.
The python-any-r1, python-single-r1 and python-r1 eclasses
all assume that the developer is responsible for explicitly putting
the dependency strings and USE requirements in correct variables.
@@ -78,7 +79,7 @@ Python unconditionally is to define the following:
The eclasses commonly use the concept of Python environment. This
means a state of environment enforcing a particular Python
implementation. Whenever the ebuild code is run inside this
@@ -112,7 +113,7 @@ via calling python_
pkg_setup in python-any-r1 and python-single-r1.
The following sections focus specifically on dependencies that
@@ -121,7 +122,7 @@ tools, libraries written in other programming languages, etc.
For these dependencies, the usual Gentoo rules apply.
Most of the modern Python build systems include all the package metadata
in the pyproject.toml file. Setuptools are using setup.cfg
and/or setup.py. Some packages also include custom code to read
@@ -321,7 +322,7 @@ or noxfile.py
-
+
@@ -338,7 +339,7 @@ or noxfile.pySphinx 4.5.0
+ Powered by Sphinx 5.0.1
& Alabaster 0.7.12
|
diff --git a/guide/buildsys.html b/guide/buildsys.html
index 711d411..ca0b5dd 100644
--- a/guide/buildsys.html
+++ b/guide/buildsys.html
@@ -1,10 +1,10 @@
-
+
-
+
Integration with build systems written in Python — Gentoo Python Guide documentation
@@ -12,6 +12,7 @@
+
@@ -34,21 +35,21 @@
Besides the build systems commonly used by Python packages there are
a few build systems written in Python and requiring the packages
to script them in Python. This makes it necessary to use a proper
Python eclass in order to declare their compatibility with Python
versions.
Meson build system is a fairly recent addition. While it is written
in Python, its scripts use a custom Python-agnostic script language.
Hence, it can be treated as an arbitrary external tool and does not need
any Python eclass.
SCons has gained Python 3 support quite recently. At the same time,
many old script files were written for Python 2 and fail when run
via Python 3 SCons. For this reason, it is necessary to use Python
@@ -62,7 +63,7 @@ The latter eclass takes care of setting as much as possible.
for them and use an appropriate eclass as detailed in the eclass choice
chapter.
-
If the package either has no other Python components than SCons, or all
of them are purely build-time and have no dependencies, it is sufficient
to inherit python-any-r1. The eclass takes care of setting
@@ -123,7 +124,7 @@ to inherit python-a
If the package needs to install some Python components, and single-impl
install is appropriate, you need to combine python-single-r1
with scons-utils. In this case, the eclass takes care of everything
@@ -288,7 +289,7 @@ package.
-
Single-impl package with conditional Python install¶
+
Single-impl package with conditional Python install¶
If the runtime part of the package uses Python only conditionally,
the use is similar to a package with unconditional build-time
and conditional runtime dependency on Python. That is, build-time
@@ -393,7 +394,7 @@ unconditionally.
When you are dealing with a pure Python package using SCons, it makes
sense to use plain python-r1 API. This means that SCons is going
to be called from a python_foreach_impl loop only.
Finally, let’s consider a package that uses SCons as a build system
and installs Python components independently of it. This could be
e.g. a C/C++ program with separate Python bindings.
@@ -540,7 +541,7 @@ case, the any-r1 API is recommended.
-
The waf build system is written in Python and bundled with the packages
using it. Therefore, it is necessary to combine waf-utils eclass
with one of the Python eclasses.
@@ -553,7 +554,7 @@ interpreter, it is necessary to add <
all waf packages (combined with individual package requirements if
applicable).
-
If waf is the only build-time Python script in the package, it is only
necessary to add PYTHON_REQ_USE and ${PYTHON_DEPS} to build-time
dependencies. If the package had other Python dependencies, you would
@@ -584,7 +585,7 @@ specify them instead.
The rules for integrating simple-impl package are roughly the same
as for pure python-single-r1 use. Again, waf requires only plain
build-time ${PYTHON_DEPS} and PYTHON_REQ_USE.
Traditionally, Python packages were organized into a hierarchical
structure with modules and subpackages being located inside the parent
package directory. When submodules are imported, they are represented
@@ -83,7 +84,7 @@ package directory:
to the build directory. However, there is a better solution.
-
Unlike traditional packages, namespace packages act as a kind of proxy.
They are not strictly bound to the containing directory, and instead
permit loading subpackages from all directories found in module search
@@ -129,7 +130,7 @@ of Python.
namespace packages in Python Packaging User Guide.
-
If the package uses PEP 420 namespaces, no special action is required.
Per PEP 420 layout, the package must not install __init__.py files
for namespaces.
@@ -198,7 +199,7 @@ sources, this dependency needs to be stripped, e.g.:
While the majority of Python standard library modules are available
unconditionally, a few are controlled by USE flags. For example,
the sqlite3 module requires sqlite flag to be enabled
@@ -127,7 +128,7 @@ SQLite support when running tests:
When packaging software for multiple Python versions, it is quite likely
that you’ll find yourself needing some packages only with some
of the versions, and not with others. This is the case with backports
@@ -201,7 +202,7 @@ Python 3.7 was removed:
The PyPy distribution includes special versions of the cffi
and greenlet packages. For this reason, packages using CFFI
and/or greenlet and supporting PyPy3 need to make the explicit
@@ -280,7 +281,7 @@ dependencies conditional to CPython:
This section describes concepts specific to the legacy mode
of the distutils-r1 eclass. When using the modern PEP 517 mode,
none of the features described here are available.
The commonly used build systems specific to Python packages can be
classified for eclass support into following groups:
@@ -56,7 +57,7 @@ are used, the package needs to have a runtime dependency
on dev-python/setuptools. Otherwise, a build-time dependency
is sufficient.
-
The most common case right now is a package using setuptools as a build
system, and therefore needing a build-time dependency only. This
is the eclass’ default. If your package does not fit this profile,
@@ -107,7 +108,7 @@ An example report is:
Distutils and setuptools are the two most common build systems
for Python packages right now. Their common feature is that they use
a setup.py script that interfaces with the build system. Generally,
@@ -148,7 +149,7 @@ be necessary if you want to force distutils (e.g. to enable clean
bootstrap) or the upstream condition requiers that.
With removal of Python 3.7, the correct DISTUTILS_USE_SETUPTOOLS
@@ -197,7 +198,7 @@ the QA check will also trigger the wrong value because of leftover
explicit dependencies on setuptools.
Besides the generated wrapper scripts, the package code itself may use
the setuptools or pkg_resources packages. The common cases
for this include getting package metadata and resource files. This
@@ -211,7 +212,7 @@ dependency. If ins
in the installed modules, please submit a patch upstream.
The newer build systems used for Python packages avoid supplying
setup.py and instead declare package’s metadata and build system
information in pyproject.toml. Examples of these build systems
@@ -231,7 +232,7 @@ points is not updated.
In the general definition, an out-of-source build is a build where
output files are placed in a directory separate from source files.
By default, distutils and its derivatives always do out-of-source builds
@@ -282,7 +283,7 @@ files are contained within the copy used for the current interpreter.
The tests are executed in src_test phase, after src_compile
installed package files into the build directory. The eclass
automatically adds appropriate PYTHONPATH so that the installed
@@ -390,7 +391,7 @@ has broken it for most of the consumers.
The distutils-r1 eclass is used to facilitate build systems using
setup.py (distutils and its derivatives, notably setuptools)
or pyproject.toml (flit, poetry). It is built on top
@@ -42,7 +43,7 @@ of python-r1
efficiently building multi-impl and single-impl packages.
By default, distutils-r1 sets appropriate metadata variables
and exports a full set of phase functions necessary to install packages
using Python build systems.
The vast majority of Python packages can be found in the Python Package
Index (PyPI). Often this includes both source (sdist) and binary
(wheel) packages. In addition to that, many packages have public VCS
@@ -141,7 +142,7 @@ that may require Internet access or have problematic dependencies
(e.g. NodeJS).
Normally, distutils-r1 uses python-r1 to build multi-impl
packages, and this is the recommended mode. However, in some cases
you will need to use python-single-r1 instead, especially if you
@@ -227,7 +228,7 @@ to be rewritten.
The majority of examples in this guide assume using setuptools build
system. However, PEP 517 mode provides support for other build systems.
In order to determine the correct build system used, read
@@ -244,11 +245,6 @@ The eclass maintains a mapping of backend paths to the respective
DISTUTILS_USE_PEP517 and automatically suggests the correct value.
The following table summarizes supported backends.
-
-
-
-
-
USE_PEP517 value
Provider package
@@ -309,9 +305,9 @@ as well). The eclass automatically uses the legacy setuptools backend
for them.
-
setuptools_scm is a package providing additional features for running
inside a VCS checkout, in particular the ability to determine version
from VCS tags. However, this works correctly only when the package
@@ -393,7 +389,7 @@ via SETUPTOOLS_SCM_
Cython is a static compiler that permits writing Python extensions
in a hybrid of C and Python. Cython files are compiled into C code
that is compatible with multiple Python interpreters. This makes it
@@ -430,7 +426,7 @@ src_configure(){
is compatible with all Python versions.
The distutils build system has a major unresolved bug regarding race
conditions. If the same source file is used to build multiple Python
extensions, the build can start multiple simultaneous compiler processes
@@ -462,7 +458,7 @@ additional .c files that
Ebuilds define phase functions in order to conveniently override parts
of the build process. distutils-r1 extends this concept
by introducing sub-phases. All src_* phases in ebuild are split
@@ -493,7 +489,7 @@ defining ${BUILD_DI
implementation. However, if in-source builds are enabled, all phases
are run in these build directories.
python_prepare_all is responsible for applying changes
to the package sources that are common to all Python implementations.
The default implementation performs the tasks of default_src_prepare
@@ -537,7 +533,7 @@ enables in-source builds.
python_configure and python_configure_all have no default
functionality. The former is convenient for running additional
configuration steps if needed by the package, the latter for defining
@@ -557,7 +553,7 @@ global environment variables.
python_compile normally builds the package. It is sometimes used
to pass additional arguments to the build step. For example, it can
be used to disable parallel extension builds in packages that are broken
@@ -578,7 +574,7 @@ the documentation (see
python_test is responsible for running tests. It has no default
implementation but you are strongly encouraged to provide one (either
directly or via distutils_enable_tests). python_test_all
@@ -591,7 +587,7 @@ to Python.
python_install installs the package’s Python part. It is usually
redefined in order to pass additional setup.py arguments
or to install additional Python modules.
@@ -620,7 +616,7 @@ such as bash completions or examples.
-
When working on packages using setuptools or modified distutils, you
sometimes need to manually invoke setup.py. The eclass provides
a esetup.py helper that wraps it with additional checks, error
@@ -673,7 +669,7 @@ beforehand (much like
-
Many packages using the setuptools build system utilize the convenient
find_packages() method to locate the Python sources. In some cases,
this method also wrongly grabs top-level test directories or other files
@@ -706,7 +702,7 @@ directories in the install phase:
Since Python performs only minimal build-time (or more precisely,
import-time) checking of correctness, it is important to run tests
of Python packages in order to catch any problems early. This is
@@ -719,7 +715,7 @@ specify a test runner that’s not strictly necessary — e.g. specify
anywhere and work just fine with built-in modules. The best way
to determine the test runner to use is to grep the test sources.
The simplest way of enabling tests is to call distutils_enable_tests
in global scope, passing the test runner name as the first argument.
This function takes care of declaring test phase, setting appropriate
@@ -758,7 +754,7 @@ setting RDEPEND
Additional test dependencies can be specified in test? conditional.
The flag normally does not need to be explicitly declared,
as distutils_enable_tests does that in the majority of cases.
In PEP 517 mode, the eclass automatically exposes a venv-style install
tree to the test phase. No explicit action in necessary.
In the legacy mode, distutils_enable_tests has an optional
@@ -807,7 +803,7 @@ to a temporary directory. More information can be found in the legacy
section.
There is a number of packages that are frequently listed as test
dependencies upstream but have little to no value for Gentoo users.
It is recommended to skip those test dependencies whenever possible.
@@ -832,7 +828,7 @@ for the former, and an environment close enough to the latter. Do not
use tox in ebuilds.
If additional pre-/post-test phase actions need to be performed,
they can be easily injected via overriding src_test() and making
it call distutils-r1_src_test:
@@ -920,7 +916,7 @@ of its logic.
so you must make sure to call it if you override src_test.
-
For the relatively frequent case of pytest-based packages needing
additional customization, a epytest helper is provided. The helper
runs pytest with a standard set of options and automatic handling
@@ -937,7 +933,7 @@ python_test(){
Test suites requiring a display to work correctly can often be appeased
usng Xvfb. If the package in question does not start Xvfb directly,
virtualx.eclass can be used to do that. Whenever possible, it is
@@ -973,7 +969,7 @@ this. In other contexts,
dev-python/sphinx is commonly used to document Python packages.
It comes with a number of plugins and themes that make it convenient
to write and combine large text documents (such as this Guide!),
@@ -986,7 +982,7 @@ for Sphinx, and packages that use plugins need to guarantee the same
implementation across all plugins. To cover all those use cases easily,
the distutils_enable_sphinx function is provided.
The most common case is a package that uses Sphinx along with autodoc.
It can be recognized by conf.py listing sphinx.ext.autodoc
in the extension list. In order to support building documentation,
@@ -1025,7 +1021,7 @@ of the supported implementations, and appropriate
-
It is not uncommon for packages to require additional third-party
extensions to Sphinx. Those include themes. In order to specify
dependencies on the additional packages, pass them as extra arguments
@@ -1058,7 +1054,7 @@ packages. However, it does not have to be the one in
-
Finally, there are packages that use Sphinx purely to build
documentation from text files, without inspecting Python code.
For those packages, the any-r1 API can be omitted entirely and plain
@@ -1092,7 +1088,7 @@ should be used, and issue a warning if it’s missing or incorrect.
The eclass has been written with the assumption that the vast majority
of its consumers will be using the Python build systems unconditionally.
For this reason, it sets the ebuild metadata variables (dependencies,
@@ -1201,7 +1197,7 @@ follows:
Some Python build systems include support for writing extensions
in the Rust programming language. Two examples of these are setuptools
using dev-python/setuptools_rust plugin and Maturin. Normally,
@@ -1346,7 +1342,7 @@ warnings. Finally, the ebuild needs to call
@@ -1363,7 +1359,7 @@ warnings. Finally, the ebuild needs to call Sphinx 4.5.0
+ Powered by Sphinx 5.0.1
& Alabaster 0.7.12
|
diff --git a/guide/eclass.html b/guide/eclass.html
index 5833a4c..ceb6066 100644
--- a/guide/eclass.html
+++ b/guide/eclass.html
@@ -1,10 +1,10 @@
-
+
-
+
Choosing between Python eclasses — Gentoo Python Guide documentation
@@ -12,6 +12,7 @@
+
@@ -34,9 +35,9 @@
The first basis for choosing Python eclass is whether Python is used
merely at build-time or at runtime as well.
A runtime use occurs if the package explicitly needs Python to be
@@ -63,7 +64,7 @@ loads Python code from the package and a compatible Python version must
be enforced.
The third basis for choosing an eclass is the build system used.
If the project uses one of Python-specific build systems, that is
distutils, setuptools, flit or poetry, the distutils-r1 eclass
@@ -114,7 +115,7 @@ packages. It provides full set of default phase functions, making
writing ebuilds much easier.
Besides the aforementioned eclasses, the suite includes a common utility
eclass python-utils-r1. This eclass is inherited by all other
eclasses, and it is considered to be a part of their API. Therefore,
@@ -208,7 +209,7 @@ and are not suitable for being used otherwise.
-
+
@@ -225,7 +226,7 @@ and are not suitable for being used otherwise.
The APIs described in this chapter are powerful but even harder to use
than those described in python-r1 chapter. You should not consider
using them unless you have a proper ninja training.
There are packages that have been ported to Python 3 only partially.
They may still have some optional dependencies that support Python 2
only, they may have some components that do not support Python 3 yet.
@@ -153,7 +154,7 @@ list.
A specific case of the restriction described above is when the build
step supports a subset of Python targets for the runtime part. This
could happen e.g. if package’s Python bindings have been ported
@@ -289,7 +290,7 @@ call).
package’s files, the any-r1 API described below is preferable to this.
-
Some packages have disjoint sets of runtime and pure build-time
dependencies. The former need to be built for all enabled
implementations, the latter only for one of them. The any-r1 API
@@ -298,7 +299,7 @@ in python-r1
to build documentation. Naturally, you’re going to build the documents
only once, not separately for every enabled target.
As the name suggests, the any-r1 API resembles the API used
by python-any-r1 eclass. The disjoint build-time dependencies
are declared using python_gen_any_dep, and need to be tested
@@ -369,7 +370,7 @@ we have used this API to add Python 3.8 support to packages before
another implementation for Sphinx.
Let’s consider the case when Python is used at build-time for something
else still. In that case, we want python_setup to work
unconditionally but enforce dependencies only with doc flag enabled.
@@ -402,7 +403,7 @@ it will use any interpreter that is supported and installed, even
if it is not enabled explicitly in PYTHON_TARGETS.
-
The alternate build dependency API also integrates with distutils-r1
eclass. If python_check_deps() is declared, the python_*_all()
sub-phase functions are called with the interpreter selected according
@@ -433,7 +434,7 @@ therefore python_ch
-
Combining any-r1 API with implementation restrictions¶
+
Combining any-r1 API with implementation restrictions¶
The functions described in this chapter are common to all three basic
eclasses. To facilitate code reuse, they are declared
in python-utils-r1.eclass. However, you should not inherit this
@@ -42,7 +43,7 @@ eclass directly and instead assume the functions are provided as part
of the API of other eclasses.
The install helpers are provided commonly for python-single-r1
and python-r1 eclasses. Their main purpose is to facilitate
installing Python scripts, modules and extensions whenever the package
@@ -108,7 +109,7 @@ extensions:
If upstream build system installs Python scripts, it should also update
their shebangs to match the interpreter used for install. Otherwise,
the scripts could end up being run via another implementation, one
@@ -157,7 +158,7 @@ to install a script with
Python modules are byte compiled in order to speed up their loading.
Byte-compilation is normally done by the build system when the modules
are installed. However, sometimes packages fail to compile them
@@ -196,7 +197,7 @@ to them can be passed to the function:
Most of the time, various build systems manage to detect and query
the Python implementation correctly for necessary build details.
Ocassionally, you need to provide those values or override bad detection
@@ -319,7 +320,7 @@ helpers instead.
Gentoo is maintaining patchsets for all CPython versions. These include
some non-upstreamable Gentoo patches and upstream backports. While it
is considered acceptable to add a new patch (e.g. a security bug fix)
@@ -134,7 +135,7 @@ create the tag and push it:
Due to high resource requirements and long build time, PyPy on Gentoo
is provided both in source and precompiled form. This creates a bit
unusual ebuild structure:
The initial porting is quite hard due to a number of circular
dependencies. To ease the process, it is recommended to temporarily
limit testing of the packages that feature many additional test
@@ -282,7 +283,7 @@ on reenabling tests in the packages where they were skipped.
Python build systems are often facing the bootstrap problem — that is,
the build system itself has some dependencies, while these dependencies
require the same build system to build. The common upstream way
@@ -378,7 +379,7 @@ dependencies will be added or vendored into flit_core.
By a version of Python we usually mean the variant of Python language
and standard library interface as used by a specific version
of CPython, the reference implementation of Python.
@@ -66,7 +67,7 @@ window of 3-4 versions of Python 3. They are provided as slots
of dev-lang/python.
-
CPython is the reference and most commonly used Python implementation.
However, there are other interpreters that aim to maintain reasonable
compatibility with it.
@@ -103,7 +104,7 @@ features and standard library modules while preserving backwards
compatibility with existing code. It is not packaged in Gentoo.
-
The support for simultaneously using multiple Python implementations
is implemented primarily through USE flags. The packages installing
or using Python files define either PYTHON_TARGETS
@@ -127,7 +128,7 @@ nor python3<
also be modified to call specific version of Python directly.
A common method of improving compatibility with older versions of Python
is to backport new standard library modules or features. Packages doing
that are generally called backports.
@@ -253,7 +254,7 @@ syntax than native asyncio code.
Migrating from old PYTHON_USEDEP syntax in python-single-r1¶
+
Migrating from old PYTHON_USEDEP syntax in python-single-r1¶
Prior to February 2020, python-single-r1 used to provide a single
PYTHON_USEDEP variable alike the two other eclasses. However,
getting it to work correctly both on single-impl and multi-impl packages
@@ -67,16 +68,12 @@ you probably got single-impl vs. multi-impl wrong.
EAPI 8 has banned everything that’s been deprecated in EAPI 7, as well
as some other obsolete stuff. The following table lists all banned
things along with their suggested replacements.
-
-
-
-
Deprecated thing
Replacement
@@ -196,7 +193,7 @@ of Python 2 support, it always evaluated to true.
All the aforementioned replacements are available in all EAPIs.
The python-r1 eclass is used to install multi-impl packages.
It is considered an expert eclass — when possible, you should prefer
using python-single-r1 instead. For packages using distutils
@@ -42,7 +43,7 @@ or a similar Python build system,
The simplest case of multi-impl package is a package without a specific
build system. The modules need to be installed manually here,
and python_foreach_impl function is used to repeat the install step
@@ -110,7 +111,7 @@ need to be executed:
When depending on other Python packages, USE dependencies need to be
declared in order to ensure that the dependencies would be built against
all the Python implementations enabled for the package. This is easily
@@ -124,7 +125,7 @@ to the dependencies:
Another typical case for this eclass is to handle a pure Python package
with a non-standard build system. In this case, it is generally
necessary to call phase functions via python_foreach_impl. Whenever
@@ -249,7 +250,7 @@ to do that:
has been removed in order to disable out-of-source builds.
-
When the package installs Python components conditionally to a USE flag,
the respective USE conditional needs to be consistently used in metadata
variables and in python_foreach_impl calls.
@@ -309,7 +310,7 @@ rules for installing Python files as they are suitable only for
single-impl installs.
-
Since Python 2.7 reached EOL, Gentoo is currently phasing out support
for Python 2. Unless your package or its reverse dependencies really
need it, you should omit it from PYTHON_COMPAT. If you’re adding
@@ -49,7 +50,7 @@ of reverse dependencies afterwards.
The absolute minimum set of targets are the current default targets
found in profiles/base/make.defaults. However, developers
are strongly encouraged to test at least the next Python 3 version
@@ -58,7 +59,7 @@ in order to ease future transition, and preferably all future versions.
coverage of PyPy3 support.
-
Adding new Python implementations to existing packages¶
+
Adding new Python implementations to existing packages¶
New Python implementations can generally be added to existing packages
without a revision bump. This is because the new dependencies are added
conditionally to new USE flags. Since the existing users can not have
@@ -70,7 +71,7 @@ will not be able to enable newly added flags and therefore the risk
of the change breaking stable systems is minimal.
-
Which packages can be (co-)maintained by the Python project?¶
+
Which packages can be (co-)maintained by the Python project?¶
A large part of the Python ecosystem is fairly consistent, making it
feasible for (co-)maintenance by the Gentoo Python team.
As a rule of thumb, Python team is ready to maintain packages specific
@@ -87,7 +88,7 @@ the package. However, if you are not a member of the project, please
do not add us without asking first.
When porting packages to a new EAPI, please take care not to port
the dependencies of Portage prematurely. This generally includes
app-portage/gemato, dev-python/setuptools and their recursive
@@ -169,7 +170,7 @@ necessary to install a new Portage version.
-
+
@@ -186,7 +187,7 @@ necessary to install a new Portage version.
This section highlights some of the known incompatible changes made
in Python that could break Python scripts and modules that used to work
in prior versions. The sections are split into retroactive changes made
@@ -44,9 +45,9 @@ to all Python releases, and information specific to every Python branch
of other problems you’ve hit while porting your packages, please let me
know and I will update it.
Historically, various urllib.parse methods have passed special
characters such as LF, CR and HT through into the split URL components.
@@ -74,10 +75,10 @@ PR#14349 for an example of impact and a fix.
Support for generator-based coroutines has been deprecated since
Python 3.8, and is finally removed in 3.11. This usually results
in the following error:
@@ -101,7 +102,7 @@ and yield
-
inspect.getargspec() and inspect.formatargspec() removal¶
+
inspect.getargspec() and inspect.formatargspec() removal¶
Upstream intends to remove distutils by Python 3.12. Python 3.10 starts
throwing deprecation warnings for various distutils modules.
The distutils.sysconfig is usually easy to port.
The following table summarizes replacements for common path getters.
-
-
-
-
distutils.sysconfig call
sysconfig replacement
@@ -284,10 +281,10 @@ no trivial replacement for the variants with
-
Python 3.9 removes the deprecated base64.encodestring()
and base64.decodestring() functions. While they were deprecated
since Python 3.1, many packages still use them today.
@@ -316,10 +313,10 @@ of base64-encoded data for any length of output.
-
python-config and pkg-config no longer list Python library by default¶
+
python-config and pkg-config no longer list Python library by default¶
Until Python 3.7, the python-X.Y pkg-config file and python-config
tool listed the Python library. Starting with 3.8, this is no longer
the case. If you are building Python extensions, this is fine (they
@@ -407,7 +404,7 @@ are provided for the purpose.
Gentoo provides one of the best frameworks for providing Python support
in packages among operating systems. This includes support for
running multiple versions of Python (while most other distributions
@@ -116,7 +117,7 @@ and improvements are welcome.
A few packages use custom pytest markers to indicate e.g. tests
requiring Internet access. These markers can be used to conveniently
disable whole test groups, e.g.:
@@ -47,7 +48,7 @@ disable whole test groups, e.g.:
There are two primary methods of skipping tests based on path (and name)
in pytest: using --ignore and --deselect.
--ignore causes pytest to entirely ignore a file or a directory
@@ -71,7 +72,7 @@ requires using the local scope.
python_test(){localEPYTEST_IGNORE=(# ignore whole file with missing dep
- --ignore tests/test_client.py
+ tests/test_client.py
)localEPYTEST_DESELECT=(# deselect a single test
@@ -89,7 +90,7 @@ requires using the local scope.
pytest-runner is a package providing pytest command to setuptools.
While it might be convenient upstream, there is no real reason to use
it in Gentoo packages. It has no real advantage over calling pytest
@@ -107,7 +108,7 @@ tests. If this is the case, the dependency must be stripped.
pytest-xdist is a plugin that makes it possible to run multiple tests
in parallel. This is especially useful for programs with large test
suites that take significant time to run single-threaded.
@@ -141,7 +142,7 @@ gain from doing so. If the package’s tests take a short time to finish,
please avoid the dependency and strip it if necessary.
-
There is a number of pytest plugins that have little value to Gentoo
users. They include plugins for test coverage
(dev-python/pytest-cov), coding style (dev-python/pytest-flake8)
@@ -166,7 +167,7 @@ to strip options enabling them from <
Besides plugins explicitly used by the package, there are a few pytest
plugins that enable themselves automatically for all test suites
when installed. In some cases, their presence causes tests of packages
@@ -182,7 +183,7 @@ it explicitly:
If a test suite invokes pytest recursively (this is particularly
the case when packaging other pytest plugins), the -p option
can be insufficient to disable problematic plugins, as it does not
@@ -207,7 +208,7 @@ the entry_points.tx
-
TypeError: _make_test_flaky() got an unexpected keyword argument ‘reruns’¶
+
TypeError: _make_test_flaky() got an unexpected keyword argument ‘reruns’¶
If you see a test error resembling the following:
TypeError: _make_test_flaky() got an unexpected keyword argument 'reruns'
An ImportPathMismatchError generally indicates that the same Python
module (or one that supposedly looks the same) has been loaded twice
using different paths, e.g.:
@@ -258,17 +259,13 @@ discovery to the actual test directories, e.g.:
-
Most of the time, a missing fixture indicates that some pytest plugin
is not installed. In rare cases, it can signify an incompatible pytest
version or package issue.
The following table maps common fixture names to their respective
plugins.
The python-single-r1 eclass is used to install single-impl packages.
It is probably the easiest eclass to use, and it is recommended over
python-r1 whenever multi-impl support would add unnecessary
@@ -42,7 +43,7 @@ complexity. However, for packages using distutils or a similar Python
build system, distutils-r1 eclass should be used instead.
The defining feature of this eclass is that it defines a pkg_setup
phase. It normally calls python_setup function in order
to determine the interpreter selected by user, and set the global
@@ -80,7 +81,7 @@ the four highlighted lines are specific to Python eclasses, plus
the implicitly exported pkg_setup phase.
When depending on other Python packages, USE dependencies need to be
declared in order to ensure that the dependencies would be built against
the Python implementation used for the package. The exact dependency
@@ -126,7 +127,7 @@ inside and outside
inside it.
The examples so far assumed that Python is used unconditionally.
If Python support is conditional to a USE flag, appropriate USE
conditionals need to be used in metadata variables, and pkg_setup
@@ -172,7 +173,7 @@ needs to be rewritten to call the default implementation conditionally:
A fairly common pattern is for Python to be required unconditionally
at build time but only conditionally at runtime. This happens e.g. when
the package is calling some helper scripts at build time, and optionally
@@ -219,7 +220,7 @@ USE-conditional:
Note that eclass-exported pkg_setup is used unconditionally here.
Finally, let’s give an example of a package where Python is needed
for two independent conditions. To make it more complex, one of them
applies to build time (tests) while the other to runtime (bindings).
@@ -273,7 +274,7 @@ the one in REQUIRED
used in dependencies.
-
Some packages do not include Python files in their build systems,
or do not install all of them. In this case, the necessary files
can be installed via one of the installation helpers.
There are a few modules used to run tests in Python packages. The most
common include the built-in unittest module, pytest and nose. There
are also some rarely used test tools and domain-specific solutions,
@@ -73,7 +74,7 @@ If you have some free time, convincing upstream to switch from nose
to pytest is a worthwhile goal.
One of the more common test-related problems is that PyPI packages
(generated via setup.pysdist) often miss some or all test files.
The latter results in no tests being run, the former in test failures
@@ -87,7 +88,7 @@ tarball:
Tests are often invoked in such a way that the Python packages
and modules from the current directory take precedence over these found
in the staging area or build directory. In fact, this is often
@@ -142,7 +143,7 @@ instead:
While generally it is preferable to fix tests, sometimes you will face
failures that cannot be easily resolved. This especially applies
to tests that are broken themselves rather than indicating real problems
@@ -245,7 +246,7 @@ simply remove that file. If all tests belonging to the package
are broken, you can use RESTRICT=test to disable testing altogether.
One of more common causes of test failures are attempts to use Internet.
With Portage blocking network access by default, packages performing
tests against remote servers often fail.
Installing extra dependencies in test environment (PEP517 mode)¶
+
Installing extra dependencies in test environment (PEP517 mode)¶
Rarely, the test suite expects some package being installed that
does not fit being packaged and installed system-wide. For example,
isort’s tests use a few example plugins that are not useful to end
@@ -453,7 +454,7 @@ into it for the purpose of testing. Note that
@@ -470,7 +471,7 @@ into it for the purpose of testing. Note that Sphinx 4.5.0
+ Powered by Sphinx 5.0.1
& Alabaster 0.7.12
|
--
cgit v1.2.3-65-gdbad