Wala shell

Advanced PHP File Manager

Server IP: 127.0.0.1 | PHP Version: 8.3.30 | Domains Hosted: 54

Name	Size	Last Modified	Actions
📁 ..	-	2026-03-27 02:58
📄 datafiles.rst ==================== Data Files Support ==================== Old packaging installation methods in the Python ecosystem have traditionally allowed installation of "data files", which are placed in a platform-specific location. However, the most common use case for data files distributed with a package is for use by the package, usually by including the data files inside the package directory. Setuptools focuses on this most common type of data files and offers three ways of specifying which files should be included in your packages, as described in the following sections. include_package_data ==================== First, you can simply use the ``include_package_data`` keyword. For example, if the package tree looks like this:: project_root_directory ├── setup.py # and/or setup.cfg, pyproject.toml └── src └── mypkg ├── __init__.py ├── data1.rst ├── data2.rst ├── data1.txt └── data2.txt and you supply this configuration: .. tab:: setup.cfg .. code-block:: ini [options] # ... packages = find: package_dir = = src include_package_data = True [options.packages.find] where = src .. tab:: setup.py .. code-block:: python from setuptools import setup, find_packages setup( # ..., packages=find_packages(where="src"), package_dir={"": "src"}, include_package_data=True ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools] # ... # By default, include-package-data is true in pyproject.toml, so you do # NOT have to specify this line. include-package-data = true [tool.setuptools.packages.find] where = ["src"] then all the ``.txt`` and ``.rst`` files will be automatically installed with your package, provided: 1. These files are included via the \|MANIFEST.in\|_ file, like so:: include src/mypkg/.txt include src/mypkg/.rst 2. OR, they are being tracked by a revision control system such as Git, Mercurial or SVN, and you have configured an appropriate plugin such as :pypi:`setuptools-scm` or :pypi:`setuptools-svn`. (See the section below on :ref:`Adding Support for Revision Control Systems` for information on how to write such plugins.) package_data ============ By default, ``include_package_data`` considers all non ``.py`` files found inside the package directory (``src/mypkg`` in this case) as data files, and includes those that satisfy (at least) one of the above two conditions into the source distribution, and consequently in the installation of your package. If you want finer-grained control over what files are included, then you can also use the ``package_data`` keyword. For example, if the package tree looks like this:: project_root_directory ├── setup.py # and/or setup.cfg, pyproject.toml └── src └── mypkg ├── __init__.py ├── data1.rst ├── data2.rst ├── data1.txt └── data2.txt then you can use the following configuration to capture the ``.txt`` and ``.rst`` files as data files: .. tab:: setup.cfg .. code-block:: ini [options] # ... packages = find: package_dir = = src [options.packages.find] where = src [options.package_data] mypkg = .txt .rst .. tab:: setup.py .. code-block:: python from setuptools import setup, find_packages setup( # ..., packages=find_packages(where="src"), package_dir={"": "src"}, package_data={"mypkg": [".txt", ".rst"]} ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools.packages.find] where = ["src"] [tool.setuptools.package-data] mypkg = [".txt", ".rst"] The ``package_data`` argument is a dictionary that maps from package names to lists of glob patterns. Note that the data files specified using the ``package_data`` option neither require to be included within a \|MANIFEST.in\|_ file, nor require to be added by a revision control system plugin. .. note:: If your glob patterns use paths, you must use a forward slash (``/``) as the path separator, even if you are on Windows. Setuptools automatically converts slashes to appropriate platform-specific separators at build time. .. note:: Glob patterns do not automatically match dotfiles (directory or file names starting with a dot (``.``)). To include such files, you must explicitly start the pattern with a dot, e.g. ``.`` to match ``.gitignore``. If you have multiple top-level packages and a common pattern of data files for all these packages, for example:: project_root_directory ├── setup.py # and/or setup.cfg, pyproject.toml └── src ├── mypkg1 │ ├── data1.rst │ ├── data1.txt │ └── __init__.py └── mypkg2 ├── data2.txt └── __init__.py Here, both packages ``mypkg1`` and ``mypkg2`` share a common pattern of having ``.txt`` data files. However, only ``mypkg1`` has ``.rst`` data files. In such a case, if you want to use the ``package_data`` option, the following configuration will work: .. tab:: setup.cfg .. code-block:: ini [options] packages = find: package_dir = = src [options.packages.find] where = src [options.package_data] = .txt mypkg1 = data1.rst .. tab:: setup.py .. code-block:: python from setuptools import setup, find_packages setup( # ..., packages=find_packages(where="src"), package_dir={"": "src"}, package_data={"": [".txt"], "mypkg1": ["data1.rst"]}, ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools.packages.find] where = ["src"] [tool.setuptools.package-data] "" = [".txt"] mypkg1 = ["data1.rst"] Notice that if you list patterns in ``package_data`` under the empty string ``""`` in ``setup.py``, and the asterisk ```` in ``setup.cfg`` and ``pyproject.toml``, these patterns are used to find files in every package. For example, we use ``""`` or ```` to indicate that the ``.txt`` files from all packages should be captured as data files. Also note how we can continue to specify patterns for individual packages, i.e. we specify that ``data1.rst`` from ``mypkg1`` alone should be captured as well. .. note:: When building an ``sdist``, the datafiles are also drawn from the ``package_name.egg-info/SOURCES.txt`` file, so make sure that this is removed if the ``setup.py`` ``package_data`` list is updated before calling ``setup.py``. .. note:: If using the ``include_package_data`` argument, files specified by ``package_data`` will not be automatically added to the manifest unless they are listed in the \|MANIFEST.in\|_ file or by a plugin like :pypi:`setuptools-scm` or :pypi:`setuptools-svn`. .. https://docs.python.org/3/distutils/setupscript.html#installing-package-data exclude_package_data ==================== Sometimes, the ``include_package_data`` or ``package_data`` options alone aren't sufficient to precisely define what files you want included. For example, consider a scenario where you have ``include_package_data=True``, and you are using a revision control system with an appropriate plugin. Sometimes developers add directory-specific marker files (such as ``.gitignore``, ``.gitkeep``, ``.gitattributes``, or ``.hgignore``), these files are probably being tracked by the revision control system, and therefore by default they will be included when the package is installed. Supposing you want to prevent these files from being included in the installation (they are not relevant to Python or the package), then you could use the ``exclude_package_data`` option: .. tab:: setup.cfg .. code-block:: ini [options] # ... packages = find: package_dir = = src include_package_data = True [options.packages.find] where = src [options.exclude_package_data] mypkg = .gitattributes .. tab:: setup.py .. code-block:: python from setuptools import setup, find_packages setup( # ..., packages=find_packages(where="src"), package_dir={"": "src"}, include_package_data=True, exclude_package_data={"mypkg": [".gitattributes"]}, ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools.packages.find] where = ["src"] [tool.setuptools.exclude-package-data] mypkg = [".gitattributes"] The ``exclude_package_data`` option is a dictionary mapping package names to lists of wildcard patterns, just like the ``package_data`` option. And, just as with that option, you can use the empty string key ``""`` in ``setup.py`` and the asterisk ```` in ``setup.cfg`` and ``pyproject.toml`` to match all top-level packages. Any files that match these patterns will be excluded* from installation, even if they were listed in ``package_data`` or were included as a result of using ``include_package_data``. Subdirectory for Data Files =========================== A common pattern is where some (or all) of the data files are placed under a separate subdirectory. For example:: project_root_directory ├── setup.py # and/or setup.cfg, pyproject.toml └── src └── mypkg ├── data │ ├── data1.rst │ └── data2.rst ├── __init__.py ├── data1.txt └── data2.txt Here, the ``.rst`` files are placed under a ``data`` subdirectory inside ``mypkg``, while the ``.txt`` files are directly under ``mypkg``. In this case, the recommended approach is to treat ``data`` as a namespace package (refer :pep:`420`). With ``package_data``, the configuration might look like this: .. tab:: setup.cfg .. code-block:: ini [options] # ... packages = find_namespace: package_dir = = src [options.packages.find] where = src [options.package_data] mypkg = .txt mypkg.data = .rst .. tab:: setup.py .. code-block:: python from setuptools import setup, find_namespace_packages setup( # ..., packages=find_namespace_packages(where="src"), package_dir={"": "src"}, package_data={ "mypkg": [".txt"], "mypkg.data": [".rst"], } ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools.packages.find] # scanning for namespace packages is true by default in pyproject.toml, so # you do NOT need to include the following line. namespaces = true where = ["src"] [tool.setuptools.package-data] mypkg = [".txt"] "mypkg.data" = [".rst"] In other words, we allow Setuptools to scan for namespace packages in the ``src`` directory, which enables the ``data`` directory to be identified, and then, we separately specify data files for the root package ``mypkg``, and the namespace package ``data`` under the package ``mypkg``. With ``include_package_data`` the configuration is simpler: you simply need to enable scanning of namespace packages in the ``src`` directory and the rest is handled by Setuptools. .. tab:: setup.cfg .. code-block:: ini [options] packages = find_namespace: package_dir = = src include_package_data = True [options.packages.find] where = src .. tab:: setup.py .. code-block:: python from setuptools import setup, find_namespace_packages setup( # ... , packages=find_namespace_packages(where="src"), package_dir={"": "src"}, include_package_data=True, ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools] # ... # By default, include-package-data is true in pyproject.toml, so you do # NOT have to specify this line. include-package-data = true [tool.setuptools.packages.find] # scanning for namespace packages is true by default in pyproject.toml, so # you need NOT include the following line. namespaces = true where = ["src"] Summary ======= In summary, the three options allow you to: ``include_package_data`` Accept all data files and directories matched by \|MANIFEST.in\|_ or added by a :ref:`plugin <Adding Support for Revision Control Systems>`. ``package_data`` Specify additional patterns to match files that may or may not be matched by \|MANIFEST.in\|_ or added by a :ref:`plugin <Adding Support for Revision Control Systems>`. ``exclude_package_data`` Specify patterns for data files and directories that should not be included when a package is installed, even if they would otherwise have been included due to the use of the preceding options. .. note:: Due to the way the build process works, a data file that you include in your project and then stop including may be "orphaned" in your project's build directories, requiring you to run ``setup.py clean --all`` to fully remove them. This may also be important for your users and contributors if they track intermediate revisions of your project using Subversion; be sure to let them know when you make changes that remove files from inclusion so they can run ``setup.py clean --all``. .. _Accessing Data Files at Runtime: Accessing Data Files at Runtime =============================== Typically, existing programs manipulate a package's ``__file__`` attribute in order to find the location of data files. For example, if you have a structure like this:: project_root_directory ├── setup.py # and/or setup.cfg, pyproject.toml └── src └── mypkg ├── data │ └── data1.txt ├── __init__.py └── foo.py Then, in ``mypkg/foo.py``, you may try something like this in order to access ``mypkg/data/data1.txt``: .. code-block:: python import os data_path = os.path.join(os.path.dirname(__file__), 'data', 'data1.txt') with open(data_path, 'r') as data_file: ... However, this manipulation isn't compatible with :pep:`302`-based import hooks, including importing from zip files and Python Eggs. It is strongly recommended that, if you are using data files, you should use :mod:`importlib.resources` to access them. In this case, you would do something like this: .. code-block:: python from importlib.resources import files data_text = files('mypkg.data').joinpath('data1.txt').read_text() :mod:`importlib.resources` was added to Python 3.7. However, the API illustrated in this code (using ``files()``) was added only in Python 3.9, [#files_api]_ and support for accessing data files via namespace packages was added only in Python 3.10 [#namespace_support]_ (the ``data`` subdirectory is a namespace package under the root package ``mypkg``). Therefore, you may find this code to work only in Python 3.10 (and above). For other versions of Python, you are recommended to use the :pypi:`importlib-resources` backport which provides the latest version of this library. In this case, the only change that has to be made to the above code is to replace ``importlib.resources`` with ``importlib_resources``, i.e. .. code-block:: python from importlib_resources import files ... See :doc:`importlib-resources:using` for detailed instructions. .. tip:: Files inside the package directory should be read-only to avoid a series of common problems (e.g. when multiple users share a common Python installation, when the package is loaded from a zip file, or when multiple instances of a Python application run in parallel). If your Python package needs to write to a file for shared data or configuration, you can use standard platform/OS-specific system directories, such as ``~/.local/config/$appname`` or ``/usr/share/$appname/$version`` (Linux specific) [#system-dirs]_. A common approach is to add a read-only template file to the package directory that is then copied to the correct system directory if no pre-existing file is found. Non-Package Data Files ====================== Historically, ``setuptools`` by way of ``easy_install`` would encapsulate data files from the distribution into the egg (see `the old docs <https://github.com/pypa/setuptools/blob/52aacd5b276fedd6849c3a648a0014f5da563e93/docs/setuptools.txt#L970-L1001>`_). As eggs are deprecated and pip-based installs fall back to the platform-specific location for installing data files, there is no supported facility to reliably retrieve these resources. Instead, the PyPA recommends that any data files you wish to be accessible at run time be included inside the package. ---- .. [#beta] Support for adding build configuration options via the ``[tool.setuptools]`` table in the ``pyproject.toml`` file. See :doc:`/userguide/pyproject_config`. .. [#system-dirs] These locations can be discovered with the help of third-party libraries such as :pypi:`platformdirs`. .. [#files_api] Reference: https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy .. [#namespace_support] Reference: https://github.com/python/importlib_resources/pull/196#issuecomment-734520374 .. \|MANIFEST.in\| replace:: ``MANIFEST.in`` .. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/	17.99 KB	2022-11-24 06:08
📄 declarative_config.rst .. _declarative config: ------------------------------------------------ Configuring setuptools using ``setup.cfg`` files ------------------------------------------------ .. note:: New in 30.3.0 (8 Dec 2016). .. important:: If compatibility with legacy builds (i.e. those not using the :pep:`517` build API) is desired, a ``setup.py`` file containing a ``setup()`` function call is still required even if your configuration resides in ``setup.cfg``. ``Setuptools`` allows using configuration files (usually :file:`setup.cfg`) to define a package’s metadata and other options that are normally supplied to the ``setup()`` function (declarative config). This approach not only allows automation scenarios but also reduces boilerplate code in some cases. .. _example-setup-config: .. code-block:: ini [metadata] name = my_package version = attr: my_package.VERSION description = My package description long_description = file: README.rst, CHANGELOG.rst, LICENSE.rst keywords = one, two license = BSD 3-Clause License classifiers = Framework :: Django Programming Language :: Python :: 3 [options] zip_safe = False include_package_data = True packages = find: install_requires = requests importlib-metadata; python_version<"3.8" [options.package_data] * = .txt, .rst hello = .msg [options.entry_points] console_scripts = executable-name = my_package.module:function [options.extras_require] pdf = ReportLab>=1.2; RXP rest = docutils>=0.3; pack ==1.1, ==1.3 [options.packages.find] exclude = examples tools* docs* my_package.tests* Metadata and options are set in the config sections of the same name. * Keys are the same as the :doc:`keyword arguments </references/keywords>` one provides to the ``setup()`` function. * Complex values can be written comma-separated or placed one per line in dangling config values. The following are equivalent: .. code-block:: ini [metadata] keywords = one, two [metadata] keywords = one two * In some cases, complex values can be provided in dedicated subsections for clarity. * Some keys allow ``file:``, ``attr:``, ``find:``, and ``find_namespace:`` directives in order to cover common usecases. * Unknown keys are ignored. Using a ``src/`` layout ======================= One commonly used configuration has all the Python source code in a subdirectory (often called the ``src/`` layout), like this:: ├── src │ └── mypackage │ ├── __init__.py │ └── mod1.py ├── setup.py └── setup.cfg You can set up your ``setup.cfg`` to automatically find all your packages in the subdirectory, using :ref:`package_dir <keyword/package_dir>`, like this: .. code-block:: ini # This example contains just the necessary options for a src-layout, set up # the rest of the file as described above. [options] package_dir= =src packages=find: [options.packages.find] where=src In this example, the value for the :ref:`package_dir <keyword/package_dir>` configuration (i.e. ``=src``) is parsed as ``{"": "src"}``. The ``""`` key has a special meaning in this context, and indicates that all the packages are contained inside the given directory. Also note that the value for ``[options.packages.find] where`` matches the value associated with ``""`` in the ``package_dir`` dictionary. .. TODO: Add the following tip once the auto-discovery is no longer experimental: Starting in version 61, ``setuptools`` can automatically infer the configurations for both ``packages`` and ``package_dir`` for projects using a ``src/`` layout (as long as no value is specified for ``py_modules``). Please see :doc:`package discovery </userguide/package_discovery>` for more details. Specifying values ================= Some values are treated as simple strings, some allow more logic. Type names used below: * ``str`` - simple string * ``list-comma`` - dangling list or string of comma-separated values * ``list-semi`` - dangling list or string of semicolon-separated values * ``bool`` - ``True`` is 1, yes, true * ``dict`` - list-comma where each entry corresponds to a key/value pair, with keys separated from values by ``=``. If an entry starts with ``=``, the key is assumed to be an empty string (e.g. ``=src`` is parsed as ``{"": "src"}``). * ``section`` - values are read from a dedicated (sub)section Special directives: * ``attr:`` - Value is read from a module attribute. ``attr:`` supports callables and iterables; unsupported types are cast using ``str()``. In order to support the common case of a literal value assigned to a variable in a module containing (directly or indirectly) third-party imports, ``attr:`` first tries to read the value from the module by examining the module's AST. If that fails, ``attr:`` falls back to importing the module. * ``file:`` - Value is read from a list of files and then concatenated .. important:: The ``file:`` directive is sandboxed and won't reach anything outside the project directory (i.e. the directory containing ``setup.cfg``/``pyproject.toml``). .. attention:: When using the ``file:`` directive, please make sure that all necessary files are included in the ``sdist``. You can do that via ``MANIFEST.in`` or using plugins such as ``setuptools-scm``. Please have a look on :doc:`/userguide/miscellaneous` for more information. Metadata -------- .. attention:: The aliases given below are supported for compatibility reasons, but their use is not advised. ============================== ================= ================= =============== ========== Key Aliases Type Minimum Version Notes ============================== ================= ================= =============== ========== name str version attr:, file:, str 39.2.0 [#meta-1]_ url home-page str download_url download-url str project_urls dict 38.3.0 author str author_email author-email str maintainer str maintainer_email maintainer-email str classifiers classifier file:, list-comma license str license_files license_file list-comma 42.0.0 description summary file:, str long_description long-description file:, str long_description_content_type str 38.6.0 keywords list-comma platforms platform list-comma provides list-comma requires list-comma obsoletes list-comma ============================== ================= ================= =============== ========== Notes: .. [#meta-1] The ``version`` file attribute has only been supported since 39.2.0. A version loaded using the ``file:`` directive must comply with PEP 440. It is easy to accidentally put something other than a valid version string in such a file, so validation is stricter in this case. Options ------- ======================= =================================== =============== ==================== Key Type Minimum Version Notes ======================= =================================== =============== ==================== zip_safe bool setup_requires list-semi 36.7.0 install_requires file:, list-semi BETA [#opt-6]_ extras_require file:, section BETA [#opt-2]_, [#opt-6]_ python_requires str 34.4.0 entry_points file:, section 51.0.0 scripts list-comma eager_resources list-comma dependency_links list-comma tests_require list-semi include_package_data bool packages find:, find_namespace:, list-comma [#opt-3]_ package_dir dict package_data section [#opt-1]_ exclude_package_data section namespace_packages list-comma [#opt-5]_ py_modules list-comma 34.4.0 data_files section 40.6.0 [#opt-4]_ ======================= =================================== =============== ==================== Notes: .. [#opt-1] In the ``package_data`` section, a key named with a single asterisk (````) refers to all packages, in lieu of the empty string used in ``setup.py``. .. [#opt-2] In the ``extras_require`` section, values are parsed as ``list-semi``. This implies that in order to include markers, they must* be dangling: .. code-block:: ini [options.extras_require] rest = docutils>=0.3; pack ==1.1, ==1.3 pdf = ReportLab>=1.2 RXP importlib-metadata; python_version < "3.8" .. [#opt-3] The ``find:`` and ``find_namespace:`` directive can be further configured in a dedicated subsection ``options.packages.find``. This subsection accepts the same keys as the ``setuptools.find_packages`` and the ``setuptools.find_namespace_packages`` function: ``where``, ``include``, and ``exclude``. The ``find_namespace:`` directive is supported since Python >=3.3. .. [#opt-4] ``data_files`` is deprecated and should be avoided. Please check :doc:`/userguide/datafiles` for more information. .. [#opt-5] ``namespace_packages`` is deprecated in favour of native/implicit namespaces (:pep:`420`). Check :doc:`the Python Packaging User Guide <PyPUG:guides/packaging-namespace-packages>` for more information. .. [#opt-6] ``file:`` directives for reading requirements are supported since version 62.6. The format for the file resembles a ``requirements.txt`` file, however please keep in mind that all non-comment lines must conform with :pep:`508` (``pip``-specify syntaxes, e.g. ``-c/-r/-e`` flags, are not supported). Library developers should avoid tightly pinning their dependencies to a specific version (e.g. via a "locked" requirements file). Compatibility with other tools ============================== Historically, several tools explored declarative package configuration in parallel. And several of them chose to place the packaging configuration within the project's :file:`setup.cfg` file. One of the first was ``distutils2``, which development has stopped in 2013. Other include ``pbr`` which is still under active development or ``d2to1``, which was a plug-in that backports declarative configuration to ``distutils``, but has had no release since Oct. 2015. As a way to harmonize packaging tools, ``setuptools``, having held the position of de facto standard, has gradually integrated those features as part of its core features. Still this has lead to some confusion and feature incompatibilities: - some tools support features others don't; - some have similar features but the declarative syntax differs; The table below tries to summarize the differences. But, please, refer to each tool documentation for up-to-date information. =========================== ========== ========== ===== === feature setuptools distutils2 d2to1 pbr =========================== ========== ========== ===== === [metadata] description-file S Y Y Y [files] S Y Y Y entry_points Y Y Y S [backwards_compat] N Y Y Y =========================== ========== ========== ===== === Y: supported, N: unsupported, S: syntax differs (see :ref:`above example<example-setup-config>`). Also note that some features were only recently added to ``setuptools``. Please refer to the previous sections to find out when.	12.43 KB	2022-11-24 06:08
📄 dependency_management.rst ===================================== Dependencies Management in Setuptools ===================================== There are three types of dependency styles offered by setuptools: 1) build system requirement, 2) required dependency and 3) optional dependency. Each dependency, regardless of type, needs to be specified according to :pep:`508` and :pep:`440`. This allows adding version :pep:`range restrictions <440#version-specifiers>` and :ref:`environment markers <environment-markers>`. .. _build-requires: Build system requirement ======================== After organizing all the scripts and files and getting ready for packaging, there needs to be a way to specify what programs and libraries are actually needed do the packaging (in our case, ``setuptools`` of course). This needs to be specified in your ``pyproject.toml`` file (if you have forgot what this is, go to :doc:`/userguide/quickstart` or :doc:`/build_meta`): .. code-block:: toml [build-system] requires = ["setuptools"] #... Please note that you should also include here any other ``setuptools`` plugin (e.g., :pypi:`setuptools-scm`, :pypi:`setuptools-golang`, :pypi:`setuptools-rust`) or build-time dependency (e.g., :pypi:`Cython`, :pypi:`cppy`, :pypi:`pybind11`). .. note:: In previous versions of ``setuptools``, this used to be accomplished with the ``setup_requires`` keyword but is now considered deprecated in favor of the :pep:`517` style described above. To peek into how this legacy keyword is used, consult our :doc:`guide on deprecated practice (WIP) </deprecated/index>`. .. _Declaring Dependencies: Declaring required dependency ============================= This is where a package declares its core dependencies, without which it won't be able to run. ``setuptools`` supports automatically downloading and installing these dependencies when the package is installed. Although there is more finesse to it, let's start with a simple example. .. tab:: pyproject.toml .. code-block:: toml [project] # ... dependencies = [ "docutils", "BazSpam == 1.1", ] # ... .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = docutils BazSpam ==1.1 .. tab:: setup.py .. code-block:: python setup( ..., install_requires=[ 'docutils', 'BazSpam ==1.1', ], ) When your project is installed (e.g., using :pypi:`pip`), all of the dependencies not already installed will be located (via `PyPI`_), downloaded, built (if necessary), and installed and 2) Any scripts in your project will be installed with wrappers that verify the availability of the specified dependencies at runtime. .. _environment-markers: Platform specific dependencies ------------------------------ Setuptools offers the capability to evaluate certain conditions before blindly installing everything listed in ``install_requires``. This is great for platform specific dependencies. For example, the ``enum`` package was added in Python 3.4, therefore, package that depends on it can elect to install it only when the Python version is older than 3.4. To accomplish this .. tab:: pyproject.toml .. code-block:: toml [project] # ... dependencies = [ "enum34; python_version<'3.4'", ] # ... .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = enum34;python_version<'3.4' .. tab:: setup.py .. code-block:: python setup( ..., install_requires=[ "enum34;python_version<'3.4'", ], ) Similarly, if you also wish to declare ``pywin32`` with a minimal version of 1.0 and only install it if the user is using a Windows operating system: .. tab:: pyproject.toml .. code-block:: toml [project] # ... dependencies = [ "enum34; python_version<'3.4'", "pywin32 >= 1.0; platform_system=='Windows'", ] # ... .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = enum34;python_version<'3.4' pywin32 >= 1.0;platform_system=='Windows' .. tab:: setup.py .. code-block:: python setup( ..., install_requires=[ "enum34;python_version<'3.4'", "pywin32 >= 1.0;platform_system=='Windows'", ], ) The environmental markers that may be used for testing platform types are detailed in :pep:`508`. .. seealso:: If environment markers are not enough an specific use case, you can also consider creating a :ref:`backend wrapper <backend-wrapper>` to implement custom detection logic. Direct URL dependencies ----------------------- .. attention:: `PyPI`_ and other standards-conformant package indices do not accept packages that declare dependencies using direct URLs. ``pip`` will accept them when installing packages from the local filesystem or from another URL, however. Dependencies that are not available on a package index but can be downloaded elsewhere in the form of a source repository or archive may be specified using a variant of :pep:`PEP 440's direct references <440#direct-references>`: .. tab:: pyproject.toml .. code-block:: toml [project] # ... dependencies = [ "Package-A @ git+https://example.net/package-a.git@main", "Package-B @ https://example.net/archives/package-b.whl", ] .. tab:: setup.cfg .. code-block:: ini [options] #... install_requires = Package-A @ git+https://example.net/package-a.git@main Package-B @ https://example.net/archives/package-b.whl .. tab:: setup.py .. code-block:: python setup( install_requires=[ "Package-A @ git+https://example.net/package-a.git@main", "Package-B @ https://example.net/archives/package-b.whl", ], ..., ) For source repository URLs, a list of supported protocols and VCS-specific features such as selecting certain branches or tags can be found in pip's documentation on `VCS support <https://pip.pypa.io/en/latest/topics/vcs-support/>`_. Supported formats for archive URLs are sdists and wheels. Optional dependencies ===================== Setuptools allows you to declare dependencies that are not installed by default. This effectively means that you can create a "variant" of your package with a set of extra functionalities. For example, let's consider a ``Package-A`` that offers optional PDF support and requires two other dependencies for it to work: .. tab:: pyproject.toml .. code-block:: toml [project] name = "Package-A" # ... [project.optional-dependencies] PDF = ["ReportLab>=1.2", "RXP"] .. tab:: setup.cfg .. code-block:: ini [metadata] name = Package-A [options.extras_require] PDF = ReportLab>=1.2 RXP .. tab:: setup.py .. code-block:: python setup( name="Package-A", ..., extras_require={ "PDF": ["ReportLab>=1.2", "RXP"], }, ) .. sidebar:: .. tip:: It is also convenient to declare optional requirements for ancillary tasks such as running tests and or building docs. The name ``PDF`` is an arbitrary :pep:`identifier <685>` of such a list of dependencies, to which other components can refer and have them installed. A use case for this approach is that other package can use this "extra" for their own dependencies. For example, if ``Package-B`` needs ``Package-A`` with PDF support installed, it might declare the dependency like this: .. tab:: pyproject.toml .. code-block:: toml [project] name = "Package-B" # ... dependencies = [ "Package-A[PDF]" ] .. tab:: setup.cfg .. code-block:: ini [metadata] name = Package-B #... [options] #... install_requires = Package-A[PDF] .. tab:: setup.py .. code-block:: python setup( name="Package-B", install_requires=["Package-A[PDF]"], ..., ) This will cause ``ReportLab`` to be installed along with ``Package-A``, if ``Package-B`` is installed -- even if ``Package-A`` was already installed. In this way, a project can encapsulate groups of optional "downstream dependencies" under a feature name, so that packages that depend on it don't have to know what the downstream dependencies are. If a later version of ``Package-A`` builds in PDF support and no longer needs ``ReportLab``, or if it ends up needing other dependencies besides ``ReportLab`` in order to provide PDF support, ``Package-B``'s setup information does not need to change, but the right packages will still be installed if needed. .. tip:: Best practice: if a project ends up no longer needing any other packages to support a feature, it should keep an empty requirements list for that feature in its ``extras_require`` argument, so that packages depending on that feature don't break (due to an invalid feature name). .. warning:: Historically ``setuptools`` also used to support extra dependencies in console scripts, for example: .. tab:: setup.cfg .. code-block:: ini [metadata] name = Package-A #... [options] #... entry_points= [console_scripts] rst2pdf = project_a.tools.pdfgen [PDF] rst2html = project_a.tools.htmlgen .. tab:: setup.py .. code-block:: python setup( name="Package-A", ..., entry_points={ "console_scripts": [ "rst2pdf = project_a.tools.pdfgen [PDF]", "rst2html = project_a.tools.htmlgen", ], }, ) This syntax indicates that the entry point (in this case a console script) is only valid when the PDF extra is installed. It is up to the installer to determine how to handle the situation where PDF was not indicated (e.g., omit the console script, provide a warning when attempting to load the entry point, assume the extras are present and let the implementation fail later). However, ``pip`` and other tools might not support this use case for extra dependencies, therefore this practice is considered deprecated. See :doc:`PyPUG:specifications/entry-points`. Python requirement ================== In some cases, you might need to specify the minimum required python version. This can be configured as shown in the example below. .. tab:: pyproject.toml .. code-block:: toml [project] name = "Package-B" requires-python = ">=3.6" # ... .. tab:: setup.cfg .. code-block:: ini [metadata] name = Package-B #... [options] #... python_requires = >=3.6 .. tab:: setup.py .. code-block:: python setup( name="Package-B", python_requires=">=3.6", ..., ) .. _PyPI: https://pypi.org	11.36 KB	2022-11-24 06:08
📄 development_mode.rst Development Mode (a.k.a. "Editable Installs") ============================================= When creating a Python project, developers usually want to implement and test changes iteratively, before cutting a release and preparing a distribution archive. In normal circumstances this can be quite cumbersome and require the developers to manipulate the ``PYTHONPATH`` environment variable or to continuously re-build and re-install the project. To facilitate iterative exploration and experimentation, setuptools allows users to instruct the Python interpreter and its import machinery to load the code under development directly from the project folder without having to copy the files to a different location in the disk. This means that changes in the Python source code can immediately take place without requiring a new installation. You can enter this "development mode" by performing an :doc:`editable installation <pip:topics/local-project-installs>` inside of a :term:`virtual environment`, using :doc:`pip's <pip:cli/pip_install>` ``-e/--editable`` flag, as shown below: .. code-block:: bash $ cd your-python-project $ python -m venv .venv # Activate your environemt with: # `source .venv/bin/activate` on Unix/macOS # or `.venv\Scripts\activate` on Windows $ pip install --editable . # Now you have access to your package # as if it was installed in .venv $ python -c "import your_python_project" An "editable installation" works very similarly to a regular install with ``pip install .``, except that it only installs your package dependencies, metadata and wrappers for :ref:`console and GUI scripts <console-scripts>`. Under the hood, setuptools will try to create a special :mod:`.pth file <site>` in the target directory (usually ``site-packages``) that extends the ``PYTHONPATH`` or install a custom :doc:`import hook <python:reference/import>`. When you're done with a given development task, you can simply uninstall your package (as you would normally do with ``pip uninstall <package name>``). Please note that, by default an editable install will expose at least all the files that would be available in a regular installation. However, depending on the file and directory organization in your project, it might also expose as a side effect files that would not be normally available. This is allowed so you can iteratively create new Python modules. Please have a look on the following section if you are looking for a different behaviour. .. admonition:: Virtual Environments You can think about virtual environments as "isolated Python runtime deployments" that allow users to install different sets of libraries and tools without messing with the global behaviour of the system. They are a safe way of testing new projects and can be created easily with the :mod:`venv` module from the standard library. Please note however that depending on your operating system or distribution, ``venv`` might not come installed by default with Python. For those cases, you might need to use the OS package manager to install it. For example, in Debian/Ubuntu-based systems you can obtain it via: .. code-block:: bash sudo apt install python3-venv Alternatively, you can also try installing :pypi:`virtualenv`. More information is available on the Python Packaging User Guide on :doc:`PyPUG:guides/installing-using-pip-and-virtual-environments`. .. note:: .. versionchanged:: v64.0.0 Editable installation hooks implemented according to :pep:`660`. Support for :pep:`namespace packages <420>` is still EXPERIMENTAL. "Strict" editable installs -------------------------- When thinking about editable installations, users might have the following expectations: 1. It should allow developers to add new files (or split/rename existing ones) and have them automatically exposed. 2. It should behave as close as possible to a regular installation and help users to detect problems (e.g. new files not being included in the distribution). Unfortunately these expectations are in conflict with each other. To solve this problem ``setuptools`` allows developers to choose a more "strict" mode for the editable installation. This can be done by passing a special configuration setting via :pypi:`pip`, as indicated below: .. code-block:: bash pip install -e . --config-settings editable_mode=strict In this mode, new files won't be exposed and the editable installs will try to mimic as much as possible the behavior of a regular install. Under the hood, ``setuptools`` will create a tree of file links in an auxiliary directory (``$your_project_dir/build``) and add it to ``PYTHONPATH`` via a :mod:`.pth file <site>`. (Please be careful to not delete this repository by mistake otherwise your files may stop being accessible). .. warning:: Strict editable installs require auxiliary files to be placed in a ``build/__editable__.`` directory (relative to your project root). Please be careful to not remove this directory while testing your project, otherwise your editable installation may be compromised. You can remove the ``build/__editable__.`` directory after uninstalling. .. note:: .. versionadded:: v64.0.0 Added new strict mode for editable installations. The exact details of how this mode is implemented may vary. Limitations ----------- - The editable term is used to refer only to Python modules inside the package directories. Non-Python files, external (data) files, executable script files, binary extensions, headers and metadata may be exposed as a snapshot of the version they were at the moment of the installation. - Adding new dependencies, entry-points or changing your project's metadata require a fresh "editable" re-installation. - Console scripts and GUI scripts MUST be specified via :doc:`entry-points </userguide/entry_point>` to work properly. - Strict editable installs require the file system to support either :wiki:`symbolic <symbolic link>` or :wiki:`hard links <hard link>`. This installation mode might also generate auxiliary files under the project directory. - There is no guarantee that the editable installation will be performed using a specific technique. Depending on each project, ``setuptools`` may select a different approach to ensure the package is importable at runtime. - There is no guarantee that files outside the top-level package directory will be accessible after an editable install. - There is no guarantee that attributes like ``__path__`` or ``__file__`` will correspond to the exact location of the original files (e.g., ``setuptools`` might employ file links to perform the editable installation). Users are encouraged to use tools like :mod:`importlib.resources` or :mod:`importlib.metadata` when trying to access package files directly. - Editable installations may not work with :doc:`namespaces created with pkgutil or pkg_resources <PyPUG:guides/packaging-namespace-packages>`. Please use :pep:`420`-style implicit namespaces [#namespaces]_. - Support for :pep:`420`-style implicit namespace packages for projects structured using :ref:`flat-layout` is still experimental. If you experience problems, you can try converting your package structure to the :ref:`src-layout`. - File system entries in the current working directory whose names coincidentally match installed packages may take precedence in :doc:`Python's import system <python:reference/import>`. Users are encouraged to avoid such scenarios [#cwd]_. .. attention:: Editable installs are not a perfect replacement for regular installs in a test environment. When in doubt, please test your projects as installed via a regular wheel. There are tools in the Python ecosystem, like :pypi:`tox` or :pypi:`nox`, that can help you with that (when used with appropriate configuration). Legacy Behavior --------------- If your project is not compatible with the new "editable installs" or you wish to replicate the legacy behavior, for the time being you can also perform the installation in the ``compat`` mode: .. code-block:: bash pip install -e . --config-settings editable_mode=compat This installation mode will try to emulate how ``python setup.py develop`` works (still within the context of :pep:`660`). .. warning:: The ``compat`` mode is transitional and will be removed in future versions of ``setuptools``, it exists only to help during the migration period. Also note that support for this mode is limited: it is safe to assume that the ``compat`` mode is offered "as is", and improvements are unlikely to be implemented. Users are encouraged to try out the new editable installation techniques and make the necessary adaptations. If the ``compat`` mode does not work for you, you can also disable the :pep:`editable install <660>` hooks in ``setuptools`` by setting an environment variable: .. code-block:: SETUPTOOLS_ENABLE_FEATURES="legacy-editable" This may cause the installer (e.g. ``pip``) to effectively run the "legacy" installation command: ``python setup.py develop`` [#installer]_. How editable installations work ------------------------------- Advanced topic There are many techniques that can be used to expose packages under development in such a way that they are available as if they were installed. Depending on the project file structure and the selected mode, ``setuptools`` will choose one of these approaches for the editable installation [#criteria]_. A non-exhaustive list of implementation mechanisms is presented below. More information is available on the text of :pep:`PEP 660 <660#what-to-put-in-the-wheel>`. - A static ``.pth`` file [#static_pth]_ can be added to one of the directories listed in :func:`site.getsitepackages` or :func:`site.getusersitepackages` to extend :obj:`sys.path`. - A directory containing a farm of file links that mimic the project structure and point to the original files can be employed. This directory can then be added to :obj:`sys.path` using a static ``.pth`` file. - A dynamic ``.pth`` file [#dynamic_pth]_ can also be used to install an "import :term:`finder`" (:obj:`~importlib.abc.MetaPathFinder` or :obj:`~importlib.abc.PathEntryFinder`) that will hook into Python's :doc:`import system <python:reference/import>` machinery. .. attention:: ``Setuptools`` offers no guarantee of which technique will be used to perform an editable installation. This will vary from project to project and may change depending on the specific version of ``setuptools`` being used. ---- .. rubric:: Notes .. [#namespaces] You may be able to use strict editable installations with namespace packages created with ``pkgutil`` or ``pkg_namespaces``, however this is not officially supported. .. [#cwd] Techniques like the :ref:`src-layout` or tooling-specific options like `tox's changedir <https://tox.wiki/en/stable/config.html#conf-changedir>`_ can be used to prevent such kinds of situations (checkout `this blog post <https://blog.ganssle.io/articles/2019/08/test-as-installed.html>`_ for more insights). .. [#installer] For this workaround to work, the installer tool needs to support legacy editable installations. (Future versions of ``pip``, for example, may drop support for this feature). .. [#criteria] ``setuptools`` strives to find a balance between allowing the user to see the effects of project files being edited while still trying to keep the editable installation as similar as possible to a regular installation. .. [#static_pth] i.e., a ``.pth`` file where each line correspond to a path that should be added to :obj:`sys.path`. See :mod:`Site-specific configuration hook <site>`. .. [#dynamic_pth] i.e., a ``.pth`` file that starts where each line starts with an ``import`` statement and executes arbitrary Python code. See :mod:`Site-specific configuration hook <site>`.	11.84 KB	2022-11-24 06:08
📄 distribution.rst .. _Specifying Your Project's Version: Specifying Your Project's Version ================================= Setuptools can work well with most versioning schemes. Over the years, setuptools has tried to closely follow the :pep:`440` scheme, but it also supports legacy versions. There are, however, a few special things to watch out for, in order to ensure that setuptools and other tools can always tell what version of your package is newer than another version. Knowing these things will also help you correctly specify what versions of other projects your project depends on. A version consists of an alternating series of release numbers and `pre-release <https://peps.python.org/pep-0440/#pre-releases>`_ or `post-release <https://peps.python.org/pep-0440/#post-releases>`_ tags. A release number is a series of digits punctuated by dots, such as ``2.4`` or ``0.5``. Each series of digits is treated numerically, so releases ``2.1`` and ``2.1.0`` are different ways to spell the same release number, denoting the first subrelease of release 2. But ``2.10`` is the tenth subrelease of release 2, and so is a different and newer release from ``2.1`` or ``2.1.0``. Leading zeros within a series of digits are also ignored, so ``2.01`` is the same as ``2.1``, and different from ``2.0.1``. Following a release number, you can have either a pre-release or post-release tag. Pre-release tags make a version be considered older than the version they are appended to. So, revision ``2.4`` is newer than release candidate ``2.4rc1``, which in turn is newer than beta release ``2.4b1`` or alpha release ``2.4a1``. Postrelease tags make a version be considered newer than the version they are appended to. So, revisions like ``2.4.post1`` are newer than ``2.4``, but older than ``2.4.1`` (which has a higher release number). In the case of legacy versions (for example, ``2.4pl1``), they are considered older than non-legacy versions. Taking that in count, a revision ``2.4pl1`` is older than ``2.4``. Note that ``2.4pl1`` is not :pep:`440`-compliant. A pre-release tag is a series of letters that are alphabetically before "final". Some examples of prerelease tags would include ``alpha``, ``beta``, ``a``, ``c``, ``dev``, and so on. You do not have to place a dot or dash before the prerelease tag if it's immediately after a number, but it's okay to do so if you prefer. Thus, ``2.4c1`` and ``2.4.c1`` and ``2.4-c1`` all represent release candidate 1 of version ``2.4``, and are treated as identical by setuptools. Note that only ``a``, ``b``, and ``rc`` are :pep:`440`-compliant pre-release tags. In addition, there are three special prerelease tags that are treated as if they were ``rc``: ``c``, ``pre``, and ``preview``. So, version ``2.4c1``, ``2.4pre1`` and ``2.4preview1`` are all the exact same version as ``2.4rc1``, and are treated as identical by setuptools. A post-release tag is the string ``.post``, followed by a non-negative integer value. Post-release tags are generally used to separate patch numbers, port numbers, build numbers, revision numbers, or date stamps from the release number. For example, the version ``2.4.post1263`` might denote Subversion revision 1263 of a post-release patch of version ``2.4``. Or you might use ``2.4.post20051127`` to denote a date-stamped post-release. Legacy post-release tags could be either a series of letters that are alphabetically greater than or equal to "final", or a dash (``-``) - for example ``2.4-r1263`` or ``2.4-20051127``. Notice that after each legacy pre or post-release tag, you are free to place another release number, followed again by more pre- or post-release tags. For example, ``0.6a9.dev41475`` could denote Subversion revision 41475 of the in- development version of the ninth alpha of release 0.6. Notice that ``dev`` is a pre-release tag, so this version is a lower version number than ``0.6a9``, which would be the actual ninth alpha of release 0.6. But the ``41475`` is a post-release tag, so this version is newer than ``0.6a9.dev``. For the most part, setuptools' interpretation of version numbers is intuitive, but here are a few tips that will keep you out of trouble in the corner cases: * Don't stick adjoining pre-release tags together without a dot or number between them. Version ``1.9adev`` is the ``adev`` prerelease of ``1.9``, not a development pre-release of ``1.9a``. Use ``.dev`` instead, as in ``1.9a.dev``, or separate the prerelease tags with a number, as in ``1.9a0dev``. ``1.9a.dev``, ``1.9a0dev``, and even ``1.9a0.dev0`` are identical versions from setuptools' point of view, so you can use whatever scheme you prefer. Of these examples, only ``1.9a0.dev0`` is :pep:`440`-compliant. * If you want to be certain that your chosen numbering scheme works the way you think it will, you can use the ``pkg_resources.parse_version()`` function to compare different version numbers:: >>> from pkg_resources import parse_version >>> parse_version("1.9.a.dev") == parse_version("1.9a0dev") True >>> parse_version("2.1-rc2") < parse_version("2.1") True >>> parse_version("0.6a9dev-r41475") < parse_version("0.6a9") True Once you've decided on a version numbering scheme for your project, you can have setuptools automatically tag your in-development releases with various pre- or post-release tags. See the following section for more details. Tagging and "Daily Build" or "Snapshot" Releases ------------------------------------------------ .. warning:: Please note that running ``python setup.py ...`` directly is no longer considered a good practice and that in the future the commands ``egg_info`` and ``rotate`` will be deprecated. As a result, the instructions and information presented in this section should be considered transitional while setuptools don't provide a mechanism for tagging releases. Meanwhile, if you can also consider using :pypi:`setuptools-scm` to achieve similar objectives. When a set of related projects are under development, it may be important to track finer-grained version increments than you would normally use for e.g. "stable" releases. While stable releases might be measured in dotted numbers with alpha/beta/etc. status codes, development versions of a project often need to be tracked by revision or build number or even build date. This is especially true when projects in development need to refer to one another, and therefore may literally need an up-to-the-minute version of something! To support these scenarios, ``setuptools`` allows you to "tag" your source and egg distributions by adding one or more of the following to the project's "official" version identifier: * A manually-specified pre-release tag, such as "build" or "dev", or a manually-specified post-release tag, such as a build or revision number (``--tag-build=STRING, -bSTRING``) * An 8-character representation of the build date (``--tag-date, -d``), as a postrelease tag You can add these tags by adding ``egg_info`` and the desired options to the command line ahead of the ``sdist`` or ``bdist`` commands that you want to generate a daily build or snapshot for. See the section below on the :ref:`egg_info <egg_info>` command for more details. (Also, before you release your project, be sure to see the section on :ref:`Specifying Your Project's Version` for more information about how pre- and post-release tags affect how version numbers are interpreted. This is important in order to make sure that dependency processing tools will know which versions of your project are newer than others). Finally, if you are creating builds frequently, and either building them in a downloadable location or are copying them to a distribution server, you should probably also check out the :ref:`rotate <rotate>` command, which lets you automatically delete all but the N most-recently-modified distributions matching a glob pattern. So, you can use a command line like:: setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3 to build an egg whose version info includes "DEV-rNNNN" (where NNNN is the most recent Subversion revision that affected the source tree), and then delete any egg files from the distribution directory except for the three that were built most recently. If you have to manage automated builds for multiple packages, each with different tagging and rotation policies, you may also want to check out the :ref:`alias <alias>` command, which would let each package define an alias like ``daily`` that would perform the necessary tag, build, and rotate commands. Then, a simpler script or cron job could just run ``setup.py daily`` in each project directory. (And, you could also define sitewide or per-user default versions of the ``daily`` alias, so that projects that didn't define their own would use the appropriate defaults.) Making "Official" (Non-Snapshot) Releases ----------------------------------------- When you make an official release, creating source or binary distributions, you will need to override the tag settings from ``setup.cfg``, so that you don't end up registering versions like ``foobar-0.7a1.dev-r34832``. This is easy to do if you are developing on the trunk and using tags or branches for your releases - just make the change to ``setup.cfg`` after branching or tagging the release, so the trunk will still produce development snapshots. Alternately, if you are not branching for releases, you can override the default version options on the command line, using something like:: setup.py egg_info -Db "" sdist bdist_egg The first part of this command (``egg_info -Db ""``) will override the configured tag information, before creating source and binary eggs. Thus, these commands will use the plain version from your ``setup.py``, without adding the build designation string. Of course, if you will be doing this a lot, you may wish to create a personal alias for this operation, e.g.:: setup.py alias -u release egg_info -Db "" You can then use it like this:: setup.py release sdist bdist_egg Or of course you can create more elaborate aliases that do all of the above. See the sections below on the :ref:`egg_info <egg_info>` and :ref:`alias <alias>` commands for more ideas.	10.07 KB	2022-11-24 06:08
📄 entry_point.rst .. _`entry_points`: ============ Entry Points ============ Entry points are a type of metadata that can be exposed by packages on installation. They are a very useful feature of the Python ecosystem, and come specially handy in two scenarios: 1. The package would like to provide commands to be run at the terminal. This functionality is known as console scripts. The command may also open up a GUI, in which case it is known as a GUI script. An example of a console script is the one provided by the :pypi:`pip` package, which allows you to run commands like ``pip install`` in the terminal. 2. A package would like to enable customization of its functionalities via plugins. For example, the test framework :pypi:`pytest` allows customization via the ``pytest11`` entry point, and the syntax highlighting tool :pypi:`pygments` allows specifying additional styles using the entry point ``pygments.styles``. .. _console-scripts: Console Scripts =============== Let us start with console scripts. First consider an example without entry points. Imagine a package defined thus:: project_root_directory ├── pyproject.toml # and/or setup.cfg, setup.py └── src └── timmins ├── __init__.py └── ... with ``__init__.py`` as: .. code-block:: python def hello_world(): print("Hello world") Now, suppose that we would like to provide some way of executing the function ``hello_world()`` from the command-line. One way to do this is to create a file ``src/timmins/__main__.py`` providing a hook as follows: .. code-block:: python from . import hello_world if __name__ == '__main__': hello_world() Then, after installing the package ``timmins``, we may invoke the ``hello_world()`` function as follows, through the `runpy <https://docs.python.org/3/library/runpy.html>`_ module: .. code-block:: bash $ python -m timmins Hello world Instead of this approach using ``__main__.py``, you can also create a user-friendly CLI executable that can be called directly without ``python -m``. In the above example, to create a command ``hello-world`` that invokes ``timmins.hello_world``, add a console script entry point to your configuration: .. tab:: pyproject.toml .. code-block:: toml [project.scripts] hello-world = "timmins:hello_world" .. tab:: setup.cfg .. code-block:: ini [options.entry_points] console_scripts = hello-world = timmins:hello_world .. tab:: setup.py .. code-block:: python from setuptools import setup setup( # ..., entry_points={ 'console_scripts': [ 'hello-world = timmins:hello_world', ] } ) After installing the package, a user may invoke that function by simply calling ``hello-world`` on the command line: .. code-block:: bash $ hello-world Hello world Note that any function configured as a console script, i.e. ``hello_world()`` in this example, should not accept any arguments. If your function requires any input from the user, you can use regular command-line argument parsing utilities like :mod:`argparse` within the body of the function to parse user input given via :obj:`sys.argv`. You may have noticed that we have used a special syntax to specify the function that must be invoked by the console script, i.e. we have written ``timmins:hello_world`` with a colon ``:`` separating the package name and the function name. The full specification of this syntax is discussed in the `last section <#entry-points-syntax>`_ of this document, and this can be used to specify a function located anywhere in your package, not just in ``__init__.py``. GUI Scripts =========== In addition to ``console_scripts``, Setuptools supports ``gui_scripts``, which will launch a GUI application without running in a terminal window. For example, if we have a project with the same directory structure as before, with an ``__init__.py`` file containing the following: .. code-block:: python import PySimpleGUI as sg def hello_world(): sg.Window(title="Hello world", layout=[[]], margins=(100, 50)).read() Then, we can add a GUI script entry point: .. tab:: pyproject.toml .. code-block:: toml [project.gui-scripts] hello-world = "timmins:hello_world" .. tab:: setup.cfg .. code-block:: ini [options.entry_points] gui_scripts = hello-world = timmins:hello_world .. tab:: setup.py .. code-block:: python from setuptools import setup setup( # ..., entry_points={ 'gui_scripts': [ 'hello-world = timmins:hello_world', ] } ) .. note:: To be able to import ``PySimpleGUI``, you need to add ``pysimplegui`` to your package dependencies. See :doc:`/userguide/dependency_management` for more information. Now, running: .. code-block:: bash $ hello-world will open a small application window with the title 'Hello world'. Note that just as with console scripts, any function configured as a GUI script should not accept any arguments, and any user input can be parsed within the body of the function. GUI scripts also use the same syntax (discussed in the `last section <#entry-points-syntax>`_) for specifying the function to be invoked. .. note:: The difference between ``console_scripts`` and ``gui_scripts`` only affects Windows systems. [#use_for_scripts]_ ``console_scripts`` are wrapped in a console executable, so they are attached to a console and can use ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` for input and output. ``gui_scripts`` are wrapped in a GUI executable, so they can be started without a console, but cannot use standard streams unless application code redirects them. Other platforms do not have the same distinction. .. note:: Console and GUI scripts work because behind the scenes, installers like :pypi:`pip` create wrapper scripts around the function(s) being invoked. For example, the ``hello-world`` entry point in the above two examples would create a command ``hello-world`` launching a script like this: [#use_for_scripts]_ .. code-block:: python import sys from timmins import hello_world sys.exit(hello_world()) .. _dynamic discovery of services and plugins: Advertising Behavior ==================== Console/GUI scripts are one use of the more general concept of entry points. Entry points more generally allow a packager to advertise behavior for discovery by other libraries and applications. This feature enables "plug-in"-like functionality, where one library solicits entry points and any number of other libraries provide those entry points. A good example of this plug-in behavior can be seen in `pytest plugins <https://docs.pytest.org/en/latest/writing_plugins.html>`_, where pytest is a test framework that allows other libraries to extend or modify its functionality through the ``pytest11`` entry point. The console/GUI scripts work similarly, where libraries advertise their commands and tools like ``pip`` create wrapper scripts that invoke those commands. Entry Points for Plugins ======================== Let us consider a simple example to understand how we can implement entry points corresponding to plugins. Say we have a package ``timmins`` with the following directory structure:: timmins ├── pyproject.toml # and/or setup.cfg, setup.py └── src └── timmins └── __init__.py and in ``src/timmins/__init__.py`` we have the following code: .. code-block:: python def hello_world(): print('Hello world') Basically, we have defined a ``hello_world()`` function which will print the text 'Hello world'. Now, let us say we want to print the text 'Hello world' in different ways. The current function just prints the text as it is - let us say we want another style in which the text is enclosed within exclamation marks:: !!! Hello world !!! Let us see how this can be done using plugins. First, let us separate the style of printing the text from the text itself. In other words, we can change the code in ``src/timmins/__init__.py`` to something like this: .. code-block:: python def display(text): print(text) def hello_world(): display('Hello world') Here, the ``display()`` function controls the style of printing the text, and the ``hello_world()`` function calls the ``display()`` function to print the text 'Hello world`. Right now the ``display()`` function just prints the text as it is. In order to be able to customize it, we can do the following. Let us introduce a new group of entry points named ``timmins.display``, and expect plugin packages implementing this entry point to supply a ``display()``-like function. Next, to be able to automatically discover plugin packages that implement this entry point, we can use the :mod:`importlib.metadata` module, as follows: .. code-block:: python from importlib.metadata import entry_points display_eps = entry_points(group='timmins.display') .. note:: Each ``importlib.metadata.EntryPoint`` object is an object containing a ``name``, a ``group``, and a ``value``. For example, after setting up the plugin package as described below, ``display_eps`` in the above code will look like this: [#package_metadata]_ .. code-block:: python ( EntryPoint(name='excl', value='timmins_plugin_fancy:excl_display', group='timmins.display'), ..., ) ``display_eps`` will now be a list of ``EntryPoint`` objects, each referring to ``display()``-like functions defined by one or more installed plugin packages. Then, to import a specific ``display()``-like function - let us choose the one corresponding to the first discovered entry point - we can use the ``load()`` method as follows: .. code-block:: python display = display_eps[0].load() Finally, a sensible behaviour would be that if we cannot find any plugin packages customizing the ``display()`` function, we should fall back to our default implementation which prints the text as it is. With this behaviour included, the code in ``src/timmins/__init__.py`` finally becomes: .. code-block:: python from importlib.metadata import entry_points display_eps = entry_points(group='timmins.display') try: display = display_eps[0].load() except IndexError: def display(text): print(text) def hello_world(): display('Hello world') That finishes the setup on ``timmins``'s side. Next, we need to implement a plugin which implements the entry point ``timmins.display``. Let us name this plugin ``timmins-plugin-fancy``, and set it up with the following directory structure:: timmins-plugin-fancy ├── pyproject.toml # and/or setup.cfg, setup.py └── src └── timmins_plugin_fancy └── __init__.py And then, inside ``src/timmins_plugin_fancy/__init__.py``, we can put a function named ``excl_display()`` that prints the given text surrounded by exclamation marks: .. code-block:: python def excl_display(text): print('!!!', text, '!!!') This is the ``display()``-like function that we are looking to supply to the ``timmins`` package. We can do that by adding the following in the configuration of ``timmins-plugin-fancy``: .. tab:: pyproject.toml .. code-block:: toml # Note the quotes around timmins.display in order to escape the dot . [project.entry-points."timmins.display"] excl = "timmins_plugin_fancy:excl_display" .. tab:: setup.cfg .. code-block:: ini [options.entry_points] timmins.display = excl = timmins_plugin_fancy:excl_display .. tab:: setup.py .. code-block:: python from setuptools import setup setup( # ..., entry_points = { 'timmins.display': [ 'excl = timmins_plugin_fancy:excl_display' ] } ) Basically, this configuration states that we are a supplying an entry point under the group ``timmins.display``. The entry point is named ``excl`` and it refers to the function ``excl_display`` defined by the package ``timmins-plugin-fancy``. Now, if we install both ``timmins`` and ``timmins-plugin-fancy``, we should get the following: .. code-block:: pycon >>> from timmins import hello_world >>> hello_world() !!! Hello world !!! whereas if we only install ``timmins`` and not ``timmins-plugin-fancy``, we should get the following: .. code-block:: pycon >>> from timmins import hello_world >>> hello_world() Hello world Therefore, our plugin works. Our plugin could have also defined multiple entry points under the group ``timmins.display``. For example, in ``src/timmins_plugin_fancy/__init__.py`` we could have two ``display()``-like functions, as follows: .. code-block:: python def excl_display(text): print('!!!', text, '!!!') def lined_display(text): print(''.join(['-' for _ in text])) print(text) print(''.join(['-' for _ in text])) The configuration of ``timmins-plugin-fancy`` would then change to: .. tab:: pyproject.toml .. code-block:: toml [project.entry-points."timmins.display"] excl = "timmins_plugin_fancy:excl_display" lined = "timmins_plugin_fancy:lined_display" .. tab:: setup.cfg .. code-block:: ini [options.entry_points] timmins.display = excl = timmins_plugin_fancy:excl_display lined = timmins_plugin_fancy:lined_display .. tab:: setup.py .. code-block:: python from setuptools import setup setup( # ..., entry_points = { 'timmins.display': [ 'excl = timmins_plugin_fancy:excl_display', 'lined = timmins_plugin_fancy:lined_display', ] } ) On the ``timmins`` side, we can also use a different strategy of loading entry points. For example, we can search for a specific display style: .. code-block:: python display_eps = entry_points(group='timmins.display') try: display = display_eps['lined'].load() except KeyError: # if the 'lined' display is not available, use something else ... Or we can also load all plugins under the given group. Though this might not be of much use in our current example, there are several scenarios in which this is useful: .. code-block:: python display_eps = entry_points(group='timmins.display') for ep in display_eps: display = ep.load() # do something with display ... Another point is that in this particular example, we have used plugins to customize the behaviour of a function (``display()``). In general, we can use entry points to enable plugins to not only customize the behaviour of functions, but also of entire classes and modules. This is unlike the case of console/GUI scripts, where entry points can only refer to functions. The syntax used for specifying the entry points remains the same as for console/GUI scripts, and is discussed in the `last section <#entry-points-syntax>`_. .. tip:: The recommended approach for loading and importing entry points is the :mod:`importlib.metadata` module, which is a part of the standard library since Python 3.8. For older versions of Python, its backport :pypi:`importlib_metadata` should be used. While using the backport, the only change that has to be made is to replace ``importlib.metadata`` with ``importlib_metadata``, i.e. .. code-block:: python from importlib_metadata import entry_points ... In summary, entry points allow a package to open its functionalities for customization via plugins. The package soliciting the entry points need not have any dependency or prior knowledge about the plugins implementing the entry points, and downstream users are able to compose functionality by pulling together plugins implementing the entry points. Entry Points Syntax =================== The syntax for entry points is specified as follows:: <name> = <package_or_module>[:<object>[.<attr>[.<nested-attr>]]] Here, the square brackets ``[]`` denote optionality and the asterisk ```` denotes repetition. ``name`` is the name of the script/entry point you want to create, the left hand side of ``:`` is the package or module that contains the object you want to invoke (think about it as something you would write in an import statement), and the right hand side is the object you want to invoke (e.g. a function). To make this syntax more clear, consider the following examples: Package or module If you supply:: <name> = <package_or_module> as the entry point, where ``<package_or_module>`` can contain ``.`` in the case of sub-modules or sub-packages, then, tools in the Python ecosystem will roughly interpret this value as: .. code-block:: python import <package_or_module> parsed_value = <package_or_module> Module-level object If you supply:: <name> = <package_or_module>:<object> where ``<object>`` does not contain any ``.``, this will be roughly interpreted as: .. code-block:: python from <package_or_module> import <object> parsed_value = <object> Nested object If you supply:: <name> = <package_or_module>:<object>.<attr>.<nested_attr> this will be roughly interpreted as: .. code-block:: python from <package_or_module> import <object> parsed_value = <object>.<attr>.<nested_attr> In the case of console/GUI scripts, this syntax can be used to specify a function, while in the general case of entry points as used for plugins, it can be used to specify a function, class or module. ---- .. [#use_for_scripts] Reference: https://packaging.python.org/en/latest/specifications/entry-points/#use-for-scripts .. [#package_metadata] Reference: https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/#using-package-metadata	17.94 KB	2022-11-24 06:08
📄 ext_modules.rst ========================== Building Extension Modules ========================== Setuptools can build C/C++ extension modules. The keyword argument ``ext_modules`` of ``setup()`` should be a list of instances of the :class:`setuptools.Extension` class. For example, let's consider a simple project with only one extension module:: <project_folder> ├── pyproject.toml └── foo.c and all project metadata configuration in the ``pyproject.toml`` file: .. code-block:: toml # pyproject.toml [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" [project] name = "mylib-foo" # as it would appear on PyPI version = "0.42" To instruct setuptools to compile the ``foo.c`` file into the extension module ``mylib.foo``, we need to add a ``setup.py`` file similar to the following: .. code-block:: python from setuptools import Extension, setup setup( ext_modules=[ Extension( name="mylib.foo", # as it would be imported # may include packages/namespaces separated by `.` sources=["foo.c"], # all sources are compiled into a single binary file ), ] ) .. seealso:: You can find more information on the `Python docs about C/C++ extensions`_. Alternatively, you might also be interested in learn about `Cython`_. If you plan to distribute a package that uses extensions across multiple platforms, :pypi:`cibuildwheel` can also be helpful. .. important:: All files used to compile your extension need to be available on the system when building the package, so please make sure to include some documentation on how developers interested in building your package from source can obtain operating system level dependencies (e.g. compilers and external binary libraries/artifacts). You will also need to make sure that all auxiliary files that are contained inside your :term:`project` (e.g. C headers authored by you or your team) are configured to be included in your :term:`sdist <Source Distribution (or "sdist")>`. Please have a look on our section on :ref:`Controlling files in the distribution`. Compiler and linker options =========================== The command ``build_ext`` builds C/C++ extension modules. It creates a command line for running the compiler and linker by combining compiler and linker options from various sources: .. Reference: `test_customize_compiler` in distutils/tests/test_sysconfig.py * the ``sysconfig`` variables ``CC``, ``CXX``, ``CCSHARED``, ``LDSHARED``, and ``CFLAGS``, * the environment variables ``CC``, ``CPP``, ``CXX``, ``LDSHARED`` and ``LDFLAGS``, ``CFLAGS``, ``CPPFLAGS``, ``LDFLAGS``, * the ``Extension`` attributes ``include_dirs``, ``library_dirs``, ``extra_compile_args``, ``extra_link_args``, ``runtime_library_dirs``. .. Ignoring AR, ARFLAGS, RANLIB here because they are used by the (obsolete?) build_clib, not build_ext. Specifically, if the environment variables ``CC``, ``CPP``, ``CXX``, and ``LDSHARED`` are set, they will be used instead of the ``sysconfig`` variables of the same names. The compiler options appear in the command line in the following order: .. Reference: "compiler_so" and distutils.ccompiler.gen_preprocess_options, CCompiler.compile, UnixCCompiler._compile * first, the options provided by the ``sysconfig`` variable ``CFLAGS``, * then, the options provided by the environment variables ``CFLAGS`` and ``CPPFLAGS``, * then, the options provided by the ``sysconfig`` variable ``CCSHARED``, * then, a ``-I`` option for each element of ``Extension.include_dirs``, * finally, the options provided by ``Extension.extra_compile_args``. The linker options appear in the command line in the following order: .. Reference: "linker_so" and CCompiler.link * first, the options provided by environment variables and ``sysconfig`` variables, * then, a ``-L`` option for each element of ``Extension.library_dirs``, * then, a linker-specific option like ``-Wl,-rpath`` for each element of ``Extension.runtime_library_dirs``, * finally, the options provided by ``Extension.extra_link_args``. The resulting command line is then processed by the compiler and linker. According to the GCC manual sections on `directory options`_ and `environment variables`_, the C/C++ compiler searches for files named in ``#include <file>`` directives in the following order: * first, in directories given by ``-I`` options (in left-to-right order), * then, in directories given by the environment variable ``CPATH`` (in left-to-right order), * then, in directories given by ``-isystem`` options (in left-to-right order), * then, in directories given by the environment variable ``C_INCLUDE_PATH`` (for C) and ``CPLUS_INCLUDE_PATH`` (for C++), * then, in standard system directories, * finally, in directories given by ``-idirafter`` options (in left-to-right order). The linker searches for libraries in the following order: * first, in directories given by ``-L`` options (in left-to-right order), * then, in directories given by the environment variable ``LIBRARY_PATH`` (in left-to-right order). Distributing Extensions compiled with Cython ============================================ When your :pypi:`Cython` extension modules are declared using the :class:`setuptools.Extension` class, ``setuptools`` will detect at build time whether Cython is installed or not. If Cython is present, then ``setuptools`` will use it to build the ``.pyx`` files. Otherwise, ``setuptools`` will try to find and compile the equivalent ``.c`` files (instead of ``.pyx``). These files can be generated using the `cython command line tool`_. You can ensure that Cython is always automatically installed into the build environment by including it as a :ref:`build dependency <build-requires>` in your ``pyproject.toml``: .. code-block:: toml [build-system] requires = [..., "cython"] Alternatively, you can include the ``.c`` code that is pre-compiled by Cython into your source distribution, alongside the original ``.pyx`` files (this might save a few seconds when building from an ``sdist``). To improve version compatibility, you probably also want to include current ``.c`` files in your :wiki:`revision control system`, and rebuild them whenever you check changes in for the ``.pyx`` source files. This will ensure that people tracking your project will be able to build it without installing Cython, and that there will be no variation due to small differences in the generate C files. Please checkout our docs on :ref:`controlling files in the distribution` for more information. ---- Extension API Reference ======================= .. autoclass:: setuptools.Extension .. _Python docs about C/C++ extensions: https://docs.python.org/3/extending/extending.html .. _Cython: https://cython.readthedocs.io/en/stable/index.html .. _directory options: https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html .. _environment variables: https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html .. _cython command line tool: https://cython.readthedocs.io/en/stable/src/userguide/source_files_and_compilation.html	7.05 KB	2022-11-24 06:08
📄 extension.rst .. _Creating ``distutils`` Extensions: Extending or Customizing Setuptools =================================== Setuptools design is based on the distutils_ package originally distributed as part of Python's standard library, effectively serving as its successor (as established in :pep:`632`). This means that ``setuptools`` strives to honor the extension mechanisms provided by ``distutils``, and allows developers to create third party packages that modify or augment the build process behavior. A simple way of doing that is to hook in new or existing commands and ``setup()`` arguments just by defining "entry points". These are mappings from command or argument names to a specification of where to import a handler from. (See the section on :ref:`Dynamic Discovery of Services and Plugins` for some more background on entry points). The following sections describe the most common procedures for extending the ``distutils`` functionality used by ``setuptools``. .. important:: Any entry-point defined in your ``setup.cfg``, ``setup.py`` or ``pyproject.toml`` files are not immediately available for use. Your package needs to be installed first, then ``setuptools`` will be able to access these entry points. For example consider a ``Project-A`` that defines entry points. When building ``Project-A``, these will not be available. If ``Project-B`` declares a :doc:`build system requirement </userguide/dependency_management>` on ``Project-A``, then ``setuptools`` will be able to use ``Project-A``' customizations. Customizing Commands -------------------- Both ``setuptools`` and ``distutils`` are structured around the command design pattern. This means that each main action executed when building a distribution package (such as creating a :term:`sdist <Source Distribution (or "sdist")>` or :term:`wheel`) correspond to the implementation of a Python class. Originally in ``distutils``, these commands would correspond to actual CLI arguments that could be passed to the ``setup.py`` script to trigger a different aspect of the build. In ``setuptools``, however, these command objects are just a design abstraction that encapsulate logic and help to organise the code. You can overwrite exiting commands (or add new ones) by defining entry points in the ``distutils.commands`` group. For example, if you wanted to add a ``foo`` command, you might add something like this to your project: .. code-block:: ini # setup.cfg ... [options.entry_points] distutils.commands = foo = mypackage.some_module:foo Assuming, of course, that the ``foo`` class in ``mypackage.some_module`` is a ``setuptools.Command`` subclass (documented below). Once a project containing such entry points has been activated on ``sys.path``, (e.g. by running ``pip install``) the command(s) will be available to any ``setuptools``-based project. In fact, this is how setuptools' own commands are installed: the setuptools project's setup script defines entry points for them! The commands ``sdist``, ``build_py`` and ``build_ext`` are especially useful to customize ``setuptools`` builds. Note however that when overwriting existing commands, you should be very careful to maintain API compatibility. Custom commands should try to replicate the same overall behavior as the original classes, and when possible, even inherit from them. You should also consider handling exceptions such as ``CompileError``, ``LinkError``, ``LibError``, among others. These exceptions are available in the ``setuptools.errors`` module. .. autoclass:: setuptools.Command :members: Supporting sdists and editable installs in ``build`` sub-commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``build`` sub-commands (like ``build_py`` and ``build_ext``) are encouraged to implement the following protocol: .. autoclass:: setuptools.command.build.SubCommand :members: Adding Arguments ---------------- .. warning:: Adding arguments to setup is discouraged as such arguments are only supported through imperative execution and not supported through declarative config. Sometimes, your commands may need additional arguments to the ``setup()`` call. You can enable this by defining entry points in the ``distutils.setup_keywords`` group. For example, if you wanted a ``setup()`` argument called ``bar_baz``, you might add something like this to your extension project: .. code-block:: ini # setup.cfg ... [options.entry_points] distutils.commands = foo = mypackage.some_module:foo distutils.setup_keywords = bar_baz = mypackage.some_module:validate_bar_baz The idea here is that the entry point defines a function that will be called to validate the ``setup()`` argument, if it's supplied. The ``Distribution`` object will have the initial value of the attribute set to ``None``, and the validation function will only be called if the ``setup()`` call sets it to a non-``None`` value. Here's an example validation function:: def assert_bool(dist, attr, value): """Verify that value is True, False, 0, or 1""" if bool(value) != value: raise SetupError( "%r must be a boolean value (got %r)" % (attr,value) ) Your function should accept three arguments: the ``Distribution`` object, the attribute name, and the attribute value. It should raise a ``SetupError`` (from the ``setuptools.errors`` module) if the argument is invalid. Remember, your function will only be called with non-``None`` values, and the default value of arguments defined this way is always ``None``. So, your commands should always be prepared for the possibility that the attribute will be ``None`` when they access it later. If more than one active distribution defines an entry point for the same ``setup()`` argument, all of them will be called. This allows multiple extensions to define a common argument, as long as they agree on what values of that argument are valid. Customizing Distribution Options -------------------------------- Plugins may wish to extend or alter the options on a ``Distribution`` object to suit the purposes of that project. For example, a tool that infers the ``Distribution.version`` from SCM-metadata may need to hook into the option finalization. To enable this feature, Setuptools offers an entry point ``setuptools.finalize_distribution_options``. That entry point must be a callable taking one argument (the ``Distribution`` instance). If the callable has an ``.order`` property, that value will be used to determine the order in which the hook is called. Lower numbers are called first and the default is zero (0). Plugins may read, alter, and set properties on the distribution, but each plugin is encouraged to load the configuration/settings for their behavior independently. Defining Additional Metadata ---------------------------- Some extensible applications and frameworks may need to define their own kinds of metadata, which they can then access using the :mod:`importlib.metadata` APIs. Ordinarily, this is done by having plugin developers include additional files in their ``ProjectName.egg-info`` directory. However, since it can be tedious to create such files by hand, you may want to create an extension that will create the necessary files from arguments to ``setup()``, in much the same way that ``setuptools`` does for many of the ``setup()`` arguments it adds. See the section below for more details. .. _Adding new EGG-INFO Files: Adding new EGG-INFO Files ~~~~~~~~~~~~~~~~~~~~~~~~~ Some extensible applications or frameworks may want to allow third parties to develop plugins with application or framework-specific metadata included in the plugins' EGG-INFO directory, for easy access via the ``pkg_resources`` metadata API. The easiest way to allow this is to create an extension to be used from the plugin projects' setup scripts (via ``setup_requires``) that defines a new setup keyword, and then uses that data to write an EGG-INFO file when the ``egg_info`` command is run. The ``egg_info`` command looks for extension points in an ``egg_info.writers`` group, and calls them to write the files. Here's a simple example of an extension defining a setup argument ``foo_bar``, which is a list of lines that will be written to ``foo_bar.txt`` in the EGG-INFO directory of any project that uses the argument: .. code-block:: ini # setup.cfg ... [options.entry_points] distutils.setup_keywords = foo_bar = setuptools.dist:assert_string_list egg_info.writers = foo_bar.txt = setuptools.command.egg_info:write_arg This simple example makes use of two utility functions defined by setuptools for its own use: a routine to validate that a setup keyword is a sequence of strings, and another one that looks up a setup argument and writes it to a file. Here's what the writer utility looks like:: def write_arg(cmd, basename, filename): argname = os.path.splitext(basename)[0] value = getattr(cmd.distribution, argname, None) if value is not None: value = "\n".join(value) + "\n" cmd.write_or_delete_file(argname, filename, value) As you can see, ``egg_info.writers`` entry points must be a function taking three arguments: a ``egg_info`` command instance, the basename of the file to write (e.g. ``foo_bar.txt``), and the actual full filename that should be written to. In general, writer functions should honor the command object's ``dry_run`` setting when writing files, and use ``logging`` to do any console output. The easiest way to conform to this requirement is to use the ``cmd`` object's ``write_file()``, ``delete_file()``, and ``write_or_delete_file()`` methods exclusively for your file operations. See those methods' docstrings for more details. .. _Adding Support for Revision Control Systems: Adding Support for Revision Control Systems ------------------------------------------------- If the files you want to include in the source distribution are tracked using Git, Mercurial or SVN, you can use the following packages to achieve that: - Git and Mercurial: :pypi:`setuptools_scm` - SVN: :pypi:`setuptools_svn` If you would like to create a plugin for ``setuptools`` to find files tracked by another revision control system, you can do so by adding an entry point to the ``setuptools.file_finders`` group. The entry point should be a function accepting a single directory name, and should yield all the filenames within that directory (and any subdirectories thereof) that are under revision control. For example, if you were going to create a plugin for a revision control system called "foobar", you would write a function something like this: .. code-block:: python def find_files_for_foobar(dirname): ... # loop to yield paths that start with `dirname` And you would register it in a setup script using something like this: .. code-block:: ini # setup.cfg ... [options.entry_points] setuptools.file_finders = foobar = my_foobar_module:find_files_for_foobar Then, anyone who wants to use your plugin can simply install it, and their local setuptools installation will be able to find the necessary files. It is not necessary to distribute source control plugins with projects that simply use the other source control system, or to specify the plugins in ``setup_requires``. When you create a source distribution with the ``sdist`` command, setuptools automatically records what files were found in the ``SOURCES.txt`` file. That way, recipients of source distributions don't need to have revision control at all. However, if someone is working on a package by checking out with that system, they will need the same plugin(s) that the original author is using. A few important points for writing revision control file finders: * Your finder function MUST return relative paths, created by appending to the passed-in directory name. Absolute paths are NOT allowed, nor are relative paths that reference a parent directory of the passed-in directory. * Your finder function MUST accept an empty string as the directory name, meaning the current directory. You MUST NOT convert this to a dot; just yield relative paths. So, yielding a subdirectory named ``some/dir`` under the current directory should NOT be rendered as ``./some/dir`` or ``/somewhere/some/dir``, but always as simply ``some/dir`` * Your finder function SHOULD NOT raise any errors, and SHOULD deal gracefully with the absence of needed programs (i.e., ones belonging to the revision control system itself. It may, however, use ``logging.warning()`` to inform the user of the missing program(s). .. _distutils: https://docs.python.org/3.9/library/distutils.html Final Remarks ------------- * To use a ``setuptools`` plugin, your users will need to add your package as a build requirement to their build-system configuration. Please check out our guides on :doc:`/userguide/dependency_management` for more information. * Directly calling ``python setup.py ...`` is considered a deprecated practice. You should not add new commands to ``setuptools`` expecting them to be run via this interface.	12.99 KB	2022-11-24 06:08
📄 index.rst ================================================== Building and Distributing Packages with Setuptools ================================================== The first step towards sharing a Python library or program is to build a distribution package [#package-overload]_. This includes adding a set of additional files containing metadata and configuration to not only instruct ``setuptools`` on how the distribution should be built but also to help installer (such as :pypi:`pip`) during the installation process. This document contains information to help Python developers through this process. Please check the :doc:`/userguide/quickstart` for an overview of the workflow. Also note that ``setuptools`` is what is known in the community as :pep:`build backend <517#terminology-and-goals>`, user facing interfaces are provided by tools such as :pypi:`pip` and :pypi:`build`. To use ``setuptools``, one must explicitly create a ``pyproject.toml`` file as described :doc:`/build_meta`. Contents ======== .. toctree:: :maxdepth: 1 quickstart package_discovery dependency_management development_mode entry_point datafiles ext_modules distribution miscellaneous extension declarative_config pyproject_config --- .. rubric:: Notes .. [#package-overload] A :term:`Distribution Package` is also referred in the Python community simply as "package" Unfortunately, this jargon might be a bit confusing for new users because the term package can also to refer any :term:`directory <package>` (or sub directory) used to organize :term:`modules <module>` and auxiliary files.	1.6 KB	2022-11-24 06:08
📄 miscellaneous.rst .. _Controlling files in the distribution: Controlling files in the distribution ===================================== For the most common use cases, ``setuptools`` will automatically find out which files are necessary for distributing the package. These include all :term:`pure Python modules <Pure Module>` in the ``py_modules`` or ``packages`` configuration, and the C sources (but not C headers) listed as part of extensions when creating a :term:`source distribution (or "sdist")`. However, when building more complex packages (e.g. packages that include non-Python files, or that need to use custom C headers), you might find that not all files present in your project folder are included in package :term:`distribution archive <Distribution Package>`. If you are using a :wiki:`Revision Control System`, such as git_ or mercurial_, and your source distributions only need to include files that you're tracking in revision control, you can use a ``setuptools`` :ref:`plugin <Adding Support for Revision Control Systems>`, such as :pypi:`setuptools-scm` or :pypi:`setuptools-svn` to automatically include all tracked files into the ``sdist``. .. _Using MANIFEST.in: Alternatively, if you need finer control over the files (e.g. you don't want to distribute :wiki:`CI/CD`-related files) or you need automatically generated files, you can add a ``MANIFEST.in`` file at the root of your project, to specify any files that the default file location algorithm doesn't catch. This file contains instructions that tell ``setuptools`` which files exactly should be part of the ``sdist`` (or not). A comprehensive guide to ``MANIFEST.in`` syntax is available at the :doc:`PyPA's Packaging User Guide <PyPUG:guides/using-manifest-in>`. .. attention:: Please note that ``setuptools`` supports the ``MANIFEST.in``, and not ``MANIFEST`` (no extension). Any documentation, tutorial or example that recommends using ``MANIFEST`` (no extension) is likely outdated. .. tip:: The ``MANIFEST.in`` file contains commands that allow you to discover and manipulate lists of files. There are many commands that can be used with different objectives, but you should try to not make your ``MANIFEST.in`` file too fine grained. A good idea is to start with a ``graft`` command (to add all files inside a set of directories) and then fine tune the file selection by removing the excess or adding isolated files. An example of ``MANIFEST.in`` for a simple project that organized according to a :ref:`src-layout` is: .. code-block:: bash # MANIFEST.in -- just for illustration graft src graft tests graft docs # `-> adds all files inside a directory include tox.ini # `-> matches file paths relative to the root of the project global-exclude ~ .py[cod] .so # `-> matches file names (regardless of directory) Once the correct files are present in the ``sdist``, they can then be used by binary extensions during the build process, or included in the final :term:`wheel <Wheel>` [#build-process]_ if you configure ``setuptools`` with ``include_package_data=True``. .. important:: Please note that, when using ``include_package_data=True``, only files inside the package directory* are included in the final ``wheel``, by default. So for example, if you create a :term:`Python project <Project>` that uses :pypi:`setuptools-scm` and have a ``tests`` directory outside of the package folder, the ``tests`` directory will be present in the ``sdist`` but not in the ``wheel`` [#wheel-vs-sdist]_. See :doc:`/userguide/datafiles` for more information. ---- .. [#build-process] You can think about the build process as two stages: first the ``sdist`` will be created and then the ``wheel`` will be produced from that ``sdist``. .. [#wheel-vs-sdist] This happens because the ``sdist`` can contain files that are useful during development or the build process itself, but not in runtime (e.g. tests, docs, examples, etc...). The ``wheel``, on the other hand, is a file format that has been optimized and is ready to be unpacked into a running installation of Python or :term:`Virtual Environment`. Therefore it only contains items that are required during runtime. .. _git: https://git-scm.com .. _mercurial: https://www.mercurial-scm.org	4.24 KB	2022-11-24 06:08
📄 package_discovery.rst .. _`package_discovery`: ======================================== Package Discovery and Namespace Packages ======================================== .. note:: a full specification for the keywords supplied to ``setup.cfg`` or ``setup.py`` can be found at :doc:`keywords reference </references/keywords>` .. important:: The examples provided here are only to demonstrate the functionality introduced. More metadata and options arguments need to be supplied if you want to replicate them on your system. If you are completely new to setuptools, the :doc:`quickstart` section is a good place to start. ``Setuptools`` provides powerful tools to handle package discovery, including support for namespace packages. Normally, you would specify the packages to be included manually in the following manner: .. tab:: setup.cfg .. code-block:: ini [options] #... packages = mypkg mypkg.subpkg1 mypkg.subpkg2 .. tab:: setup.py .. code-block:: python setup( # ... packages=['mypkg', 'mypkg.subpkg1', 'mypkg.subpkg2'] ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml # ... [tool.setuptools] packages = ["mypkg", "mypkg.subpkg1", "mypkg.subpkg2"] # ... If your packages are not in the root of the repository or do not correspond exactly to the directory structure, you also need to configure ``package_dir``: .. tab:: setup.cfg .. code-block:: ini [options] # ... package_dir = = src # directory containing all the packages (e.g. src/mypkg, src/mypkg/subpkg1, ...) # OR package_dir = mypkg = lib # mypkg.module corresponds to lib/module.py mypkg.subpkg1 = lib1 # mypkg.subpkg1.module1 corresponds to lib1/module1.py mypkg.subpkg2 = lib2 # mypkg.subpkg2.module2 corresponds to lib2/module2.py # ... .. tab:: setup.py .. code-block:: python setup( # ... package_dir = {"": "src"} # directory containing all the packages (e.g. src/mypkg, src/mypkg/subpkg1, ...) ) # OR setup( # ... package_dir = { "mypkg": "lib", # mypkg.module corresponds to lib/mod.py "mypkg.subpkg1": "lib1", # mypkg.subpkg1.module1 corresponds to lib1/module1.py "mypkg.subpkg2": "lib2" # mypkg.subpkg2.module2 corresponds to lib2/module2.py # ... ) .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools] # ... package-dir = {"" = "src"} # directory containing all the packages (e.g. src/mypkg1, src/mypkg2) # OR [tool.setuptools.package-dir] mypkg = "lib" # mypkg.module corresponds to lib/module.py "mypkg.subpkg1" = "lib1" # mypkg.subpkg1.module1 corresponds to lib1/module1.py "mypkg.subpkg2" = "lib2" # mypkg.subpkg2.module2 corresponds to lib2/module2.py # ... This can get tiresome really quickly. To speed things up, you can rely on setuptools automatic discovery, or use the provided tools, as explained in the following sections. .. important:: Although ``setuptools`` allows developers to create a very complex mapping between directory names and package names, it is better to keep it simple and reflect the desired package hierarchy in the directory structure, preserving the same names. .. _auto-discovery: Automatic discovery =================== .. warning:: Automatic discovery is a beta feature and might change in the future. See :ref:`custom-discovery` for other methods of discovery. By default ``setuptools`` will consider 2 popular project layouts, each one with its own set of advantages and disadvantages [#layout1]_ [#layout2]_ as discussed in the following sections. Setuptools will automatically scan your project directory looking for these layouts and try to guess the correct values for the :ref:`packages <declarative config>` and :doc:`py_modules </references/keywords>` configuration. .. important:: Automatic discovery will only be enabled if you don't provide any configuration for ``packages`` and ``py_modules``. If at least one of them is explicitly set, automatic discovery will not take place. Note: specifying ``ext_modules`` might also prevent auto-discover from taking place, unless your opt into :doc:`pyproject_config` (which will disable the backward compatible behaviour). .. _src-layout: src-layout ---------- The project should contain a ``src`` directory under the project root and all modules and packages meant for distribution are placed inside this directory:: project_root_directory ├── pyproject.toml # AND/OR setup.cfg, setup.py ├── ... └── src/ └── mypkg/ ├── __init__.py ├── ... ├── module.py ├── subpkg1/ │ ├── __init__.py │ ├── ... │ └── module1.py └── subpkg2/ ├── __init__.py ├── ... └── module2.py This layout is very handy when you wish to use automatic discovery, since you don't have to worry about other Python files or folders in your project root being distributed by mistake. In some circumstances it can be also less error-prone for testing or when using :pep:`420`-style packages. On the other hand you cannot rely on the implicit ``PYTHONPATH=.`` to fire up the Python REPL and play with your package (you will need an `editable install`_ to be able to do that). .. _flat-layout: flat-layout ----------- (also known as "adhoc") The package folder(s) are placed directly under the project root:: project_root_directory ├── pyproject.toml # AND/OR setup.cfg, setup.py ├── ... └── mypkg/ ├── __init__.py ├── ... ├── module.py ├── subpkg1/ │ ├── __init__.py │ ├── ... │ └── module1.py └── subpkg2/ ├── __init__.py ├── ... └── module2.py This layout is very practical for using the REPL, but in some situations it can be more error-prone (e.g. during tests or if you have a bunch of folders or Python files hanging around your project root). To avoid confusion, file and folder names that are used by popular tools (or that correspond to well-known conventions, such as distributing documentation alongside the project code) are automatically filtered out in the case of flat-layout: .. autoattribute:: setuptools.discovery.FlatLayoutPackageFinder.DEFAULT_EXCLUDE .. autoattribute:: setuptools.discovery.FlatLayoutModuleFinder.DEFAULT_EXCLUDE .. warning:: If you are using auto-discovery with flat-layout, ``setuptools`` will refuse to create :term:`distribution archives <Distribution Package>` with multiple top-level packages or modules. This is done to prevent common errors such as accidentally publishing code not meant for distribution (e.g. maintenance-related scripts). Users that purposefully want to create multi-package distributions are advised to use :ref:`custom-discovery` or the ``src-layout``. There is also a handy variation of the flat-layout for utilities/libraries that can be implemented with a single Python file: single-module distribution ^^^^^^^^^^^^^^^^^^^^^^^^^^ A standalone module is placed directly under the project root, instead of inside a package folder:: project_root_directory ├── pyproject.toml # AND/OR setup.cfg, setup.py ├── ... └── single_file_lib.py .. _custom-discovery: Custom discovery ================ If the automatic discovery does not work for you (e.g., you want to include in the distribution top-level packages with reserved names such as ``tasks``, ``example`` or ``docs``, or you want to exclude nested packages that would be otherwise included), you can use the provided tools for package discovery: .. tab:: setup.cfg .. code-block:: ini [options] packages = find: #or packages = find_namespace: .. tab:: setup.py .. code-block:: python from setuptools import find_packages # or from setuptools import find_namespace_packages .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml # ... [tool.setuptools.packages] find = {} # Scanning implicit namespaces is active by default # OR find = {namespaces = false} # Disable implicit namespaces Finding simple packages ----------------------- Let's start with the first tool. ``find:`` (``find_packages()``) takes a source directory and two lists of package name patterns to exclude and include, and then returns a list of ``str`` representing the packages it could find. To use it, consider the following directory:: mypkg ├── pyproject.toml # AND/OR setup.cfg, setup.py └── src ├── pkg1 │ └── __init__.py ├── pkg2 │ └── __init__.py ├── additional │ └── __init__.py └── pkg └── namespace └── __init__.py To have setuptools to automatically include packages found in ``src`` that start with the name ``pkg`` and not ``additional``: .. tab:: setup.cfg .. code-block:: ini [options] packages = find: package_dir = =src [options.packages.find] where = src include = pkg* # alternatively: `exclude = additional` .. note:: ``pkg`` does not contain an ``__init__.py`` file, therefore ``pkg.namespace`` is ignored by ``find:`` (see ``find_namespace:`` below). .. tab:: setup.py .. code-block:: python setup( # ... packages=find_packages( where='src', include=['pkg'], # alternatively: `exclude=['additional']` ), package_dir={"": "src"} # ... ) .. note:: ``pkg`` does not contain an ``__init__.py`` file, therefore ``pkg.namespace`` is ignored by ``find_packages()`` (see ``find_namespace_packages()`` below). .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools.packages.find] where = ["src"] include = ["pkg"] # alternatively: `exclude = ["additional"]` namespaces = false .. note:: When using ``tool.setuptools.packages.find`` in ``pyproject.toml``, setuptools will consider :pep:`implicit namespaces <420>` by default when scanning your project directory. To avoid ``pkg.namespace`` from being added to your package list you can set ``namespaces = false``. This will prevent any folder without an ``__init__.py`` file from being scanned. .. important:: ``include`` and ``exclude`` accept strings representing :mod:`glob` patterns. These patterns should match the full* name of the Python module (as if it was written in an ``import`` statement). For example if you have ``util`` pattern, it will match ``util/__init__.py`` but not ``util/files/__init__.py``. The fact that the parent package is matched by the pattern will not dictate if the submodule will be included or excluded from the distribution. You will need to explicitly add a wildcard (e.g. ``util``) if you want the pattern to also match submodules. .. _Namespace Packages: Finding namespace packages -------------------------- ``setuptools`` provides ``find_namespace:`` (``find_namespace_packages()``) which behaves similarly to ``find:`` but works with namespace packages. Before diving in, it is important to have a good understanding of what :pep:`namespace packages <420>` are. Here is a quick recap. When you have two packages organized as follows: .. code-block:: bash /Users/Desktop/timmins/foo/__init__.py /Library/timmins/bar/__init__.py If both ``Desktop`` and ``Library`` are on your ``PYTHONPATH``, then a namespace package called ``timmins`` will be created automatically for you when you invoke the import mechanism, allowing you to accomplish the following: .. code-block:: pycon >>> import timmins.foo >>> import timmins.bar as if there is only one ``timmins`` on your system. The two packages can then be distributed separately and installed individually without affecting the other one. Now, suppose you decide to package the ``foo`` part for distribution and start by creating a project directory organized as follows:: foo ├── pyproject.toml # AND/OR setup.cfg, setup.py └── src └── timmins └── foo └── __init__.py If you want the ``timmins.foo`` to be automatically included in the distribution, then you will need to specify: .. tab:: setup.cfg .. code-block:: ini [options] package_dir = =src packages = find_namespace: [options.packages.find] where = src ``find:`` won't work because ``timmins`` doesn't contain ``__init__.py`` directly, instead, you have to use ``find_namespace:``. You can think of ``find_namespace:`` as identical to ``find:`` except it would count a directory as a package even if it doesn't contain ``__init__.py`` file directly. .. tab:: setup.py .. code-block:: python setup( # ... packages=find_namespace_packages(where='src'), package_dir={"": "src"} # ... ) When you use ``find_packages()``, all directories without an ``__init__.py`` file will be disconsidered. On the other hand, ``find_namespace_packages()`` will scan all directories. .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools.packages.find] where = ["src"] When using ``tool.setuptools.packages.find`` in ``pyproject.toml``, setuptools will consider :pep:`implicit namespaces <420>` by default when scanning your project directory. After installing the package distribution, ``timmins.foo`` would become available to your interpreter. .. warning:: Please have in mind that ``find_namespace:`` (setup.cfg), ``find_namespace_packages()`` (setup.py) and ``find`` (pyproject.toml) will scan all* folders that you have in your project directory if you use a :ref:`flat-layout`. If used naïvely, this might result in unwanted files being added to your final wheel. For example, with a project directory organized as follows:: foo ├── docs │ └── conf.py ├── timmins │ └── foo │ └── __init__.py └── tests └── tests_foo └── __init__.py final users will end up installing not only ``timmins.foo``, but also ``docs`` and ``tests.tests_foo``. A simple way to fix this is to adopt the aforementioned :ref:`src-layout`, or make sure to properly configure the ``include`` and/or ``exclude`` accordingly. .. tip:: After :ref:`building your package <building>`, you can have a look if all the files are correct (nothing missing or extra), by running the following commands: .. code-block:: bash tar tf dist/.tar.gz unzip -l dist/.whl This requires the ``tar`` and ``unzip`` to be installed in your OS. On Windows you can also use a GUI program such as 7zip_. Legacy Namespace Packages ========================= The fact you can create namespace packages so effortlessly above is credited to `PEP 420 <https://www.python.org/dev/peps/pep-0420/>`_. It used to be more cumbersome to accomplish the same result. Historically, there were two methods to create namespace packages. One is the ``pkg_resources`` style supported by ``setuptools`` and the other one being ``pkgutils`` style offered by ``pkgutils`` module in Python. Both are now considered deprecated despite the fact they still linger in many existing packages. These two differ in many subtle yet significant aspects and you can find out more on `Python packaging user guide <https://packaging.python.org/guides/packaging-namespace-packages/>`_. ``pkg_resource`` style namespace package ---------------------------------------- This is the method ``setuptools`` directly supports. Starting with the same layout, there are two pieces you need to add to it. First, an ``__init__.py`` file directly under your namespace package directory that contains the following: .. code-block:: python __import__("pkg_resources").declare_namespace(__name__) And the ``namespace_packages`` keyword in your ``setup.cfg`` or ``setup.py``: .. tab:: setup.cfg .. code-block:: ini [options] namespace_packages = timmins .. tab:: setup.py .. code-block:: python setup( # ... namespace_packages=['timmins'] ) And your directory should look like this .. code-block:: bash foo ├── pyproject.toml # AND/OR setup.cfg, setup.py └── src └── timmins ├── __init__.py └── foo └── __init__.py Repeat the same for other packages and you can achieve the same result as the previous section. ``pkgutil`` style namespace package ----------------------------------- This method is almost identical to the ``pkg_resource`` except that the ``namespace_packages`` declaration is omitted and the ``__init__.py`` file contains the following: .. code-block:: python __path__ = __import__('pkgutil').extend_path(__path__, __name__) The project layout remains the same and ``pyproject.toml/setup.cfg`` remains the same. ---- .. [#beta] Support for adding build configuration options via the ``[tool.setuptools]`` table in the ``pyproject.toml`` file is still in beta stage. See :doc:`/userguide/pyproject_config`. .. [#layout1] https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure .. [#layout2] https://blog.ionelmc.ro/2017/09/25/rehashing-the-src-layout/ .. _editable install: https://pip.pypa.io/en/stable/cli/pip_install/#editable-installs .. _7zip: https://www.7-zip.org	18.35 KB	2022-11-24 06:08
📄 pyproject_config.rst .. _pyproject.toml config: ----------------------------------------------------- Configuring setuptools using ``pyproject.toml`` files ----------------------------------------------------- .. note:: New in 61.0.0 .. important:: If compatibility with legacy builds or versions of tools that don't support certain packaging standards (e.g. :pep:`517` or :pep:`660`), a simple ``setup.py`` script can be added to your project [#setupcfg-caveats]_ (while keeping the configuration in ``pyproject.toml``): .. code-block:: python from setuptools import setup setup() Starting with :pep:`621`, the Python community selected ``pyproject.toml`` as a standard way of specifying project metadata. ``Setuptools`` has adopted this standard and will use the information contained in this file as an input in the build process. The example below illustrates how to write a ``pyproject.toml`` file that can be used with ``setuptools``. It contains two TOML tables (identified by the ``[table-header]`` syntax): ``build-system`` and ``project``. The ``build-system`` table is used to tell the build frontend (e.g. :pypi:`build` or :pypi:`pip`) to use ``setuptools`` and any other plugins (e.g. ``setuptools-scm``) to build the package. The ``project`` table contains metadata fields as described by :doc:`PyPUG:specifications/declaring-project-metadata` guide. .. _example-pyproject-config: .. code-block:: toml [build-system] requires = ["setuptools", "setuptools-scm"] build-backend = "setuptools.build_meta" [project] name = "my_package" description = "My package description" readme = "README.rst" requires-python = ">=3.7" keywords = ["one", "two"] license = {text = "BSD 3-Clause License"} classifiers = [ "Framework :: Django", "Programming Language :: Python :: 3", ] dependencies = [ "requests", 'importlib-metadata; python_version<"3.8"', ] dynamic = ["version"] [project.optional-dependencies] pdf = ["ReportLab>=1.2", "RXP"] rest = ["docutils>=0.3", "pack ==1.1, ==1.3"] [project.scripts] my-script = "my_package.module:function" # ... other project metadata fields as specified in: # https://packaging.python.org/en/latest/specifications/declaring-project-metadata/ .. _setuptools-table: Setuptools-specific configuration ================================= .. warning:: Support for declaring configurations not standardized by :pep:`621` (i.e. the ``[tool.setuptools]`` table), is still in beta stage and might change in future releases. While the standard ``project`` table in the ``pyproject.toml`` file covers most of the metadata used during the packaging process, there are still some ``setuptools``-specific configurations that can be set by users that require customization. These configurations are completely optional and probably can be skipped when creating simple packages. They are equivalent to the :doc:`/references/keywords` used by the ``setup.py`` file, and can be set via the ``tool.setuptools`` table: ========================= =========================== ========================= Key Value Type (TOML) Notes ========================= =========================== ========================= ``platforms`` array ``zip-safe`` boolean If not specified, ``setuptools`` will try to guess a reasonable default for the package ``eager-resources`` array ``py-modules`` array See tip below ``packages`` array or ``find`` directive See tip below ``package-dir`` table/inline-table Used when explicitly listing ``packages`` ``namespace-packages`` array Deprecated - Use implicit namespaces instead (:pep:`420`) ``package-data`` table/inline-table See :doc:`/userguide/datafiles` ``include-package-data`` boolean ``True`` by default ``exclude-package-data`` table/inline-table ``license-files`` array of glob patterns Provisional - likely to change with :pep:`639` (by default: ``['LICEN[CS]E', 'COPYING', 'NOTICE', 'AUTHORS']``) ``data-files`` table/inline-table Discouraged - check :doc:`/userguide/datafiles` ``script-files`` array Deprecated - equivalent to the ``script`` keyword in ``setup.py`` (should be avoided in favour of ``project.scripts``) ``provides`` array Ignored by pip ``obsoletes`` array Ignored by pip ========================= =========================== ========================= .. note:: The `TOML value types`_ ``array`` and ``table/inline-table`` are roughly equivalent to the Python's :obj:`list` and :obj:`dict` data types, respectively. Please note that some of these configurations are deprecated or at least discouraged, but they are made available to ensure portability. New packages should avoid relying on deprecated/discouraged fields, and existing packages should consider alternatives. .. tip:: When both ``py-modules`` and ``packages`` are left unspecified, ``setuptools`` will attempt to perform :ref:`auto-discovery`, which should cover most popular project directory organization techniques, such as the :ref:`src-layout` and the :ref:`flat-layout`. However if your project does not follow these conventional layouts (e.g. you want to use a ``flat-layout`` but at the same time have custom directories at the root of your project), you might need to use the ``find`` directive [#directives]_ as shown below: .. code-block:: toml [tool.setuptools.packages.find] where = ["src"] # list of folders that contain the packages (["."] by default) include = ["my_package"] # package names should match these glob patterns ([""] by default) exclude = ["my_package.tests"] # exclude packages matching these glob patterns (empty by default) namespaces = false # to disable scanning PEP 420 namespaces (true by default) Note that the glob patterns in the example above need to be matched by the entire* package name. This means that if you specify ``exclude = ["tests"]``, modules like ``tests.my_package.test1`` will still be included in the distribution (to remove them, add a wildcard to the end of the pattern: ``"tests"``). Alternatively, you can explicitly list the packages in modules: .. code-block:: toml [tool.setuptools] packages = ["my_package"] .. _dynamic-pyproject-config: Dynamic Metadata ================ Note that in the first example of this page we use ``dynamic`` to identify which metadata fields are dynamically computed during the build by either ``setuptools`` itself or the plugins installed via ``build-system.requires`` (e.g. ``setuptools-scm`` is capable of deriving the current project version directly from the ``git`` :wiki:`version control` system). Currently the following fields can be listed as dynamic: ``version``, ``classifiers``, ``description``, ``entry-points``, ``scripts``, ``gui-scripts`` and ``readme``. When these fields are expected to be provided by ``setuptools`` a corresponding entry is required in the ``tool.setuptools.dynamic`` table [#entry-points]_. For example: .. code-block:: toml # ... [project] name = "my_package" dynamic = ["version", "readme"] # ... [tool.setuptools.dynamic] version = {attr = "my_package.VERSION"} readme = {file = ["README.rst", "USAGE.rst"]} In the ``dynamic`` table, the ``attr`` directive [#directives]_ will read an attribute from the given module [#attr]_, while ``file`` will read the contents of all given files and concatenate them in a single string. ========================== =================== ================================================================================================= Key Directive Notes ========================== =================== ================================================================================================= ``version`` ``attr``, ``file`` ``readme`` ``file`` ``description`` ``file`` One-line text ``classifiers`` ``file`` Multi-line text with one classifier per line ``entry-points`` ``file`` INI format following :doc:`PyPUG:specifications/entry-points` (``console_scripts`` and ``gui_scripts`` can be included) ``dependencies`` ``file`` ``requirements.txt`` format (``#`` comments and blank lines excluded) BETA* ``optional-dependencies`` ``file`` ``requirements.txt`` format per group (``#`` comments and blank lines excluded) BETA ========================== =================== ================================================================================================= Supporting ``file`` for dependencies is meant for a convenience for packaging applications with possibly strictly versioned dependencies. Library packagers are discouraged from using overly strict (or "locked") dependency versions in their ``dependencies`` and ``optional-dependencies``. Currently, when specifying ``optional-dependencies`` dynamically, all of the groups must be specified dynamically; one can not specify some of them statically and some of them dynamically. Also note that the file format for specifying dependencies resembles a ``requirements.txt`` file, however please keep in mind that all non-comment lines must conform with :pep:`508` (``pip``-specify syntaxes, e.g. ``-c/-r/-e`` flags, are not supported). .. attention:: When using the ``file`` directive, please make sure that all necessary files are included in the ``sdist``. You can do that via ``MANIFEST.in`` or using plugins such as ``setuptools-scm``. Please have a look on :doc:`/userguide/miscellaneous` for more information. ---- .. rubric:: Notes .. [#setupcfg-caveats] ``pip`` may allow editable install only with ``pyproject.toml`` and ``setup.cfg``. However, this behavior may not be consistent over various ``pip`` versions and other packaging-related tools (``setup.py`` is more reliable on those scenarios). .. [#entry-points] Dynamic ``scripts`` and ``gui-scripts`` are a special case. When resolving these metadata keys, ``setuptools`` will look for ``tool.setuptool.dynamic.entry-points``, and use the values of the ``console_scripts`` and ``gui_scripts`` :doc:`entry-point groups <PyPUG:specifications/entry-points>`. .. [#directives] In the context of this document, directives are special TOML values that are interpreted differently by ``setuptools`` (usually triggering an associated function). Most of the times they correspond to a special TOML table (or inline-table) with a single top-level key. For example, you can have the ``{find = {where = ["src"], exclude=["tests*"]}}`` directive for ``tool.setuptools.packages``, or ``{attr = "mymodule.attr"}`` directive for ``tool.setuptools.dynamic.version``. .. [#attr] ``attr`` is meant to be used when the module attribute is statically specified (e.g. as a string, list or tuple). As a rule of thumb, the attribute should be able to be parsed with :func:`ast.literal_eval`, and should not be modified or re-assigned. .. _TOML value types: https://toml.io/en/v1.0.0	11.44 KB	2022-11-24 06:08
📄 quickstart.rst ========== Quickstart ========== Installation ============ You can install the latest version of ``setuptools`` using :pypi:`pip`:: pip install --upgrade setuptools Most of the times, however, you don't have to... Instead, when creating new Python packages, it is recommended to use a command line tool called :pypi:`build`. This tool will automatically download ``setuptools`` and any other build-time dependencies that your project might have. You just need to specify them in a ``pyproject.toml`` file at the root of your package, as indicated in the :ref:`following section <basic-use>`. .. _install-build: You can also :doc:`install build <build:installation>` using :pypi:`pip`:: pip install --upgrade build This will allow you to run the command: ``python -m build``. .. important:: Please note that some operating systems might be equipped with the ``python3`` and ``pip3`` commands instead of ``python`` and ``pip`` (but they should be equivalent). If you don't have ``pip`` or ``pip3`` available in your system, please check out :doc:`pip installation docs <pip:installation>`. Every python package must provide a ``pyproject.toml`` and specify the backend (build system) it wants to use. The distribution can then be generated with whatever tool that provides a ``build sdist``-like functionality. .. _basic-use: Basic Use ========= When creating a Python package, you must provide a ``pyproject.toml`` file containing a ``build-system`` section similar to the example below: .. code-block:: toml [build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta" This section declares what are your build system dependencies, and which library will be used to actually do the packaging. In addition to specifying a build system, you also will need to add some package information such as metadata, contents, dependencies, etc. This can be done in the same ``pyproject.toml`` [#beta]_ file, or in a separated one: ``setup.cfg`` or ``setup.py`` [#setup.py]_. The following example demonstrates a minimum configuration (which assumes the project depends on :pypi:`requests` and :pypi:`importlib-metadata` to be able to run): .. tab:: pyproject.toml .. code-block:: toml [project] name = "mypackage" version = "0.0.1" dependencies = [ "requests", 'importlib-metadata; python_version<"3.8"', ] See :doc:`/userguide/pyproject_config` for more information. .. tab:: setup.cfg .. code-block:: ini [metadata] name = mypackage version = 0.0.1 [options] install_requires = requests importlib-metadata; python_version < "3.8" See :doc:`/userguide/declarative_config` for more information. .. tab:: setup.py [#setup.py]_ .. code-block:: python from setuptools import setup setup( name='mypackage', version='0.0.1', install_requires=[ 'requests', 'importlib-metadata; python_version == "3.8"', ], ) See :doc:`/references/keywords` for more information. Finally, you will need to organize your Python code to make it ready for distributing into something that looks like the following (optional files marked with ``#``):: mypackage ├── pyproject.toml # and/or setup.cfg/setup.py (depending on the configuration method) \| # README.rst or README.md (a nice description of your package) \| # LICENCE (properly chosen license information, e.g. MIT, BSD-3, GPL-3, MPL-2, etc...) └── mypackage ├── __init__.py └── ... (other Python files) With :ref:`build installed in your system <install-build>`, you can then run:: python -m build You now have your distribution ready (e.g. a ``tar.gz`` file and a ``.whl`` file in the ``dist`` directory), which you can :doc:`upload <twine:index>` to PyPI_! Of course, before you release your project to PyPI_, you'll want to add a bit more information to help people find or learn about your project. And maybe your project will have grown by then to include a few dependencies, and perhaps some data files and scripts. In the next few sections, we will walk through the additional but essential information you need to specify to properly package your project. .. TODO: A previous generation of this document included a section called "Python packaging at a glance". This is a nice title, but the content removed because it assumed the reader had familiarity with the history of setuptools and PEP 517. We should take advantage of this nice title and add this section back, but use it to explain important concepts of the ecosystem, such as "sdist", "wheel", "index". It would also be nice if we could have a diagram for that (explaining for example that "wheels" are built from "sdists" not the source tree). .. _setuppy_discouraged: .. admonition:: Info: Using ``setup.py`` :class: seealso Setuptools offers first class support for ``setup.py`` files as a configuration mechanism. It is important to remember, however, that running this file as a script (e.g. ``python setup.py sdist``) is strongly discouraged, and that the majority of the command line interfaces are (or will be) deprecated (e.g. ``python setup.py install``, ``python setup.py bdist_wininst``, ...). We also recommend users to expose as much as possible configuration in a more declarative way via the :doc:`pyproject.toml <pyproject_config>` or :doc:`setup.cfg <declarative_config>`, and keep the ``setup.py`` minimal with only the dynamic parts (or even omit it completely if applicable). See `Why you shouldn't invoke setup.py directly`_ for more background. .. _Why you shouldn't invoke setup.py directly: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html Overview ======== Package discovery ----------------- For projects that follow a simple directory structure, ``setuptools`` should be able to automatically detect all :term:`packages <package>` and :term:`namespaces <namespace>`. However, complex projects might include additional folders and supporting files that not necessarily should be distributed (or that can confuse ``setuptools`` auto discovery algorithm). Therefore, ``setuptools`` provides a convenient way to customize which packages should be distributed and in which directory they should be found, as shown in the example below: .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml # ... [tool.setuptools.packages] find = {} # Scan the project directory with the default parameters # OR [tool.setuptools.packages.find] # All the following settings are optional: where = ["src"] # ["."] by default include = ["mypackage"] # [""] by default exclude = ["mypackage.tests"] # empty by default namespaces = false # true by default .. tab:: setup.cfg .. code-block:: ini [options] packages = find: # OR `find_namespace:` if you want to use namespaces [options.packages.find] # (always `find` even if `find_namespace:` was used before) # This section is optional as well as each of the following options: where=src # . by default include=mypackage # * by default exclude=mypackage.tests* # empty by default .. tab:: setup.py [#setup.py]_ .. code-block:: python from setuptools import find_packages # or find_namespace_packages setup( # ... packages=find_packages( # All keyword arguments below are optional: where='src', # '.' by default include=['mypackage'], # [''] by default exclude=['mypackage.tests'], # empty by default ), # ... ) When you pass the above information, alongside other necessary information, ``setuptools`` walks through the directory specified in ``where`` (defaults to ``.``) and filters the packages it can find following the ``include`` patterns (defaults to ````), then it removes those that match the ``exclude`` patterns (defaults to empty) and returns a list of Python packages. For more details and advanced use, go to :ref:`package_discovery`. .. tip:: Starting with version 61.0.0, setuptools' automatic discovery capabilities have been improved to detect popular project layouts (such as the :ref:`flat-layout` and :ref:`src-layout`) without requiring any special configuration. Check out our :ref:`reference docs <package_discovery>` for more information, but please keep in mind that this functionality is still considered beta* and might change in future releases. Entry points and automatic script creation ------------------------------------------- Setuptools supports automatic creation of scripts upon installation, that run code within your package if you specify them as :doc:`entry points <PyPUG:specifications/entry-points>`. An example of how this feature can be used in ``pip``: it allows you to run commands like ``pip install`` instead of having to type ``python -m pip install``. The following configuration examples show how to accomplish this: .. tab:: pyproject.toml .. code-block:: toml [project.scripts] cli-name = "mypkg.mymodule:some_func" .. tab:: setup.cfg .. code-block:: ini [options.entry_points] console_scripts = cli-name = mypkg.mymodule:some_func .. tab:: setup.py [#setup.py]_ .. code-block:: python setup( # ... entry_points={ 'console_scripts': [ 'cli-name = mypkg.mymodule:some_func', ] } ) When this project is installed, a ``cli-name`` executable will be created. ``cli-name`` will invoke the function ``some_func`` in the ``mypkg/mymodule.py`` file when called by the user. Note that you can also use the ``entry-points`` mechanism to advertise components between installed packages and implement plugin systems. For detailed usage, go to :doc:`entry_point`. Dependency management --------------------- Packages built with ``setuptools`` can specify dependencies to be automatically installed when the package itself is installed. The example below shows how to configure this kind of dependencies: .. tab:: pyproject.toml .. code-block:: toml [project] # ... dependencies = [ "docutils", "requests <= 0.4", ] # ... .. tab:: setup.cfg .. code-block:: ini [options] install_requires = docutils requests <= 0.4 .. tab:: setup.py [#setup.py]_ .. code-block:: python setup( # ... install_requires=["docutils", "requests <= 0.4"], # ... ) Each dependency is represented by a string that can optionally contain version requirements (e.g. one of the operators <, >, <=, >=, == or !=, followed by a version identifier), and/or conditional environment markers, e.g. ``sys_platform == "win32"`` (see :doc:`PyPUG:specifications/version-specifiers` for more information). When your project is installed, all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed. This, of course, is a simplified scenario. You can also specify groups of extra dependencies that are not strictly required by your package to work, but that will provide additional functionalities. For more advanced use, see :doc:`dependency_management`. .. _Including Data Files: Including Data Files -------------------- Setuptools offers three ways to specify data files to be included in your packages. For the simplest use, you can simply use the ``include_package_data`` keyword: .. tab:: pyproject.toml (BETA) [#beta]_ .. code-block:: toml [tool.setuptools] include-package-data = true # This is already the default behaviour if your are using # pyproject.toml to configure your build. # You can deactivate that with `include-package-data = false` .. tab:: setup.cfg .. code-block:: ini [options] include_package_data = True .. tab:: setup.py [#setup.py]_ .. code-block:: python setup( # ... include_package_data=True, # ... ) This tells setuptools to install any data files it finds in your packages. The data files must be specified via the \|MANIFEST.in\|_ file or automatically added by a :ref:`Revision Control System plugin <Adding Support for Revision Control Systems>`. For more details, see :doc:`datafiles`. Development mode ---------------- ``setuptools`` allows you to install a package without copying any files to your interpreter directory (e.g. the ``site-packages`` directory). This allows you to modify your source code and have the changes take effect without you having to rebuild and reinstall. Here's how to do it:: pip install --editable . See :doc:`development_mode` for more information. .. tip:: Prior to :ref:`pip v21.1 <pip:v21-1>`, a ``setup.py`` script was required to be compatible with development mode. With late versions of pip, projects without ``setup.py`` may be installed in this mode. If you have a version of ``pip`` older than v21.1 or is using a different packaging-related tool that does not support :pep:`660`, you might need to keep a ``setup.py`` file in file in your repository if you want to use editable installs. A simple script will suffice, for example: .. code-block:: python from setuptools import setup setup() You can still keep all the configuration in :doc:`pyproject.toml </userguide/pyproject_config>` and/or :doc:`setup.cfg </userguide/declarative_config>` Uploading your package to PyPI ------------------------------ After generating the distribution files, the next step would be to upload your distribution so others can use it. This functionality is provided by :pypi:`twine` and is documented in the :doc:`Python packaging tutorial <PyPUG:tutorials/packaging-projects>`. Transitioning from ``setup.py`` to ``setup.cfg`` ------------------------------------------------ To avoid executing arbitrary scripts and boilerplate code, we are transitioning into a full-fledged ``setup.cfg`` to declare your package information instead of running ``setup()``. This inevitably brings challenges due to a different syntax. :doc:`Here </userguide/declarative_config>` we provide a quick guide to understanding how ``setup.cfg`` is parsed by ``setuptools`` to ease the pain of transition. .. _packaging-resources: Resources on Python packaging ============================= Packaging in Python can be hard and is constantly evolving. `Python Packaging User Guide <https://packaging.python.org>`_ has tutorials and up-to-date references that can help you when it is time to distribute your work. .. \|MANIFEST.in\| replace:: ``MANIFEST.in`` .. _MANIFEST.in: https://packaging.python.org/en/latest/guides/using-manifest-in/ ---- .. rubric:: Notes .. [#setup.py] New projects are advised to avoid ``setup.py`` configurations (beyond the minimal stub) when custom scripting during the build is not necessary. Examples are kept in this document to help people interested in maintaining or contributing to existing packages that use ``setup.py``. Note that you can still keep most of configuration declarative in :doc:`setup.cfg <declarative_config>` or :doc:`pyproject.toml <pyproject_config>` and use ``setup.py`` only for the parts not supported in those files (e.g. C extensions). See :ref:`note <setuppy_discouraged>`. .. [#beta] Support for adding build configuration options via the ``[tool.setuptools]`` table in the ``pyproject.toml`` file is still in beta stage. See :doc:`/userguide/pyproject_config`. .. _PyPI: https://pypi.org	15.79 KB	2022-11-24 06:08

Terminal