OpenStack-Ansible python_venv_build

此 Ansible 角色为 OpenStack-Ansible 项目准备一个 python venv,但也可用于其他项目。

该角色要求在执行之前存在以下条件

  • virtualenv >= 1.10(以支持使用 never-download 选项)

  • pip >= 7.1(以支持使用 constraints 选项)在创建虚拟环境后。

用例

此角色构建用于以下用例

  1. 使用构建主机(又名 repo 服务器)

    • 在 repo 服务器上构建 python wheels,并提供给定的 python 包列表。

    • 在 repo 服务器上准备 requirements.txt 和 constraints.txt 文件,并使用它们来确保构建和安装过程都一致且幂等。

    • 在构建主机上,安装构建时所需的发行包。

    • 在任意数量的目标主机上,创建一个 virtualenv,并使用 pip 的 --find-links 选项将这些构建的 wheels 安装到其中。

    • 在任意数量的目标主机上,安装运行时所需的发行包。

    • 重用先前构建的 wheels 以加速任何后续构建。

  2. 不使用构建主机

    • 在任意数量的目标主机上,创建一个 virtualenv,然后本地安装构建和运行时所需的发行包,然后本地编译和安装给定的 python 包列表。

    • 这消除了对 repo 服务器的需求,但由于需要安装的依赖项数量增加以及编译发生在每个目标主机上,因此需要更长的时间。

    • 只有当只有一个目标主机(没有容器),并且安装到 venv 中的任何包都不会在同一主机上由此角色构建的任何其他 venv 再次使用时,构建主机才没有好处。

您可能需要查看 Python Build/Install Process Simplification 规范,以了解导致创建此角色的背景。

流程

  1. 检查先决条件。

  2. 如果启用了 wheel 构建,并且环境中存在 repo 服务器,则在 repo 服务器上执行以下操作

    1. 安装执行 python wheel 编译所需的发行包。

    2. 为 pip 编译一组 requirements 和 source-constraints,用于构建 wheels。这些也用于确定两者是否发生更改,以实现幂等性。

    3. 编译 python wheels,并创建一个安装时 constraints 文件。安装时 constraints 文件包含带有其版本的 python 包列表 - 这与 source-constraints 不同,后者可能包含 git SHA。

  3. 然后,在目标主机上开始安装 python 包

    1. 如果启用了 wheel 构建

      1. 仅安装 python 包运行时所需的发行包。

      2. venv_install_destination_path 创建一个 python venv。

      3. 在 venv 路径中准备 venv 的 requirements 和 constraints 文件。

      4. 使用 pip 的 --find-links 选项从 repo 服务器上的 wheels 安装 python 包,以确保它们优先于默认的 pypi 索引。

      5. 如果有任何 venv_packages_to_symlink,则将从这些包安装到系统中的适当 python 库链接到 virtualenv。这为与 C 绑定紧密耦合的 python 库提供支持,这些库可能无法作为 wheel 移植。

    2. 如果未启用 wheel 构建

      1. 安装编译和 python 包运行时所需的发行包。

      2. venv_install_destination_path 创建一个 python venv。

      3. 在 venv 路径中准备 venv 的 requirements 和 constraints 文件。在这种情况下,constraints 文件将包含与 repo 服务器上的 source-constraints 文件相同的内容(如果存在)。

      4. 从默认的 pip 索引安装 python 包。在安装过程中,pip 将对任何具有 git SHA 作为 constraints 的包进行 git clone 并进行构建。

      5. 如果有任何 venv_packages_to_symlink,则将从这些包安装到系统中的适当 python 库链接到 virtualenv。这为与 C 绑定紧密耦合的 python 库提供支持,这些库可能无法作为 wheel 移植。

  4. 如果设置了任何 venv_facts_when_changed,则在目标主机上的 /etc/ansible/facts.d 中实现它们。

默认变量

#
# Required variables
#

# The path where venvs are extracted to
# on the target host during an install, for example:
# venv_install_destination_path: "/openstack/venvs/myvenv"

#
# Optional variables
#

# Select the tooling which will be used for package installation
# Valid options: pip, uv
venv_install_tool: pip

# Distribution packages which must be installed
# on all hosts when building python wheels.
venv_build_base_distro_package_list: "{{ _venv_build_base_distro_package_list }}"
venv_install_base_distro_package_list: "{{ _venv_install_base_distro_package_list }}"

# Distribution packages which must be installed
# on the host for the purpose of building the
# python wheels.
venv_build_distro_package_list: []

# Distribution packages which must be installed
# on the host when installing the venv.
venv_install_distro_package_list: []

# Set the package install state for packages
# Options are 'present' and 'latest'
venv_distro_package_state: "latest"
venv_pip_package_state: "latest"

# The time in seconds that the distribution package
# cache is valid for. This is only used by the apt
# package manager.
venv_distro_cache_valid_time: 600

# Default python packages which will be installed
# into every venv.
venv_default_pip_packages: []

# Python packages which must be installed
# into the venv.
venv_pip_packages: []

# Don't use the site-wide PIP configuration file when
# upgrading PIP (some operating systems have issued
# with upgrades w/ extra-index-urls)
# ref: https://github.com/pypa/pip/issues/4195
venv_pip_upgrade_noconf: false

# A list of constraints to be applied when building
# or installing python packages.
venv_build_constraints: []

# A list of pip constraints to be applied as global
# constraints ahead of the list in venv_build_constraints.
# This is useful for global pins across all venvs.
venv_build_global_constraints: []

# Arguments to pass to pip when building the wheels
venv_pip_build_args: ""

# Environemnt to be set for building the wheels
venv_pip_build_env: {}

# Default arguments to pass to pip when installing into
# the venv.
venv_default_pip_install_args: >-
  {%- if (groups[venv_build_group] is defined) and (groups[venv_build_group] | length > 0) %}
  {%- if (venv_wheel_build_enable | bool) %}
  --find-links {{ openstack_repo_url | default('https://') }}/os-releases/{{ openstack_release | default('master') }}/{{ _venv_build_dist_arch }}/wheels/
  {%- endif %}
  --trusted-host {{ (openstack_repo_url | default('https://')) | urlsplit('hostname') -}}
  {%- endif %}

# Arguments to pass to pip when installing into the venv
venv_pip_install_args: ""

# Environemnt to be set for building the wheels
venv_pip_install_env:
  REQUESTS_CA_BUNDLE: "{{ _venv_install_ca_bundle_path }}"

# Some python packages have C bindings which tend to be very
# particular about the version of their underlying shared libraries.
# To ensure things run smoothly for stable releases, we opt to
# use the distro packages for these python packages and symlink the
# appropriate python library files and their bindings into the venv.
# This variable should contain the list of packages installed which
# should be symlinked into the venv.
venv_packages_to_symlink: []

# The python executable to use for creating the venv
venv_python_executable: "python3"

# Enable the recreation of the venv from scratch.
# This is useful if you think the venv may be corrupted
# or if you have changed options which means that packages
# should be removed from the venv.
# Under normal circumstances, the installs will be done
# into the existing venv over the top of any previously
# installed packages.
venv_rebuild: false

# Enable the recreation of the pre-built wheels.
# This is useful if wheels build has been interrupted or
# previously failed because of connectivity or other issues.
venv_wheels_rebuild: false

# Enable/disable the build of python wheels
# If the package concerned is built from a tarball, rather
# than from a git source or pypi, then this may be best to
# set to false.
venv_wheel_build_enable: "{{ (venv_build_group in groups and groups[venv_build_group] | length > 0) }}"

# Set the name of the ansible group, that will be used
# to identify suitable host to build wheels on
venv_build_group: repo_all

# Set the host where the wheels will be built.
# If this host is not the same as the target host, then
# python wheels will be built in order to speed up the
# subsequent venv builds on this host and others. When
# this is the same as the target host, then we will not
# bother building wheels.
venv_build_host: "{{ venv_build_targets[_venv_build_dist_arch] }}"

# The owner of directories and files held on the build host.
# venv_build_host_user_name: "root"
# venv_build_host_group_name: "root"

# The path for the wheel build venv.
# This is the path where a venv will be created on the
# build host for the purpose of building the wheels.
venv_build_host_venv_path: "/openstack/venvs/wheel-builder-{{ venv_python_executable }}"

# The path where the requirements/constraints are stored
# on the build host in order to ensure the build process
# is idempotent.
venv_build_host_requirements_path: >-
  /var/www/repo/os-releases/{{ openstack_release | default('master') }}/{{ _venv_build_dist_arch }}/requirements

# The path where the wheels are cached on the build host
# for speeding up the build process.
venv_build_host_wheel_path: >-
  /var/www/repo/os-releases/{{ openstack_release | default('master') }}/{{ _venv_build_dist_arch }}/wheels

# The facts to set when the venv changes during a
# build, or the installation of a venv.
# Eg:
# set_facts_when_changed:
#   - section: glance
#     option: venv_tag
#     value: "{{ glance_venv_tag }}"
venv_facts_when_changed: []

# The INI file name to use for the fact setting.
venv_facts_dest: "openstack_ansible"

参考

可以在以下位置找到该项目的文档

https://docs.openstack.org/ansible-role-python_venv_build/2025.2/

项目主页位于

https://launchpad.net/openstack-ansible

可以在以下位置找到该项目的发行说明

https://docs.openstack.org/releasenotes/ansible-role-python_venv_build/

项目源代码仓库位于:

https://git.openstack.org/cgit/openstack/ansible-role-python_venv_build

可以在以下位置找到 bug 跟踪器

https://bugs.launchpad.net/openstack-ansible