Skip to content
Install

The warm Python path

How Volant runs ansible-core's Python modules without starting an interpreter per task.

Most Ansible modules are Python programs. Volant runs them unchanged, but it does not start a new interpreter for each task. For the most common modules it often does not run Python at all: the agent has native versions that answer when a task’s arguments allow it.

flowchart TB
  H[Python helper with ansible-core] --> B[union blob: one zip per run, named by its hash]
  B -- sent once, cached, hash verified --> AG[agent on the host]
  AG --> S[warm Python server, module_utils loaded once]
  S -- fork --> T1[module A]
  S -- fork --> T2[module B]
  class B,S key
  1. The controller asks a helper process to build ansible-core’s payload for every module the run needs, unless the cache already holds the result.
  2. Volant merges them into one zip, the union blob, with every module_utils file they import. The blob is named by the BLAKE3 hash of its contents.
  3. The blob is sent to each host once and cached there. The agent checks the hash before using it, and a host that already has it receives nothing.
  4. The agent starts a Python server as soon as the controller has sent the payload, while the tasks before the first one that needs Python still run. It keeps one server for each interpreter, payload and the part of the task environment an interpreter reads as it starts (PYTHON*, LD_*, the locale, TZ, HOME and the temporary directory); the child sets the rest before the module runs. Each target user has an agent of its own. The server loads the shared code from the blob once, then forks a child for each task. The child runs the module and writes its JSON result back over a pipe. While the child runs, the server imports that module into itself, so the next child of the same module starts with it already loaded.

The union is built before the first connection, so it has to hold every module a host might reach. It carries the modules the plays name, those of every task and handler file of the roles in play (their meta/main.yml dependencies included), and those of any file an include names literally. A module named only in a file for another platform, such as setup-RedHat.yml next to setup-Debian.yml, is in it too, so an include_tasks: setup-{{ ansible_os_family }}.yml finds its module whichever file a host picks. A name there that no module answers to is left out, and fails by name only if a host reaches it.

The blob travels as raw bytes, compressed with deflate when that is shorter, and the agent checks the hash of the decompressed bytes.

Module results are untrusted and are never rendered as controller templates. See Trusted and untrusted values. As ansible-core does on the controller, Volant adds stdout_lines and stderr_lines to a result that has stdout or stderr without them, split with Python’s splitlines() rules.

The controller needs Python 3 with ansible-core. When VOLANT_PYTHON is set, Volant uses it and nothing else. Otherwise it tries its managed runtime, then the active virtual environment, then python3 on your PATH. If none of them has ansible-core, it installs the managed runtime.

The host needs a Python 3 interpreter, but not ansible-core: the blob carries the module and every file it imports. The agent reports the interpreters it finds when it starts.

These numbers come from an Ubuntu 24.04 host with Python 3.12, over SSH. Each figure is a median.

Cost of running one moduleUbuntu 24.04 host, Python 3.12, over SSH, median. Shorter is faster.
  • Ansible, default settings765 msAnsible, default settings765 ms
  • Ansible, pipelining469 msAnsible, pipelining469 ms
  • Payload as a plain process340 msPayload as a plain process340 ms
  • Fork, nothing preloaded266 msFork, nothing preloaded266 ms
  • Fork, module_utils loaded12.8 msFork, module_utils loaded12.8 msWhat Volant does
Show as a table
Cost of running one module
Ansible, default settings765 ms
Ansible, pipelining469 ms
Payload as a plain process340 ms
Fork, nothing preloaded266 ms
Fork, module_utils loaded12.8 ms

The cost per module follows the work the module does. apt takes 755 ms, mostly in dpkg, where ping takes 12.8 ms: the warm path removes overhead, not work.

Sending the modules as one archive saves 72.7 percent of the bytes, because the files they share travel once. For ping, stat, file, lineinfile and apt, that is 631,050 bytes against 2,307,982 for one zip per module. Loading the archive costs 264.5 ms, once per server.

Decision record 0006 has the reasoning and a measurement on a full playbook.

A module from an installed collection, such as ansible.posix.sysctl or community.general.ufw, runs on the warm Python path like a builtin one.

Before the first connection, Volant asks the controller’s ansible-core what each dotted name is. ansible-core looks in the collection paths it is configured with, such as ANSIBLE_COLLECTIONS_PATH, and then in the collections/ directory of the managed runtime, where volant runtime install --collections puts them. Volant installs nothing during a run, and nothing is fetched to answer.

A name that is not one of ansible-core’s own modules gets one of five answers. This includes an ansible.builtin. name that ansible-core’s ansible_builtin_runtime.yml routes to a collection:

What ansible-core finds What Volant does
a module it can build adds it to the union and runs it
a module its collection runs through an action plugin, directly or through runtime.yml routing runs the action plugin in the Python helper, as ansible-core’s own
an action plugin that needs a Windows host or a network connection stops the run: the module needs an action plugin from its collection, which this release does not run
no module by that name stops the run with the reference’s sentence
a module it cannot run: removed, PowerShell only, or failing to build stops the run, with the reason

In the union, a builtin keeps its short name and a collection module is keyed by its full name. Two collections that each ship a module called sysctl stay apart, and community.general.command is never mistaken for command.

A collection module’s arguments written as a string (ansible.posix.sysctl: name=net.ipv4.ip_forward value=1) are split into key=value pairs the way ansible-core’s parse_kv splits them.

The refusals come before the first connection, with exit code 4:

task 'Forward': couldn't resolve module/action 'ns.coll.mod'. This often indicates a misspelling, missing collection, or incorrect module path. The collection 'ns.coll' is not installed on the controller: install it with volant runtime install --collections ns.coll
task 'Forward': module 'ns.coll.mod' cannot run: <reason>

The install hint appears only when the collection itself is missing. A module missing from a collection that is installed gets the first sentence alone. ansible.builtin.nosuch also gets it alone: ansible-core’s routing decides what such a name becomes, and there is no collection to install. Without an ansible-core on the controller, a name that is not one of ansible-core’s own modules is refused with the same first sentence and the reason nothing could look it up.

A collection module named only in a role’s file for another platform is resolved too. If it can be built, it joins the union. Anything else is left out without stopping the run, and the include that reaches it fails for that host before any of the file’s tasks run:

task 'Forward': module 'ansible.posix.sysctl' was not resolved to a module before the run, so no payload holds it: its collection is not installed on the controller, no module path holds it, or it is named only in a file nothing read before the first connection

Some ansible-core modules only work together with an action plugin that runs on the controller first. package picks the host’s package manager, service picks its init system, and template renders the file on the controller before anything is sent. Sending those modules without their plugin would do something different from what the playbook asked.

Volant has its own version of eight of these plugins: copy, dnf, fetch, package, reboot, service, template and unarchive run on the controller as a short sequence of sub-tasks, each an ordinary Python module or a bare command sent over the host’s existing connection. Action plugins describes each one. The other plugins, such as the ones of uri and script, run as ansible-core’s own Python in the helper.

assert, fail and pause have action plugins too, but none of them calls a module, so Volant runs them on the controller. setup runs as an ordinary module, natively or in Python, so fact gathering works.

Building the union costs a Python start, an ansible-core import and a build of every module. ansible-core reads every module_utils file a module imports, and the modules of one run share most of them, so the helper reads each file once for the whole union: 1.5 to 1.8 seconds on the four Galaxy roles of the performance page, against 3.7 to 4.4 seconds when each module read them again. The controller keeps the result under $XDG_CACHE_HOME/volant/unions/, or ~/.cache/volant/unions/, in a directory private to your account.

An entry is looked up by everything the build read that is not a file: the Volant version, the interpreter, the modules and collections asked for, the ANSIBLE_* and PYTHON* environment, the ansible.cfg ansible-core would read, and the working directory. It also lists every source file the helper read, with its size and modification time. A run reuses the entry only if each of those files is unchanged and the zip still has its recorded hash. Upgrading ansible-core or a collection in place therefore rebuilds the union.

A cache that cannot be read, is corrupt or cannot be written never fails a run: the union is rebuilt, and a directory that cannot be written prints one warning per run. The cache holds up to 1 GiB. Past that, the entries no run has used for longest go first, and an entry no run has used for 30 days is removed. To clear it, remove the directory.

The agent carries Rust versions of these modules:

Module Answered natively when
setup on Debian and Ubuntu, for the facts listed under Facts
stat the path is plain and absolute, and the checksum is SHA-1
file state is file, directory or absent, or unset, and link when nothing is at the path yet, or the link is already there and points at src
apt packages named exactly, without a version: a fresh cache and packages already installed or absent are only read; a stale cache is updated and missing packages are installed with the same apt-get commands the module runs, under policy_rc_d when the task sets it; a package marked automatically installed is marked manual, as the module marks it
systemd, systemd_service name, state, enabled and daemon_reload on the system scope
lineinfile the regular expression means the same in Rust as in Python, and backrefs is off
copy the module that the copy action plugin runs, for a file staged from the controller
package_facts the manager is apt and every package is in a settled state
service_facts the host runs systemd
user, group on Debian and Ubuntu, for the common arguments

Anything outside that goes to ansible-core’s Python module. A task environment, SELinux, extended attributes, remote_src, a regular expression only Python understands, or a package half installed all hand back, among other cases. dnf and every other module always run in Python.

A native module receives the arguments the Python module would have received. It first decides, by reading only, whether it can give the answer the reference would give. If it cannot, it hands the task back before touching the host, and the Python module does the real work. Its result, invocation included, is the one the reference module returns, key for key, and a test compares them for every case the module covers. The result comes back as untrusted data, through the same path as a Python module’s.

On a host whose configuration turns apt’s package cache off, as Docker’s Debian and Ubuntu images do, apt builds it again from the package lists on every call: 1.3 seconds each on Ubuntu 24.04. The native apt then points its apt-get, apt-cache and apt-mark calls at a cache in the connection’s own directory, removed when the connection ends. What they print and what they leave on the host do not change.

Set [volant] native_modules = false, or VOLANT_NATIVE_MODULES=0, to send every task to Python. --profile shows which path each task took and, for a hand-back, the reason. See Command line.