Text on lists: building a systemd ExecStart line with regex_replace, join and replace

ansible - 2026-10-03

Sixteenth entry in the Shaping data in Ansible series. Part 7 turned text into data. This one goes the other way: it turns a list of options into one line of text, the command a systemd service runs. systemd starts and supervises services on most Linux systems; each service is described by a unit file, and its ExecStart= line is the command to run. Building that line from variables takes three things: string filters applied to every item of a list, join to glue them, and escaping, because systemd reads ExecStart= with rules of its own. Everything below ran with ansible-core 2.21.4, and every systemd result came from GitHub's Ubuntu 24.04 runners, which ship systemd 255, in the example's CI.

The unit and its options

The unit backs up a PostgreSQL database with pg_dump. It's a template unit, pg-backup@.service: starting pg-backup@app.service runs it with app as its instance, which the unit reads as %i. %i is a specifier, one of the % sequences systemd replaces when it loads the unit.

The options, in vars/backup.yml:

pg_backup_options:          # options that take a value
  format: custom
  compress: 6
pg_backup_flags:            # options without one
  - no_owner
pg_backup_exclude_tables:   # tables left out of the dump
  - audit log
  - tmp$work
  - cache_50%
  - report_%n
  - tmp_${work}

The table names are all legal in PostgreSQL when quoted, and each has a character systemd treats specially: a space, a $, a %.

A filter on every item: map('regex_replace')

map applies a filter to each item of a list (part 4). With regex_replace, which replaces what a regular expression matches, it turns names into options:

option_args: "{{ pg_backup_options | items | map('join', '=') | map('regex_replace', '^', '--') | list }}"
flag_args: "{{ pg_backup_flags | map('regex_replace', '_', '-') | map('regex_replace', '^', '--') | list }}"
table_args: "{{ pg_backup_exclude_tables | map('regex_replace', '^', '--exclude-table=') | list }}"

• items turns the dict into [key, value] pairs, and map('join', '=') makes each one format=custom.
• regex_replace('^', '--') adds a prefix: ^ matches the start of the string, an empty match, so the replacement is inserted there.
• regex_replace('_', '-') turns no_owner into no-owner, pg_dump's spelling.

That gave ['--format=custom', '--compress=6'], ['--no-owner'], and one --exclude-table=… per table.

format can't be mapped

format looks like the natural way to add a prefix: '--exclude-table=%s' | format('audit log') gave --exclude-table=audit log, and '--exclude-table=' ~ 'audit log' the same with ~, Jinja's concatenation operator. But neither can be applied to every item with map. map passes each item as the filter's input, and for format the input is the format string. pg_backup_exclude_tables | map('format') treated each table name as a format, and failed on the third: "incomplete format", from the % in cache_50%. regex_replace('^', prefix) is the way to add a prefix to every item.

join, and what systemd does with the line

join(' ') glues the list into one string:

ExecStart=/usr/bin/pg_dump --format=custom --compress=6 --no-owner --exclude-table=audit log
  --exclude-table=tmp$work --exclude-table=cache_50% --exclude-table=report_%n
  --exclude-table=tmp_${work} --file=/srv/backups/%i.dump %i

(one line in the unit file). systemd-analyze verify, which checks unit files, accepted it. To see what the program actually receives, the example's CI starts the unit with ExecStart= pointing at a small script that prints its arguments. For pg-backup-naive@app.service:

And --file=/srv/backups/%i.dump %i arrived as --file=/srv/backups/app.dump and app, as intended. The systemd.service manual, Command lines, explains each row:

• Whitespace separates arguments, unless quoted (the rules are in systemd.syntax). pg_dump would have excluded a table named audit, which doesn't exist, and received log as an extra argument.
• $FOO is replaced only "as a separate word", so tmp$work passed through. ${FOO} is replaced "as part of a word, or as a word of its own", and "variables whose value is not known at expansion time are treated as empty strings": tmp_${work} lost its end.
• Specifiers are replaced anywhere: %n is the "full unit name". % followed by a space isn't a specifier, and systemd 255 left cache_50% alone, although systemd.unit says specifiers "must be known and resolvable for the setting to be valid". Don't count on it.

Escaping the data for systemd

The same manuals give the escapes: "to pass a literal dollar sign, use $$", and %% for "a single percent sign". Double quotes keep a space inside one argument. Applied to the table arguments only:

table_args_escaped: >-
  {{ table_args | map('replace', '%', '%%') | map('replace', '$', '$$')
  | map('regex_replace', '^(.*)$', '"\1"') | list }}

replace replaces a fixed string, with no regular expression. regex_replace('^(.*)$', '"\1"') wraps each argument in quotes: (.*) captures the whole argument, and \1 puts it back. The escaped line, … "--exclude-table=audit log" "--exclude-table=tmp$$work" "--exclude-table=cache_50%%" "--exclude-table=report_%%n" "--exclude-table=tmp_$${work}" --file=/srv/backups/%i.dump %i, delivered all five names exactly as written.

Escape the data, not the line. --file=/srv/backups/%i.dump %i is written by hand and must keep its %i; escaping the whole line would turn it into %%i, a literal %i. That's why the example escapes table_args before joining, not the result of join. These escapes cover the five names here; a name containing a " or a \ would need those escaped too.

How many backslashes: count for YAML, not for Jinja

The \1 above has one backslash, because the expression sits in a YAML block scalar (>-), where YAML changes nothing. Ansible then passes the Jinja string through with its backslashes unchanged; plain Jinja would have turned '\\1' into \1, but in Ansible it reaches the regular expression as \\1, a literal backslash followed by 1. Tested on 'x y' | regex_replace('(x)', '[…1]'), with ansible-core 2.21.4 and 2.18.19 alike:

Only YAML's quoting counts: in double quotes, write each backslash twice; anywhere else, once. The Ansible docs' examples, Searching strings with regular expressions, are in double quotes, which is why they show \\1.

Which one

• A filter on every item of a list: map('regex_replace', …) or map('replace', …); to add a prefix, regex_replace('^', prefix).
• One string from one value: format or ~, but never map('format'): the item becomes the format string.
• Many arguments into one line: join(' '), after escaping each argument for whatever reads the line.
• An ExecStart= line from data: quote each argument, double % and $ in the data, and leave hand-written specifiers like %i alone. Check the result by running the unit, not with systemd-analyze verify.
• Backslashes in a regex: one in a block scalar or single quotes, two in double quotes.

The example repository

The series' companion repository, abdelhousni/ansible-data-shaping-series, builds both lines on the local machine, writing the two units to its out/ directory, and its CI compares the output with the expected one. A separate CI job, ci/systemd-check.sh, installs both units on a runner with systemd, starts them for the instance app, and compares the arguments the program received with ci/expected-systemd.txt.

Sources

• ansible-core 2.21.4: plugins/filter/core.py (regex_replace is Python's re.subn), and Jinja 3.1.6's items, join, replace and format filters.
• Ansible docs, from ansible/ansible-documentation: playbook_guide/playbooks_filters.rst, Manipulating strings and Searching strings with regular expressions.
• systemd 255 manuals, from systemd/systemd: systemd.service.xml (Command lines: variables, $$), systemd.unit.xml and standard-specifiers.xml (Specifiers: %i, %n, %%).
• The systemd results came from GitHub's ubuntu-24.04 runners (Ubuntu 24.04 ships systemd 255), on 2026-10-03; the Ansible results from ansible-core 2.21.4 on the local machine, the backslash table also from ansible-core 2.18.19. No PostgreSQL server was backed up.

Created 2026-10-03T03:41:37+02:00 · Edit