Prevent duplicate key warnings in Ansible 2

Question

I use a lot of YAML anchors and references in my roles to keep the logic in a single spot instead of repeating myself in multiple tasks. Following is a very very basic example.

- &sometask
  name: "Some Task"
  some_module: with a lot of parameters
  with_items: list_A

- <<: *sometask
  name: "Some OTHER Task"
  with_items: list_B

_{^{This example might not show how this is actually useful, but it is. Imagine you loop over a list of dicts, passing various keys from each dict to the module, maybe having quite complex "when", "failed_when" and "changed_when" conditions. You simply want to DRY.}}

So instead of defining the whole task twice, I use an anchor to the first one and merge all its content into a new task, then override the differing pieces. That works fine.

Just to be clear, this is basic YAML functionality and has nothing to do with Ansible itself.

The result of above definition (and what Ansible sees when it parsed the YAML file) would evaluate to:

- name: "Some Task"
  some_module: with a lot of parameters
  with_items: list_A

- name: "Some Task"
  some_module: with a lot of parameters
  with_items: list_A
  name: "Some OTHER Task"
  with_items: list_B

Ansible 2 now has a feature to complain when keys have been defined multiple times in a task. It still works, but creates unwanted noise when running the playbook:

TASK [Some OTHER Task] *******************************************************
 [WARNING]: While constructing a mapping from /some/file.yml, line 42, column 3, found a duplicate dict key (name).  Using last defined value only.

 [WARNING]: While constructing a mapping from /some/file.yml, line 42, column 3, found a duplicate dict key (with_items).  Using last defined value only.

Ansible configuration allows to prevent deprecation_warnings and command_warnings. Is there a way to also prevent this kind of warning?

Solution 1

As of Ansible 2.9.0, this can be achieved by setting the ANSIBLE_DUPLICATE_YAML_DICT_KEY environment variable to ignore. Other possible values for this variable are warn, which is the default and preserves the original behaviour, and error, which makes the playbook execution fail.

See this pull request for details about the implementation.

Solution 2

Coming in late here I'm going to disagree with the other answers and endorse YAML merge. Playbook layout is highly subjective and what's best for you depends on the config you need to describe.

Yes, ansible has merge-like functionality with includes or with_items / with_dict loops.

The use case I've found for YAML merge is where tasks have only a few outliers, therefore a default value which can be overridden is the most compact and readable representation. Having ansible complain about perfectly valid syntax is frustrating.

The comment in the relevant ansible code suggests The Devs Know Better than the users.

Most of this is from yaml.constructor.SafeConstructor. We replicate it here so that we can warn users when they have duplicate dict keys (pyyaml silently allows overwriting keys)

PyYAML silently allows "overwriting" keys because key precedence is explicitly dealt with in the YAML standard.