How to remove the line breaker character '\n' from the result of lookup() module in Ansible?

Question

I am using [file lookup] which reads the whole file and stores the content in a variable. My play looks something like this:

  - name: Store foo.xml contents in a variable
    set_fact:
     foo_content: "{{ lookup('file', 'foo.xml' ) | replace('\n', '')}}"

So the above code reads the foo.xml file and stores it in the variable, but the problem is when the foo.xml has line breaks in it, it also includes the line break in the variable.

My foo.xml is this file:

<?xml version="1.0" encoding="utf-8"?>
<initialize_param>
    <secrets>
        <my_secret id="99">3VMjII6Hw+pd1zHV5THSI712y421USUS8124487128745812sajfhsakjfasbfvcasvnjasjkvbhasdfasgfsfaj5G8A9+n8CkLxk7Dqu0G8Jclg0eb1A5xeFzR3rrJHrb2GBBa7PJNVx8tFJP3AtF6ek/F/WvlBIs2leX2fq+/bGryKlySuFmbcwBsThmPJC5Z5AwPJgGZx</my_secret>
    </secrets>
</initialize_param>

The output removes line break \n but also incudes the tabs \r & \t

I need to got rid of the \n , need to get rid of extra formatting too (\r & \t), Moreover after the replace filter I get the error while firing a DB Update query as

stderr: /bin/sh: 1: cannot open ?xml: No such file

Solution 1

Use the Jinja trim filter:

"{{ lookup('file', 'foo.xml' ) | trim }}"

Solution 2

You can do that with the replace filter?

contents: "{{ lookup('file', '/etc/foo.txt') | replace('\n', '')}}"

Solution 3

You may use the regex_replace filter since the trim doesn't clear other line break characters as you mentioned in the question.

"{{  some_stdout_to_clear | regex_replace('[\\r\\n\\t]+','') }}"

Comments

[Mikko Ohtamaa]

My guess is that \n is just artifact of debug output and it is correctly handled there.

[Nishant Singh]

nope, it actually stores it as \n and then when i push the variable in a database it throws a error :

[Mikko Ohtamaa]

Can you please clarify what kind of error and how you are pushing to database?

[Nishant Singh]

stderr: /bin/sh: 2: Syntax error: newline unexpected

[Mikko Ohtamaa]

Please edit the question with full information, including your script, databse info and so on. The given error message does not give any more information.

[Mikko Ohtamaa]

It is easier to help if one doesn't need to guess what is being done

[Mikko Ohtamaa]

Yes, now it does make a lot of more sense :)

[Mikko Ohtamaa]

Looks like the error happens when you pass the file content as is as a filename to some script. Script content is missing from the question so it s not possible to tell what the script is doing with the content and why it fails.

[Nishant Singh]

Yes it does it.. but what if the txt file gets replaced with .xml.. m gonna probably reframe my question

[Nishant Singh]

How can we use multiple replace in lookup ?? i am trying to remove more charecters like \r soo expecting something like this contents: "{{ lookup('file', '/etc/foo.txt') | replace('\n', '') | replace('\r', '') }}"

[007]

Wow, it worked. I swear I was working on this for hours. There was a trailing \n in a JSON string I was trying to post to a server using the uri_module. replace('\n', '') wasn't getting rid of it, I thought, what the hell, I'll try this, and it WORKED. argh. I think there was multiple levels of issues with this but thank you sir!

[Ryan Fisher]

Using the Jinja trim filter as in @andreas-maier 's answer is a more appropriate and readable solution.

[udondan]

@RyanFisher No it's not. trim removes leading and trailing characters, while the op wanted to remove all such characters, also in the middle. Also note, the question has been heavily modified since the answer was posted, therefore might not make too much sense. Anyway, thanks for the downvote. :)

[JCotton]

In my case trim filter would have worked (only had a trailing newline). However, my variable was Vault encrypted and trim didn't decrypt first. replace handled it correctly (decrypt, then remove newline). Thanks!

[Yorai Levi]

This is the only solution that worked