paste url into terminal(urxvt, zsh) failed, some characters get escaped

Question

In recent months, I find that if I copy the URL in Chrome and then Shift-Insert in the urxvt, the pasted URL is escaped.

For example, the original URL is:

http://example.com/?a=c

the pasted content is:

http://example.com/\?a\=c

But if I paste it into other places, such as in the web browser or in the vim, there is no escape.

What I use is Arch Linux, urxvt, zsh, oh-my-zsh.

Solution 1

This is a Z Shell module known as "url-quote-magic" in action. It is trying to ensure that you end up with what you intended even though you completely ignored shell metacharacters and quoting. It detects when (in ZLE) an unquoted word looks like a URL, with a schema on the front, and changes the way that character self-insertion happens to the rest of the word.

If the paste operation had simply entered

http://example.com/?a=c

when run

fail to execute

A well-known example by Vivek Verma

~$  mpv https://www.youtube.com/watch?v=HcgJRQWxKnw
zsh: no matches found: https://www.youtube.com/watch?v=HcgJRQWxKnw
~$  

Remember: The Z Shell has a lot of filename expansion characters — not only including ?, [, ], and *; but also = (command name search), < and > (number ranges), ~, ^, and #. And that's not even including the Korn shell compatibility mechanisms. See the zshexpn manual page for the quite lengthy details.

url-quote-magic determined that this wasn't a quoted word, recognized the http: schema prefix, and changed the ? and = into \? and \= so that they wouldn't invoke filename expansion.

So unless you actually want, for some reason, URLs that you've pasted or typed in (without your adding any enclosing single quotes, note) to be subject to all of the filename expansions and either fail to work or (in the rare extreme surprise case) produce unexpected matches, you should probably be glad that this automatic quoting of shell metacharacters in what you clearly think of as URLs is being done for you. ☺

Solution 2

While @JdeBP provided great explanation of what's happening, I don't seem to see here what can be done about this behaviour.

Simply enclosing the URL link with double quotes prevents zsh from escaping in upon paste in:

"http://example.com/?a=c"

pasted content:

"http://example.com/?a=c"

Comments

[phk]

You want that it will be never escaped at all when you post it in the shell?

[宇宙人]

Do you mean that it just displays http://...\?a\=c and helps zsh avoids expanding. If I do something like aria2c http://...\?a\=c in zsh, the actual command to run is still aria2c http://...?a=c?

[宇宙人]

No, not actually. Under following situation, the command gets affected. In some cases, the urls contain spaces (some urls even contain both CJK characters and spaces), so I must wrap the url with quotes and unescape all \?s back to ?s.

[Michael]

Hello from 5 years in the future. This doesn't consider if a URL is pasted inside quotes which breaks them.

[Stéphane Chazelas]

*, ?, [...]. <->, (a|b)... (and x#, ^x, a~b... with extendedglob) is filename generation aka globbing (POSIX calls it pathname expansion). =cmd, ~user, ~12 is the one called filename expansion in zsh and is not the same as globbing.

[AdminBee]

Welcome to the site, and thank you for your contribution. Please note however that the OP was about pasting a URL copied from the address bar of a web browser, so it would seem difficult (or at least very cumbersome) to apply this solution to that scenario. You may want to add a short note on this.

[bradym]

This requires quoting the text before pasting it - which I didn't realize when I first read your answer. I thought you were saying you could paste the text into the command line as long as you were pasting it inside quotes. For the record that doesn't work. :)

[Radim Hrazdil]

Yes, indeed the quoting needs to be added before pasting it. It didn't realize from the OP question that he wants to avoid doing that. I don't have an answer in that case :)