How to input a regex in string.replace?

Question

I need some help on declaring a regex. My inputs are like the following:

this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>. 
and there are many other lines in the txt files
with<[3> such tags </[3>

The required output is:

this is a paragraph with in between and then there are cases ... where the number ranges from 1-100. 
and there are many other lines in the txt files
with such tags

I've tried this:

#!/usr/bin/python
import os, sys, re, glob
for infile in glob.glob(os.path.join(os.getcwd(), '*.txt')):
    for line in reader: 
        line2 = line.replace('<[1> ', '')
        line = line2.replace('</[1> ', '')
        line2 = line.replace('<[1>', '')
        line = line2.replace('</[1>', '')
        
        print line

I've also tried this (but it seems like I'm using the wrong regex syntax):

        line2 = line.replace('<[*> ', '')
        line = line2.replace('</[*> ', '')
        line2 = line.replace('<[*>', '')
        line = line2.replace('</[*>', '')

I dont want to hard-code the replace from 1 to 99.

Solution 1

This tested snippet should do it:

import re
line = re.sub(r"</?\[\d+>", "", line)

Edit: Here's a commented version explaining how it works:

line = re.sub(r"""
  (?x) # Use free-spacing mode.
  <    # Match a literal '<'
  /?   # Optionally match a '/'
  \[   # Match a literal '['
  \d+  # Match one or more digits
  >    # Match a literal '>'
  """, "", line)

Regexes are fun! But I would strongly recommend spending an hour or two studying the basics. For starters, you need to learn which characters are special: "metacharacters" which need to be escaped (i.e. with a backslash placed in front - and the rules are different inside and outside character classes.) There is an excellent online tutorial at: www.regular-expressions.info. The time you spend there will pay for itself many times over. Happy regexing!

Solution 2

str.replace() does fixed replacements. Use re.sub() instead.

Solution 3

I would go like this (regex explained in comments):

import re

# If you need to use the regex more than once it is suggested to compile it.
pattern = re.compile(r"</{0,}\[\d+>")

# <\/{0,}\[\d+>
# 
# Match the character “<” literally «<»
# Match the character “/” literally «\/{0,}»
#    Between zero and unlimited times, as many times as possible, giving back as needed (greedy) «{0,}»
# Match the character “[” literally «\[»
# Match a single digit 0..9 «\d+»
#    Between one and unlimited times, as many times as possible, giving back as needed (greedy) «+»
# Match the character “>” literally «>»

subject = """this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>. 
and there are many other lines in the txt files
with<[3> such tags </[3>"""

result = pattern.sub("", subject)

print(result)

If you want to learn more about regex I recomend to read Regular Expressions Cookbook by Jan Goyvaerts and Steven Levithan.

Solution 4

The easiest way

import re

txt='this is a paragraph with<[1> in between</[1> and then there are cases ... where the<[99> number ranges from 1-100</[99>.  and there are many other lines in the txt files with<[3> such tags </[3>'

out = re.sub("(<[^>]+>)", '', txt)
print out

Solution 5

replace method of string objects does not accept regular expressions but only fixed strings (see documentation: http://docs.python.org/2/library/stdtypes.html#str.replace).

You have to use re module:

import re
newline= re.sub("<\/?\[[0-9]+>", "", line)

View more solutions

Comments

[Admin]

Also worth noting that your pattern should look something like "</{0-1}\d{1-2}>" or whatever variant of regexp notation python uses.

[pcurry]

Also don't neglect The Book on Regular Expressions - Mastering Regular Expressions, by Jeffrey Friedl

[avi]

What does fixed replacements mean?

[Gunay Anach]

@avi Probably he meant fixed word replacement rather partial word locating through regex.

[winklerrr]

You should use \d+ instead of [0-9]+

[winklerrr]

From the python docs: {0,} is the same as *, {1,} is equivalent to +, and {0,1} is the same as ?. It’s better to use *, +, or ? when you can, simply because they’re shorter and easier to read.

[vstepaniuk]

fixed (literal, constant) strings

[Carson]

Another good reference sees w3schools.com/python/python_regex.asp

[RufusVS]

The commented version mentions (?x) free-spacing mode, but that is not in the snippet. Is that a default or something?

[ridgerunner]

@RufusVS - The '(?x)' inside the regex text tells the regex engine compiler that this regex is written in free-spacing mode. You could alternatively add the: 're.VERBOSE' compilation flag to the function call.

[MikeM]

691 votes and a bounty for that? You'd be lucky to get a single vote nowadays. And Rufus already knew it meant free-spacing mode, he just didn't know what that was - and you have still not explained it to him.

[RufusVS]

@ridgerunner Actually, my point was, you have the expression (?x) present in your commented version of the pattern, but NOT in the uncommented version above it that you called tested snippet. (Edit: I googled it and found out the free-spacing expression is only needed because you have comments and spaces in the explanatory version, not needed in the tested snippet version. Got it now.)

[Jacob Myer]

Every answer on this entire site for a regex question should be written this way with comments explaining each part