Javascript - regex - word boundary (\b) issue

Question

I have a difficulty using \b and greek characters in a regex.

At this example [a-zA-ZΆΈ-ώἀ-ῼ]* succeeds to mark all the words I want (both greek and english). Now consider that I want to find words with 2 letters. For the English language I use something like this: \b[a-zA-Z]{2}\b. Can you help me write a regex that succeeds to mark words in Greek with 2 letters? (Why? My final goal is to remove them).

text used:

Greek MONOTONIC: Το γάρ ούν και παρ' υμίν λεγόμενον, ώς ποτε Φαέθων Ηλίου παίς το του πατρός άρμα ζεύξας δια το μή δυνατός είναι κατά την του πατρός οδόν ελαύνειν τα τ' επί της γής ξυνέκαυσε και αυτός κεραυνωθείς διεφθάρη, τούτο μύθου μέν σχήμα έχον λέγεται, το δέ αληθές εστι των περί γήν και κατ' ουρανόν ιόντων παράλλαξις και διά μακρόν χρόνον γιγνομένη των επί γής πυρί πολλώ φθορά.

Greek POLYTONIC: Τὸ γὰρ οὖν καὶ παρ' ὑμῖν λεγόμενον, ὥς ποτε Φαέθων Ἡλίου παῖς τὸ τοῦ πατρὸς ἅρμα ζεύξας διὰ τὸ μὴ δυνατὸς εἶναι κατὰ τὴν τοῦ πατρὸς ὁδὸν ἐλαύνειν τὰ τ' ἐπὶ τῆς γῆς ξυνέκαυσε καὶ αὐτὸς κεραυνωθεὶς διεφθάρη, τοῦτο μύθου μὲν σχῆμα ἔχον λέγεται, τὸ δὲ ἀληθές ἐστι τῶν περὶ γῆν καὶ κατ' οὐρανὸν ἰόντων παράλλαξις καὶ διὰ μακρὸν χρόνον γιγνομένη τῶν ἐπὶ τῆς γῆς πυρὶ πολλῷ φθορά.

ENGLISH: For in truth the story that is told in your country as well as ours, how once upon a time Phaethon, son of Helios, yoked his father's chariot, and, because he was unable to drive it along the course taken by his father, burnt up all that was upon the earth and himself perished by a thunderbolt,—that story, as it is told, has the fashion of a legend, but the truth of it lies in the occurrence of a shifting of the bodies in the heavens which move round the earth, and a destruction of the things on the earth by fierce fire, which recurs at long intervals.

what I've tried so far:

// 1
txt = txt.replace(/\b[a-zA-ZΆΈ-ώἀ-ῼ]{2}\b/g, '');

// 2
tokens = txt.split(/\s+/);
txt = tokens.filter(function(token){ return token.length > 2}).join(' ');

// 3
tokens = txt.split(' ');
txt = tokens.filter(function(token){ return token.length != 3}).join(' ') );

2 & 3 were suggested to my question here: Javascript - regex - how to remove words with specified length

EDIT

Read also:

• Why can't I use accented characters next to a word boundary?
• Javascript + Unicode regexes

Solution 1

Since Javascript doesn't have the lookbehind feature and since word boundaries work only with members of the \w character class, the only way is to use groups (and capturing groups if you want to make a replacement):

(?m)(^|[^a-zA-ZΆΈ-ώἀ-ῼ\n])([a-zA-ZΆΈ-ώἀ-ῼ]{2})(?![a-zA-ZΆΈ-ώἀ-ῼ])

example to remove 2 letters words:

txt = txt.replace(/(^|[^a-zA-ZΆΈ-ώἀ-ῼ\n])([a-zA-ZΆΈ-ώἀ-ῼ]{2})(?![a-zA-ZΆΈ-ώἀ-ῼ])/gm, '\1');

Solution 2

You can use \S

Rather than write a match for "word characters plus these characters" it may be appropriate to use a regex that matches not-whitespace:

\S

It's broader in scope, but simpler to write/use.

If that's too broad - use an exclusive list rather than an inclusive list:

[^\s\.]

That is - any character that is not whitespace and not a dot. In this way it's also easy to add to the exceptions.

Don't try to use \b

Word boundaries don't work with none-ascii characters which is easy to demonstrate:

> "yay".match(/\b.*\b/)
["yay"]
> "γaγ".match(/\b.*\b/)
["a"]

Therefore it's not possible to use \b to detect words with greek characters - every character is a matching boundary.

Match 2 character words

The following pattern can be used to match two character words:

pattern = /(^|[\s\.,])(\S{2})(?=$|[\s\.,])/g;

(More accurately: to match two none-whitespace sequences).

That is:

(^|[\s\.,]) - start of string or whitespace/punctuation (back reference 1)
(\S{2})     - two not-whitespace characters (back reference 2)
($|[\s\.,]) - end of string or whitespace/punctuation (positive lookahead)

That pattern can be used like so to remove matching words:

"input string".replace(pattern);

Here's a jsfiddle demonstrating the patterns use on the texts in the question.

Comments

[donfuxx]

won't match if 2 letter word is at end of sentence or followed by comma I guess

[disklosr]

You're right. But for simple text content one can easily add other symbols/puctuation like coma, (semi)column, dot...

[tgogos]

Your approach is the most close to what I want but sometimes removes line feeds. (Demo)

[Casimir et Hippolyte]

@antithesis: indeed, see my edit, I have corrected the problem.

[tgogos]

Thank you for all the job you've done, but can you check some issues with line feeds and ellipses? DEMO

[AD7six]

ellipses are easy to account for - I added that permutation to the answer. new lines just should just need a multi-line flag, I'll test it later. It would appear I've already answered your other question - see the js fiddle.

[tgogos]

Can you please explain to me what does ^| do inside the 1st parentheses?

[Casimir et Hippolyte]

@antithesis: | is a regex special character and means "OR", ^ is an anchor for the start of the line. The first parenthesis replace the word boundary (since you can't use this feature with greek alphabet). The first parenthesis means: "start of the line OR a character that is not a letter (greek or latin)".

[tgogos]

Thank you! I also used 0-9 inside the first and the third match because I was removing words like "2TB" or "mp3".