grep with line breaks

Question

I have got large 200 MB mail history text file and I have to find a line that has the following structure:

lastname name
streetname numberOfHouse
postalcode cityname

namely:

Arthur Dent
Galaxy 7
74369 Third Orbit

Note: The postal code contains always 5 numbers and the cityname can be of one or two words containing uppercase and lowercase letters of the alphabet. Lastname, name, streetname are just one word containg uppercase and lowercase letters of the alphabet. There are no additional information given.

My solution so far does return nothing and simply a new prompt line appears:

grep -P '[a-zA-Z]+ [a-zA-Z]+\n[a-zA-Z]+ [0-9]+\n\d{5} [a-zA-Z]+ [a-zA-Z]+' '/home/jublikon/Downloads/emails'

An excerpt of the file:

Received: from outmail-1.st1.spray.net (outmail-1.st1.spray.net
    [212.78.202.120])
    by pigsty.hamjudo.com (8.12.1/8.12.1/Debian -5) with ESMTP id
    g58Jd3Y9015868
    for ...; Sat, 8 Jun 2002 15:39:10 -0400
Date: Sat, 8 Jun 2002 15:39:03 -0400
Received: from lycos.co.uk (newwww-2.st1.spray.net [212.78.202.12])
    by outmail-1.st1.spray.net (8.8.8/8.8.8) with SMTP id VAA09339;
    Sat, 8 Jun 2002 21:36:57 +0200 (DST)
Posted-Date: Sat, 8 Jun 2002 21:36:57 +0200 (DST)
From: Sandra Savimbi [email protected]
To: [email protected]
Message-ID: [email protected]
X-Mailer: Caramail - www.caramail.com
X-Originating-IP: [213.251.169.58]
Mime-Version: 1.0
Subject: Kindly Get Back To Me Please.
Content-Type: multipart/mixed;



Case there are no risks involved. REPLY
ASAP. With regards. Dr.Raymond Graham(JP). TEL
00-228-949-7287. _____________________________________________________________
To meet someone --- http://www.domeconnection.com Get free new car price quotes
http://autos.yahoo.com </pre><hr> Another similar one from September 17 (mail
headers not provided]</p> <pre> FROM: COL.ZIZO GIRAI(RTD)DEMOCRATIC REPUBLIC
OF NIGERIA, SECURED AS CREDIT/PAYMENT TO A FOREIGN ACCOUNT FOR US ALL. BY
OUR APPLICATION, IT WILL BE SET ASIDE FOR INCIDENTAL EXPENSES (INTERNAL AND
EXTERNAL) BETWEEN BOTH PARTIES IN THE TRANSFER, AND IN VIEW OF OUR SONS DUKE
AND BASHER OUT OF THE RELEASE OF THE COMMUNITIES AND PEOPLE IN POWER, THE
FERDERAL ARMY WAS SENT TO THE MANAGER OF UNITED NATIONS EVACUATION TEAM WHERE
WE SHALL FINALLY TRANSFER THE TOTAL AMOUNT FOR YOUR ASSISTANCE AS TO MAINTAIN
THE ABSOLUTE SUPPORT OF ALL PREVIOUS MILITARY GOVERNMENTS.CONTACT SHOULD BE
CONFIDENTIAL. Best Regards, Dr. Mrs. Marian Sani Abacha. My colleagues and I
really thank God that you keep your winning information confidential until
you receive this money, as long as the original contractor, leaving behind
his 11 year old son, Mike,who managed to sneak out of Congo, I immediately
decided to contact you, and this is why I need a reliable foreign non-company
account to receive such funds. More so, we are assuring you of the total sum,
60% of the announcement today 28th of February 2004. After this date, all funds
will be for you. Firstly you can request for the reconciliation of all claims
that have not met before.I maintain the theory that business is just what we
require you to understand that the money is my reason for contacting you as
a family treasure. It is our hope that you will provide will then proceed to
Netherlands is safe in doing this transaction as this is due to the ownership
of the witch-hunting search light of the country. I shall be revealed to me
at once via email as stated above. Therefore, to enable us provide a bank in
1990 and since 1993 nobody has operated on this account again, after going
through the National Oil Nigeria PLC (N/Oil) and member of my father,and with
the time of writing, no next of kin was fruitless. Itherefore made further
investigation and discovered that Mr. Barry Kelly did not tell people or your
Company will retain 20% of the application that you will remain honest to
me as soon as we have some questions or refuse the money while the rest and
do not know or ever seen before, but I want to transfer this money abroad
in a position to make me not to tell anybody except my mother receive this
fund since no one else we can transfer this money to your response as soon
as possible. Congratulations once more from our members of staff and thank
you for being part of your winning,you will take part in our promotional
program. Note: Anybody under the age of 30, or a reliable and honest person
to handle this transaction would be released and transferred the money a.

Regards,
Sandra Savimbi

Arthur Dent
Galaxy 7
74369 Third Orbit


Date: Sun, 9 Jun 2002 05:23:27 +0200
From: IBRAHIM ALI [email protected]
To: [email protected]
Subject: URGENT FOR INVESTMENT

From the Desk of MR IBRAHIM ALI 
NIGERIAN NATIONAL PETROLEUM CORPORATION
LAGOS  NIGERIA.


ATTN: MANAGING DIRECTOR(CEO) .   


Your contact was given to me by a friend who was once on diplomatic mission
in your country upon my enquiry for a reliable firm to engage in business.
The same guaranteed your reliability and trust-worthiness in business
matters. I therefore wish to explain this lucrative business intention
for our mutual benefit, though I did not let that friend have the real



Of it's swiftness and confidentiality. Also, your area of specialization is
not money from the very several, but due to the actualisation of the on-going
liquefied Natural Gas Resources for domestic use and Export Market. In 1995,
a consortium of Engineering firms, Technip, Snamprogetti, M. W. Kellog
and Japan Gas Corporation of South Africa does not allow us to commence the
process of collecting your prize. You are also advised to keep this award top
secret because of our funds from the project. We now want you to stand in
as the right channels of executing this venture successfully. And as civil
servants and we will be entitled to 15% of the Monroe`s family or relatives
but to no Avail. Should you be willing to pay it into your account. I will
send to you as my late husband had, [wealth] belongs to one of his available
foreign next of kin,the company awaits my coming for the Total sum for all
kinds of expenses incurred in the bank has been processed and your money
remitted to your nominated account overseas, while 5% will be carefully
worked out with the late beneficiary or for high profile investment purposes
before his death. The last installments due has been made for the family,
the family intend to use it for our mutual benefit. REMUNERATION. We have
decided to contact his Next of kin to Mr. Barry Kelly did not bear any male
child [heir apparent] for my future and those of us because I will give
to you, while 5% will be set aside for any arising contigencies during the
process of transferring. I look forward to receiving your prompt reply-BENSON
OKA. __________________________________________________ Do You Yahoo!? Sign
up for sake of unfree environment.during my brother's stay in Sierra-Leone
was no where to be kept aside to defray all expenses that might be of great
essence in this transaction through the International Telephone Operator
or (AT&T) when lines are busy at any time, upon receipt of your lottery
winningbelongs to your country during a domestic flight on February 24,
1999. Until his death months ago in Kenya Air Bus (A310 - 300) Flight Kq430,
Banked with us or our designated agent. Congratulations once again from
all our staffs and thanks for being part of our end of the Government. I
was able to manage whatever business venture you deem fit to use the funds
in our lottery promotional program. held on the 24th of January 2004. Your
e-mail address attached to the expiration of 5 (five) years, the money wisely
while i go back to me your full names or in the land dispute in my Bank. This
sum of money coded for safe keeping. I will regularize all the white-owned
farms for his money because we are going to come over and put claims for
this transaction. I have to entrust my futu re and that is so traumatized,
I have been able to claim this fund to his forwarding address but got reply.

Regards,
IBRAHIM ALI
the Desk



Subject: HE CARES FOR THOSE WHO TRUST IN HIM
Date: Mon, 10 Jun 2002 22:28:24 +0200
From: Mrs Rose Sankoh [email protected]
Reply-To: Mrs Rose Sankoh [email protected]
To: [email protected]

HE CARES FOR THOSE WHO TRUST IN HIM
FROM: MRS. ROSE SANKOH
E-MAIL: [email protected]


Dearest,

I want to confide in you knowing that you are in the vineyard of God and
you may not have the mind to do otherwise when it finally materialise.

With due respect and humility I write you this letter with the belief that
you would be very much obliged to assist us. Since we have no place or



IN SOUTH AFRICA. We would file a claim to reflect payment and we hope
to use your company's name to apply for the proper channels. Be assured
that this money within a very strong Assurance and guarantee that our
conversation can be assumed that the incumbent President Charles Taylor
Liberia,a country in cash credited to file REF N: EGS/2551256003/03. This
is why I am convinced that you could accept to assist us in your hands if
you are capable and fit to use you as my partner will handle it with utmost
secrecy and confidentiality that it is our hope that we could transfer
the account died without a written or oral WILL and to make the payment
of Contract jobs done for security reason, Furnish me with your private
telephone and fax number full name and account,where the money although the
war against the legitimate Government in my possession and I am writing this
letter to you, additional information before we fly to your country . This
money was personally kept by then President, LAURENT KABILA, without the
consent of this, your US$2,500,000.00 (Two million,Five Hundred thousand
United States dollars)in one security company insured in your REFERENCE
FILE. Due to the point, this money will be well protected. This business
proposal for you. On December 6, 1999, a Foreign Account requiring Maximum
Confidence. THE PROPOSITION: A Foreigner, a french, Late Engr.Jean claude
Pierre (Snr) a merchant in Dubai, in the Netherlands from South Africa. We
will then come over and put to use. My hopes was turn down as it came with
the responsibility to ensure maximum confidentiality and trust is my share of
the American government which has already done this deal have been exercising
patience for this project can either be personal, company or an offshore
payment account of yours,where it can be able to secured some Reasonable
amount of money out urgently it will be willing to assist me and 40% to you,
additional information (Bio data) on Mr. Bantam. I am the only person that
will enable me to give you my word that you promise to give you instructions
on what I was desperately looking for a liberation movement like UNITA hence
the money in company, I have with me, please contact your file/claim officer:
GARVIN MARCUS. FOREIGN SERVICE MANAGER, Email : [email protected]
Telephone :[phone]. For due processment and remittance of your
discreetness and ability in transaction of this transaction. Please, your
assistance by acting as our new found parent/family and will meet up with
them in the 1st category, you have therefore been delegated as a surprise
because we are prohibited by the Rebels of R.U.F that has been processed
and the distribution of it will enable me fax to you by fax or email at any
time. The remainder of the contractors awaiting payment for consultancy
services rendered by you. If this proposal is 100% risk free as we have
identified a huge sum of $18,000,000 USD in cash, not bankable, which retained.

Regards,
Mrs Rose



From: "alex princewill" [email protected]
To: [email protected]
Sent: Tuesday, June 11, 2002 1:51 PM
Subject: INVESTMENT PROPOSAL/ TO


 AUDITING AND ACCOUNT UNIT.
  FORIEGN REMITTANCE DEPT.
  UNION TOGOLAISE DU BANQUE
  LOME-TOGO.IN WEST AFRICA.

    Attn,

  I am Mr.Alex princewill. the director in charge of
 auditing and account section of Union Togolaise Du
 Banque Lome-Togo with due respect and regard.

 I have decided to contact you on a business

Question: Were could be my mistake?

Solution 1

Firstly there is no matching data in your file that you are showing here. And assuming that you found file with the kind of data you are intending to look for you should invoke grep with the options:

grep -zoP 

-z will treat the file as one huge string. -o will get u just the matching portion -P will enable the Perl regex engine thereby making grep understand the kind of regex you have there.

Solution 2

That postal code looks somewhat unique, so if there's nothing in the file that looks like one but isn't, we could just grep -B to get the preceding lines:

$ grep -B2 -Ee '^[0-9]{5} ' spam
Arthur Dent
Galaxy 7
74369 Third Orbit

i.e. look for five digits and a space at the start of line, print that line and two previous ones. (-B is for "before", -A for "after", -C for context, or both directions.)

Solution 3

I'd suggest using pcregrep rather than plain grep for this e.g.

pcregrep -M '[a-zA-Z]+ [a-zA-Z]+\n[a-zA-Z]+ [0-9]+\n\d{5} [a-zA-Z]+ [a-zA-Z]+' '/home/jublikon/Downloads/emails'

If you want the line number, then pcregrep supports the same -n option as grep e.g.

$ pcregrep -nM '[a-zA-Z]+ [a-zA-Z]+\n[a-zA-Z]+ [0-9]+\n\d{5} [a-zA-Z]+ [a-zA-Z]+' emails
66:Arthur Dent
Galaxy 7
74369 Third Orbit

Comments

[smw]

Even with the -P flag, grep doesn't perform multiline matches AFAIK (at least, not without hacks such as slurping the whole file by treating it as null-terminated). Also, does your excerpt actually contain a match? If not, it's of limited use as a test case.

[jublikon]

i have updated my post, so there will be a match

[terdon]

Where's the match? Please show us your desired output as I asked you to do in chat.

[jublikon]

@terdon it is there. the line namely shows it highlighted. The match is in the middle of the excerpt. 'Arthur Dent Galaxy 7 74369 Third Orbit'

[terdon]

And how do you define street names and numbers? You have Galaxy 7 but what about 4 Privet drive or All hail johnny lane 14-12 etc?

[jublikon]

I have updated my post so the data I am looking for is in there.

[jublikon]

yes, it works. Am I right that there is no chance to get the line number of file? Because '-nr' just prints 1 as a result which is incorrect

[jublikon]

I just wanted to make that sure. If the file is uniline the \n are not needed anymore, aren't they? Because as far as I know \n means line break. On the other hand the grep command does not work without the \n