python & smtplib: Is sending mail via gmail using oauth2 possible?

Question

So I can login to and send mail through gmail using smtplib (using the script below), but I was just wondering if using oauth2 was an option like with imaplib? I didn't see anything on the smtplib documentation page about oauth and I haven't found anything googling. Thanks.

#! /usr/bin/python

import smtplib

to = 'myemailaddress'
gmail_user = 'myemailaddress'
gmail_pwd = 'passwd'
smtpserver = smtplib.SMTP("smtp.gmail.com",587)
smtpserver.ehlo()
smtpserver.starttls()
smtpserver.ehlo
smtpserver.login(gmail_user, gmail_pwd)
header = 'To:' + to + '\n' + 'From: ' + gmail_user + '\n' + 'Subject:testing \n'
print header
msg = header + '\n this is test msg from me \n\n'
smtpserver.sendmail(gmail_user, to, msg)
print 'done!'
smtpserver.close();

Edit:

Thanks to samy.vilar for the very detailed explaination provided in his answer. However, I'm having a little trouble. Here is my script:

#! /usr/bin/python

import oauth2 as oauth
import oauth2.clients.smtp as smtplib

consumer = oauth.Consumer('anonymous', 'anonymous')
token = oauth.Token('1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0', 'NysqNqVTulFsdHpSRrPP56sF')

url = "https://mail.google.com/mail/b/[email protected]/smtp/"

conn = smtplib.SMTP('smtp.googlemail.com', 587)
conn.set_debuglevel(True)
conn.ehlo('test')
conn.starttls()

conn.authenticate(url, consumer, token)

header = 'To:[email protected]\n' + 'From: [email protected]\n' + 'Subject:testing \n'
msg = header + '\n this is test msg from me \n\n'
conn.sendmail('[email protected]', '[email protected]', msg)

What is puzzling me, is that it appears to authenticate ok:

send: 'ehlo test\r\n'
reply: '250-mx.google.com at your service, [75.173.8.127]\r\n'
reply: '250-SIZE 35882577\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-STARTTLS\r\n'
reply: '250 ENHANCEDSTATUSCODES\r\n'
reply: retcode (250); Msg: mx.google.com at your service, [75.173.8.127]
SIZE 35882577
8BITMIME
STARTTLS
ENHANCEDSTATUSCODES
send: 'STARTTLS\r\n'
reply: '220 2.0.0 Ready to start TLS\r\n'
reply: retcode (220); Msg: 2.0.0 Ready to start TLS
send: 'AUTH XOAUTH R0VUIGh0dHBzOi8vbWFpbC5nb29nbGUuY29tL21haWwvYi90ZXN0aW5nLm9hdXRoLjFAZ21haWwuY29tL3NtdHAvIG9hdXRoX2JvZHlfaGFzaD0iMmptajdsNXJTdzB5VmIlMkZ2bFdBWWtLJTJGWUJ3ayUzRCIsb2F1dGhfY29uc3VtZXJfa2V5PSJhbm9ueW1vdXMiLG9hdXRoX25vbmNlPSI3Nzc0ODMyIixvYXV0aF9zaWduYXR1cmU9IkxuckZHODdxdHRxZUhsUlQ1emRndmtEZ1UzTSUzRCIsb2F1dGhfc2lnbmF0dXJlX21ldGhvZD0iSE1BQy1TSEExIixvYXV0aF90aW1lc3RhbXA9IjEzNDIxNDI3NzIiLG9hdXRoX3Rva2VuPSIxJTJGTUk2QjJEcUpQNEZFa0RSTFVLckQ1bDQ2c1EwNzU4LTJ1Y0VLQlktRGVCMCIsb2F1dGhfdmVyc2lvbj0iMS4wIg==\r\n'
reply: '235 2.7.0 Accepted\r\n'
reply: retcode (235); Msg: 2.7.0 Accepted

But then when it comes to sending the email, it seems to forget that conn was already authenticated:

send: 'ehlo [192.168.2.4]\r\n'
reply: '250-mx.google.com at your service, [75.173.8.127]\r\n'
reply: '250-SIZE 35882577\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-AUTH LOGIN PLAIN XOAUTH\r\n'
reply: '250 ENHANCEDSTATUSCODES\r\n'
reply: retcode (250); Msg: mx.google.com at your service, [75.173.8.127]
SIZE 35882577
8BITMIME
AUTH LOGIN PLAIN XOAUTH
ENHANCEDSTATUSCODES
send: 'mail FROM:<[email protected]> size=107\r\n'
reply: '530-5.5.1 Authentication Required. Learn more at\r\n'
reply: '530 5.5.1 http://support.google.com/mail/bin/answer.py?answer=14257 tu7sm3163839pbc.55\r\n'
reply: retcode (530); Msg: 5.5.1 Authentication Required. Learn more at
5.5.1 http://support.google.com/mail/bin/answer.py?answer=14257 tu7sm3163839pbc.55
send: 'rset\r\n'
reply: '250 2.1.5 Flushed tu7sm3163839pbc.55\r\n'
reply: retcode (250); Msg: 2.1.5 Flushed tu7sm3163839pbc.55
Traceback (most recent call last):
  File "./gmail_send3.py", line 47, in <module>
    conn.sendmail('[email protected]', '[email protected]', msg)
  File "/usr/lib/python2.7/smtplib.py", line 713, in sendmail
    raise SMTPSenderRefused(code, resp, from_addr)
smtplib.SMTPSenderRefused: (530, '5.5.1 Authentication Required. Learn more at\n5.5.1 http://support.google.com/mail/bin/answer.py?answer=14257 tu7sm3163839pbc.55', '[email protected]')

Solution

Interesting, I think it is possible, I found the following links
https://developers.google.com/google-apps/gmail/oauth_overview

UPDATE
https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough
This is the new walk through using oauth2, it seems like xoauth has being declared obsolete, I've update the URL to get the older version xoauth.py file, in case any one still needs it, though it looks like Google has changed their API and this walkthrough is no longer applicable.

you can use xoauth directly or https://github.com/simplegeo/python-oauth2/ to actually communicate I recommend the latter, its more versatile.

Apparently you first need to download xoauth.py script.

$ wget https://raw.githubusercontent.com/google/gmail-oauth2-tools/master/obsolete/python/xoauth.py
$ python xoauth.py --generate_oauth_token [email protected]

xoauth.py:74: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
import sha
oauth_token_secret: HFJEvjcTfiXSPxgLzDh-1yaH
oauth_token: 4/EwUxCtY9ye1kdtb4uNJIcaUe9KXl
oauth_callback_confirmed: true
To authorize token, visit this url and follow the directions to generate a verification code:
https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=4%2FEwUxCtY9ye1kdtb4uNJIcaUe9KXl
Enter verification code: 7XjT15fqk1aNe8152d9oTRcJ
oauth_token: 1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0
oauth_token_secret: NysqNqVTulFsdHpSRrPP56sF

lets test it:

$ python xoauth.py --test_imap_authentication [email protected] \
 --oauth_token=1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0 --oauth_token_secret=NysqNqVTulFsdHpSRrPP56sF

xoauth.py:74: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
xoauth string (before base64-encoding):
GET https://mail.google.com/mail/b/[email protected]/imap/ oauth_consumer_key="anonymous",oauth_nonce="18010070659685102619",oauth_signature="jTJv%2FAFATpzfq%2BZTLAAxFNmWPi0%3D",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1342084141",oauth_token="1%2FMI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0",oauth_version="1.0"

XOAUTH string (base64-encoded): R0VUIGh0dHBzOi8vbWFpbC5nb29nbGUuY29tL21haWwvYi90ZXN0aW5nLm9hdXRoLjFAZ21haWwuY29tL2ltYXAvIG9hdXRoX2NvbnN1bWVyX2tleT0iYW5vbnltb3VzIixvYXV0aF9ub25jZT0iMTgwMTAwNzA2NTk2ODUxMDI2MTkiLG9hdXRoX3NpZ25hdHVyZT0ialRKdiUyRkFGQVRwemZxJTJCWlRMQUF4Rk5tV1BpMCUzRCIsb2F1dGhfc2lnbmF0dXJlX21ldGhvZD0iSE1BQy1TSEExIixvYXV0aF90aW1lc3RhbXA9IjEzNDIwODQxNDEiLG9hdXRoX3Rva2VuPSIxJTJGTUk2QjJEcUpQNEZFa0RSTFVLckQ1bDQ2c1EwNzU4LTJ1Y0VLQlktRGVCMCIsb2F1dGhfdmVyc2lvbj0iMS4wIg==

09:01.40 > COKI1 AUTHENTICATE XOAUTH
09:01.44 < + 
09:01.45 write literal size 444
09:02.68 < * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE
09:02.68 < COKI1 OK [email protected] Testing Oauth authenticated (Success)
09:02.68 > COKI2 SELECT INBOX
09:03.09 < * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
09:03.09 < * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Flags permitted.
09:03.09 < * OK [UIDVALIDITY 3] UIDs valid.
09:03.09 < * 3 EXISTS
09:03.09 < * 0 RECENT
09:03.09 < * OK [UIDNEXT 4] Predicted next UID.
09:03.09 < COKI2 OK [READ-WRITE] INBOX selected. (Success)

looks like we are good to go. you may want to set up a virtualenv

$ git clone https://github.com/simplegeo/python-oauth2.git
$ cd python-oauth2/
$ sudo python setup.py install

and taken right from the docs:

import oauth2 as oauth
import oauth2.clients.imap as imaplib

# Set up your Consumer and Token as per usual. Just like any other
# three-legged OAuth request.
consumer = oauth.Consumer('anonymous', 'anonymous')
token = oauth.Token('1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0', 'NysqNqVTulFsdHpSRrPP56sF')

# Setup the URL according to Google's XOAUTH implementation. Be sure
# to replace the email here with the appropriate email address that
# you wish to access.
url = "https://mail.google.com/mail/b/[email protected]/imap/"

conn = imaplib.IMAP4_SSL('imap.googlemail.com')
conn.debug = 4 

# This is the only thing in the API for impaplib.IMAP4_SSL that has 
# changed. You now authenticate with the URL, consumer, and token.
conn.authenticate(url, consumer, token)

# Once authenticated everything from the impalib.IMAP4_SSL class will 
# work as per usual without any modification to your code.
conn.select('INBOX')
print conn.list()


>>> conn.authenticate(url, consumer, token)
20:11.73 > EPKK1 AUTHENTICATE XOAUTH
20:11.78 < + 
20:11.78 write literal size 496
20:11.93 < * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE
20:11.93 < EPKK1 OK [email protected] Testing Oauth authenticated (Success)
>>> conn.select('INBOX')
20:17.47 > EPKK2 SELECT INBOX
20:17.58 < * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
20:17.58 < * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen \*)] Flags permitted.
20:17.58 < * OK [UIDVALIDITY 3] UIDs valid.
20:17.58 < * 3 EXISTS
20:17.58 < * 0 RECENT
20:17.58 < * OK [UIDNEXT 4] Predicted next UID.
20:17.58 < EPKK2 OK [READ-WRITE] INBOX selected. (Success)
('OK', ['3'])
>>> print conn.list()
20:20.23 > EPKK3 LIST "" *
20:20.28 < * LIST (\HasNoChildren) "/" "INBOX"
20:20.28 < * LIST (\Noselect \HasChildren) "/" "[Gmail]"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/All Mail"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/Drafts"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/Important"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/Sent Mail"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/Spam"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/Starred"
20:20.28 < * LIST (\HasNoChildren) "/" "[Gmail]/Trash"
20:20.28 < EPKK3 OK Success
('OK', ['(\\HasNoChildren) "/" "INBOX"', '(\\Noselect \\HasChildren) "/" "[Gmail]"', '(\\HasNoChildren) "/" "[Gmail]/All Mail"', '(\\HasNoChildren) "/" "[Gmail]/Drafts"', '(\\HasNoChildren) "/" "[Gmail]/Important"', '(\\HasNoChildren) "/" "[Gmail]/Sent Mail"', '(\\HasNoChildren) "/" "[Gmail]/Spam"', '(\\HasNoChildren) "/" "[Gmail]/Starred"', '(\\HasNoChildren) "/" "[Gmail]/Trash"'])
>>> 

for smtp:

import oauth2 as oauth
import oauth2.clients.smtp as smtplib

# Set up your Consumer and Token as per usual. Just like any other
# three-legged OAuth request.
# Set up your Consumer and Token as per usual. Just like any other
# three-legged OAuth request.
consumer = oauth.Consumer('anonymous', 'anonymous')
token = oauth.Token('1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0', 'NysqNqVTulFsdHpSRrPP56sF')

# Setup the URL according to Google's XOAUTH implementation. Be sure
# to replace the email here with the appropriate email address that
# you wish to access.
url = "https://mail.google.com/mail/b/[email protected]/smtp/"

conn = smtplib.SMTP('smtp.googlemail.com', 587)
conn.set_debuglevel(True)
conn.ehlo('test')
conn.starttls()

# Again the only thing modified from smtplib.SMTP is the authenticate
# method, which works identically to the imaplib.IMAP4_SSL method.
conn.authenticate(url, consumer, token)



>>> conn.ehlo('test')
send: 'ehlo test\r\n'
reply: '250-mx.google.com at your service, [142.255.57.49]\r\n'
reply: '250-SIZE 35882577\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-STARTTLS\r\n'
reply: '250 ENHANCEDSTATUSCODES\r\n'
reply: retcode (250); Msg: mx.google.com at your service, [142.255.57.49]
SIZE 35882577
8BITMIME
STARTTLS
ENHANCEDSTATUSCODES
(250, 'mx.google.com at your service, [142.255.57.49]\nSIZE 35882577\n8BITMIME\nSTARTTLS\nENHANCEDSTATUSCODES')
>>> conn.starttls()
send: 'STARTTLS\r\n'
reply: '220 2.0.0 Ready to start TLS\r\n'
reply: retcode (220); Msg: 2.0.0 Ready to start TLS
(220, '2.0.0 Ready to start TLS')
>>> conn.authenticate(url, consumer, token)
send: 'AUTH XOAUTH R0VUIGh0dHBzOi8vbWFpbC5nb29nbGUuY29tL21haWwvYi90ZXN0aW5nLm9hdXRoLjFAZ21haWwuY29tL3NtdHAvIG9hdXRoX2JvZHlfaGFzaD0iMmptajdsNXJTdzB5VmIlMkZ2bFdBWWtLJTJGWUJ3ayUzRCIsb2F1dGhfY29uc3VtZXJfa2V5PSJhbm9ueW1vdXMiLG9hdXRoX25vbmNlPSI4MTEyMDkxNCIsb2F1dGhfc2lnbmF0dXJlPSJSaUFsTGdQWnpBSkNQJTJGWmx5aGRpYU1CV0xiTSUzRCIsb2F1dGhfc2lnbmF0dXJlX21ldGhvZD0iSE1BQy1TSEExIixvYXV0aF90aW1lc3RhbXA9IjEzNDIwODU2NzIiLG9hdXRoX3Rva2VuPSIxJTJGTUk2QjJEcUpQNEZFa0RSTFVLckQ1bDQ2c1EwNzU4LTJ1Y0VLQlktRGVCMCIsb2F1dGhfdmVyc2lvbj0iMS4wIg==\r\n'
reply: '235 2.7.0 Accepted\r\n'
reply: retcode (235); Msg: 2.7.0 Accepted
>>>

and to send emails:

>>> conn.authenticate(url, consumer, token)
send: 'AUTH XOAUTH R0VUIGh0dHBzOi8vbWFpbC5nb29nbGUuY29tL21haWwvYi90ZXN0aW5nLm9hdXRoLjFAZ21haWwuY29tL3NtdHAvIG9hdXRoX2JvZHlfaGFzaD0iMmptajdsNXJTdzB5VmIlMkZ2bFdBWWtLJTJGWUJ3ayUzRCIsb2F1dGhfY29uc3VtZXJfa2V5PSJhbm9ueW1vdXMiLG9hdXRoX25vbmNlPSI2OTg3ODM3NiIsb2F1dGhfc2lnbmF0dXJlPSIlMkZjUGslMkJRVWVJY1RaYXp1ekkwR1FzdkdtbDFBJTNEIixvYXV0aF9zaWduYXR1cmVfbWV0aG9kPSJITUFDLVNIQTEiLG9hdXRoX3RpbWVzdGFtcD0iMTM0MjA4NjAxNCIsb2F1dGhfdG9rZW49IjElMkZNSTZCMkRxSlA0RkVrRFJMVUtyRDVsNDZzUTA3NTgtMnVjRUtCWS1EZUIwIixvYXV0aF92ZXJzaW9uPSIxLjAi\r\n'
reply: '235 2.7.0 Accepted\r\n'
reply: retcode (235); Msg: 2.7.0 Accepted
>>> header = 'To:[email protected]\n' + 'From: [email protected]\n' + 'Subject:testing \n'
>>> msg = header + '\n this is test msg from me \n\n'
>>> conn.sendmail('[email protected]', '[email protected]', msg)
send: 'mail FROM:<[email protected]> size=107\r\n'
reply: '250 2.1.0 OK gb7sm6540492qab.12\r\n'
reply: retcode (250); Msg: 2.1.0 OK gb7sm6540492qab.12
send: 'rcpt TO:<[email protected]>\r\n'
reply: '250 2.1.5 OK gb7sm6540492qab.12\r\n'
reply: retcode (250); Msg: 2.1.5 OK gb7sm6540492qab.12
send: 'data\r\n'
reply: '354  Go ahead gb7sm6540492qab.12\r\n'
reply: retcode (354); Msg: Go ahead gb7sm6540492qab.12
data: (354, 'Go ahead gb7sm6540492qab.12')
send: 'To:[email protected]\r\nFrom: [email protected]\r\nSubject:testing \r\n\r\n this is test msg from me \r\n\r\n.\r\n'
reply: '250 2.0.0 OK 1342086030 gb7sm6540492qab.12\r\n'
reply: retcode (250); Msg: 2.0.0 OK 1342086030 gb7sm6540492qab.12
data: (250, '2.0.0 OK 1342086030 gb7sm6540492qab.12')
{}
>>>

make sure you authenticate before sending out emails.
hope this helps ...

UPDATE
There may be a bug that requires re-authentication, the following should work.

import oauth2 as oauth
import oauth2.clients.smtp as smtplib

consumer = oauth.Consumer('anonymous', 'anonymous')
token = oauth.Token('1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0', 'NysqNqVTulFsdHpSRrPP56sF')

url = "https://mail.google.com/mail/b/[email protected]/smtp/"

conn = smtplib.SMTP('smtp.googlemail.com', 587)
conn.set_debuglevel(True)
conn.ehlo('test')
conn.starttls()

conn.authenticate(url, consumer, token)

header = 'To:[email protected]\n' + 'From: [email protected]\n' + 'Subject:testing \n'
msg = header + '\n this is test msg from me \n\n'
try:
    conn.sendmail('[email protected]', '[email protected]', msg)
except Exception as ex:
    print str(ex)
    print 'retying ...'
    conn.authenticate(url, consumer, token)
    conn.sendmail('[email protected]', '[email protected]', msg)

though you can also use xoauth directly as such:

import time
import smtplib
import xoauth

consumer = xoauth.OAuthEntity('anonymous', 'anonymous')
access_token = xoauth.OAuthEntity('1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0', 'NysqNqVTulFsdHpSRrPP56sF')

xoauth_string = xoauth.GenerateXOauthString(consumer, access_token, '[email protected]', 'smtp', '[email protected]', str(xoauth.random.randrange(2**64 - 1)), str(int(time.time())))

smtp_conn = smtplib.SMTP('smtp.gmail.com', 587)
smtp_conn.set_debuglevel(True)
smtp_conn.ehlo()
smtp_conn.starttls()
smtp_conn.ehlo()
smtp_conn.docmd('AUTH', 'XOAUTH ' + xoauth.base64.b64encode(xoauth_string))

header = 'To:[email protected]\n' + 'From: [email protected]\n' + 'Subject:testing \n'
msg = header + '\n this is test msg from me \n\n'
smtp_conn.sendmail('[email protected]', '[email protected]', msg)

this seems to always work, I should submit a ticket to python-oauth2 team, thank you.

THOUGH THIS ALSO SEEM TO ALWAYS WORK

import oauth2 as oauth
import oauth2.clients.smtp as smtplib

consumer = oauth.Consumer('anonymous', 'anonymous')
token = oauth.Token('1/MI6B2DqJP4FEkDRLUKrD5l46sQ0758-2ucEKBY-DeB0', 'NysqNqVTulFsdHpSRrPP56sF')

url = "https://mail.google.com/mail/b/[email protected]/smtp/"

conn = smtplib.SMTP('smtp.googlemail.com', 587)
conn.set_debuglevel(True)
conn.ehlo('test')
conn.starttls()
conn.ehlo()
conn.authenticate(url, consumer, token)
header = 'To:[email protected]\n' + 'From: [email protected]\n' + 'Subject:testing \n'
msg = header + '\n this is test msg from me \n\n'
conn.sendmail('[email protected]', '[email protected]', msg)

it looks you need to call conn.ehlo() before authenticating.