In this post we are going to look at the Gmail API and how to request the full message body.
Reading message data from the Google Gmail API can be a little more complicated then you think. Gmail itself is an SMTP or mail server. Mail format is not as simple as JSON the format of an email is in mime format. (Multipurpose Internet Mail Extensions)
Reading the actual body of the email message itself needs to be done in a way to retrieve the full message payload.
I created this sample in python but I have created a similar version in C#. Two different clients, same week. Everyone is using Google APIs for simple backend systems.
I would like to thank the author of read full message in email with Gmail API on stack overflow which prompted this tutorial. A full sample can be found on Using the Gmail API request the full message body.
Setup
In this sample we are just going to use a simple installed application credentials. I have a YouTube Video that shows you how to create installed application credentials. If you already have your own credentials.json file then you are all set. Just make sure that you have enabled the Gmail api under Library.
You will need to have the following libraries
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
Authorization
I created a standard authorization method. This method is based upon one of the authorization methods from the QuickStart google has given us. I have altered it a little for my own use. My Authorize method takes a credentials_file_path and a token_file_path. credentials_file_path is the path to the credentials.json file you download from Google developer console, and token_file_path is where our user credentials will be stored.
When this code runs the access token and refresh token are returned to us. I store these in token_file_path for use later, this way I will not have to authorize this application every time I run it. Below is an example of the format of this file.
{
contence of token.json
“token”: “[redacted]”,
“refresh_token”: “[redacted]”,
“token_uri”: “https://oauth2.googleapis.com/token”,
“client_id”: “[redacted]”,
“client_secret”: “[redacted]”,
“scopes”: [ “https://mail.google.com/”],
“expiry”: “2022-11-18T12:04:52.280846Z”
}
Authorizing the application
The first thing my script does when it runs is call authorize, by passing it the path to my credentials.json and the path to the file I would like to store my user token in.
if __name__ == '__main__':
creds = Authorize('C:\\YouTube\\dev\\credentials.json', "token.json")
ListMessages(creds)
Listing messages
Now that we have authorized the application lets get a list of messages back from Gmail. The way the Gmail api works is that we first call the users.message.list method and ask it to return the messages for this user. Unfortunately for us this method only returns back a list of message id’s and a thread id’s.
def ListMessages(credentials):
try:
# create a gmail service object
service = build('gmail', 'v1', credentials=credentials)
# Call the Gmail v1 API
results = service.users().messages().list(userId='me').execute()
messages = results.get('messages', [])
if not messages:
print('No messages where found.')
return
print('Messages:')
for message in messages:
getMessage(credentials, message['id'])
except HttpError as error:
# TODO(developer) - Handle errors from gmail API.
print(f'An error occurred: {error}')
My ListMessages method takes the credentials as a parameter passed to it. This was returned from the Authorize method. It then creates a Gmail service object. All of the Google apis work on the concept of a service object. We are currently creating a Gmail service object, but we could just as easily be creating a google drive service object.
Get a single message
So as I mentoned we are only getting a messageid and a threadid back from a list messages. So the next step is to request the full message back from the user.messages.get method. Yes this means you need to preform a get on every single message that was returned by the list messages method.
def getMessage(credentials, message_id):
# get a message
try:
service = build('gmail', 'v1', credentials=credentials)
# Call the Gmail v1 API, retrieve message data.
message = service.users().messages().get(userId='me', id=message_id, format='raw').execute()
# Message snippet only.
print('Message snippet: %s' % message['snippet'])
except HttpError as error:
# TODO(developer) - Handle errors from gmail API.
print(f'A message get error occurred: {error}')
Now if you run the method above. You will find that only a snippet of the message is returned. Its not the full message itself.
Retrieve the message headers
Lets look at reading the messgae headers, we would like to know who sent the message to it was intended for and the subject. So lets change a few things in the getMessage method.
def getMessage(credentials, message_id):
# get a message
try:
service = build('gmail', 'v1', credentials=credentials)
# Call the Gmail v1 API, retrieve message data.
message = service.users().messages().get(userId='me', id=message_id, format='raw').execute()
# Parse the raw message.
mime_msg = email.message_from_bytes(base64.urlsafe_b64decode(message['raw']))
print(mime_msg['from'])
print(mime_msg['to'])
print(mime_msg['subject'])
# Message snippet only.
print('Message snippet: %s' % message['snippet'])
except HttpError as error:
# TODO(developer) - Handle errors from gmail API.
print(f'A message get error occurred: {error}')
What we are going to do is load the raw message from the get response. This is the full mime message returned by the server. However its in base64 format. To deal with that we first parse it from urlsafe_b64decode and then use the email method message_from_bytes to convert it into something we can read. Which then allows us to check the from, to and subject headaers.
Retrieve the full message body
Okay but what about the full message body. Snippet may be good for somethings but we really want to read the full message body.
def getMessage(credentials, message_id):
# get a message
try:
service = build('gmail', 'v1', credentials=credentials)
# Call the Gmail v1 API, retrieve message data.
message = service.users().messages().get(userId='me', id=message_id, format='raw').execute()
# Parse the raw message.
mime_msg = email.message_from_bytes(base64.urlsafe_b64decode(message['raw']))
print(mime_msg['from'])
print(mime_msg['to'])
print(mime_msg['subject'])
print("----------------------------------------------------")
# Find full message body
message_main_type = mime_msg.get_content_maintype()
if message_main_type == 'multipart':
for part in mime_msg.get_payload():
if part.get_content_maintype() == 'text':
print(part.get_payload())
elif message_main_type == 'text':
print(mime_msg.get_payload())
print("----------------------------------------------------")
# Message snippet only.
# print('Message snippet: %s' % message['snippet'])
except HttpError as error:
# TODO(developer) - Handle errors from gmail API.
print(f'A message get error occurred: {error}')
To do that we need to add one more thing. We need to read the payload from the mime message and if its text we can then out put it to the user. Remember this is the full message so it is bound to contain some HTML in it among other things.
Conclusion
Reading messages from the Gmail api isnt as strait forward as you would think. First off you need to use the message.get method to see the full message. Then the raw message needs to be converted from base64 into something that we can use.