Python requests post to api gateway

Question

This version of the AWS Code Sample Catalog has been replaced by the AWS Code Library, which contains new and updated code examples.

The examples listed on this page are code samples written in Python that demonstrate how to interact with Amazon API Gateway.

For more information, see the AWS SDK for Python (Boto3) Getting Started and the API Gateway Developer Guide.

REST API to an AWS Service

I am writing a python program that calls AWS API Gateway to reach dynamoDB.

And it all works fine when I use python "requests": r = requests.post("https://xxxxxxx.execute-api.us-east-1.amazonaws.com/Prod/log-entries", data={'logbookTimestamp': timestamp, 'Name': "Fred"})

Now in order to run this as a lambda function, I want to use "urllib3" instead of "requests" because urllib3 is included by default in lambda's python. So now I am trying to do the same with urllib3 but can't get it to work. I've read the urllib3 user guide here (https://urllib3.readthedocs.io/en/latest/user-guide.html#json) and it says I need to encode the JSON data before sending it so I've done this:

http = urllib3.PoolManager()
fields = {'logbookTimestamp': timestamp, 'Name': "Fred"}
encoded_fields = json.dumps(fields).encode('utf-8')
link = "https://xxxxxxx.execute-api.us-east-1.amazonaws.com/Prod/log-entries"
r = http.request('POST',
                 link,
                 body=encoded_fields,
                 headers={'Content-Type': 'application/json'}
                 )

When I look at the output of both in CloudWatch I see that the data are formatted differently.

With requests: (c084a37e-43d8-464a-9dcf-e40c28922ece) Method request body before transformations: logbookTimestamp=2020%3A12%3A15%3A20%3A11%3A02&Name=Fred

With urllib3: (9b8d84e9-2403-462c-b25f-945a927d1e66) Method request body before transformations: { "logbookTimestamp": "2020:12:15:21:31:21", "Name": "Fred" }

and returns the following: (9b8d84e9-2403-462c-b25f-945a927d1e66) Endpoint response body before transformations: { "statusCode": 500, "headers": { "Content-Type": "text/html; charset=utf-8", "Content-Length": "290" }, "body": "\n500 Internal Server Error\n

Internal Server Error

\n

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

\n" } (9b8d84e9-2403-462c-b25f-945a927d1e66) Endpoint response body before transformations: {"statusCode": 500, "headers": {"Content-Type": "text/html; charset=utf-8", "Content-Length": "290"}

I can't figure out how to get the data in the format in "urllib3" that will be accepted like it is with "requests". Any thoughts? Thanks!

Python script to invoke an AWS API Gateway endpoint that requires IAM authentication. Uses boto for signing the request, and requests for issuing it.

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters

	#!/usr/bin/env python3
	"""
	Provides a simple Python wrapper for invoking an API Gateway endpoint using IAM signed requests.

	Example:
	python3 apigateway-invoke.py GET \
	https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/MethodName \| jq .
	"""

	try:
	from urllib.parse import urlparse, urlencode, parse_qs
	except ImportError:
	from urlparse import urlparse, parse_qs
	from urllib import urlencode

	import re
	import sys

	import requests
	from boto3 import Session
	from botocore.auth import SigV4Auth
	from botocore.awsrequest import AWSRequest


	def signing_headers(method, url_string, body):
	# Adapted from:
	# https://github.com/jmenga/requests-aws-sign/blob/master/requests_aws_sign/requests_aws_sign.py
	region = re.search("execute-api.(.*).amazonaws.com", url_string).group(1)
	url = urlparse(url_string)
	path = url.path or '/'
	querystring = ''
	if url.query:
	querystring = '?' + urlencode(
	parse_qs(url.query, keep_blank_values=True), doseq=True)

	safe_url = url.scheme + '://' + url.netloc.split(
	':')[0] + path + querystring
	request = AWSRequest(method=method.upper(), url=safe_url, data=body)
	SigV4Auth(Session().get_credentials(), "execute-api",
	region).add_auth(request)
	return dict(request.headers.items())


	if __name__ == "__main__":
	method = sys.argv[1]
	url = sys.argv[2]

	if not sys.stdin.isatty():
	body = sys.stdin.read()
	else:
	body = None

	r = requests.get(url, headers=signing_headers(method, url, body))
	print(r.content.decode("utf-8"))