User Input
This example demonstrates how to accept user input (delivered as HTTP query params) and return an expiring S3 URL to fetch content. The source for this is the s3ItemInfo function defined as part of the SpartaApplication.
Define the Lambda Function
Our function will accept two params:
bucketName
: The S3 bucket name storing the assetkeyName
: The S3 item key
Those params will be passed as part of the URL query string. The function will fetch the item metadata, generate an expiring URL for public S3 access, and return a JSON response body with the item data.
Because s3ItemInfo is expected to be invoked by the API Gateway, we’ll use the AWS Lambda Go type in the function signature:
import (
spartaEvents "github.com/mweagle/Sparta/aws/events"
)
func s3ItemInfo(ctx context.Context, apigRequest spartaEvents.APIGatewayRequest) (*itemInfoResponse, error) {
logger, _ := ctx.Value(sparta.ContextKeyLogger).(*logrus.Logger)
lambdaContext, _ := awsLambdaContext.FromContext(ctx)
logger.WithFields(logrus.Fields{
"RequestID": lambdaContext.AwsRequestID,
}).Info("Request received")
getObjectInput := &s3.GetObjectInput{
Bucket: aws.String(apigRequest.QueryParams["bucketName"]),
Key: aws.String(apigRequest.QueryParams["keyName"]),
}
awsSession := spartaAWS.NewSession(logger)
svc := s3.New(awsSession)
result, err := svc.GetObject(getObjectInput)
if nil != err {
return nil, err
}
presignedReq, _ := svc.GetObjectRequest(getObjectInput)
url, err := presignedReq.Presign(5 * time.Minute)
if nil != err {
return nil, err
}
return &itemInfoResponse{
S3: result,
URL: url,
}, nil
}
The sparta.APIGatewayRequest fields correspond to the Integration Response Mapping template discussed in the previous example (see the full mapping template here).
Once the event is unmarshaled, we can use it to fetch the S3 item info:
getObjectInput := &s3.GetObjectInput{
Bucket: aws.String(lambdaEvent.QueryParams["bucketName"]),
Key: aws.String(lambdaEvent.QueryParams["keyName"]),
}
Assuming there are no errors (including the case where the item does not exist), the remainder of the function fetches the data, generates a presigned URL, and returns a JSON response:
awsSession := spartaAWS.NewSession(logger)
svc := s3.New(awsSession)
result, err := svc.GetObject(getObjectInput)
if nil != err {
return nil, err
}
presignedReq, _ := svc.GetObjectRequest(getObjectInput)
url, err := presignedReq.Presign(5 * time.Minute)
if nil != err {
return nil, err
}
return &itemInfoResponse{
S3: result,
URL: url,
}, nil
Create the API Gateway
The next step is to create a new API instance via sparta.NewAPIGateway()
apiStage := sparta.NewStage("v1")
apiGateway := sparta.NewAPIGateway("SpartaImagerAPI", apiStage)
Create Lambda Binding
Next we create an sparta.LambdaAWSInfo
struct that references the s3ItemInfo
function:
var iamDynamicRole = sparta.IAMRoleDefinition{}
iamDynamicRole.Privileges = append(iamDynamicRole.Privileges,
sparta.IAMRolePrivilege{
Actions: []string{"s3:GetObject"},
Resource: resourceArn,
})
s3ItemInfoLambdaFn := sparta.HandleAWSLambda(sparta.LambdaName(s3ItemInfo),
s3ItemInfo,
iamDynamicRole)
s3ItemInfoOptions.Options = &sparta.LambdaFunctionOptions{
Description: "Get information about an item in S3 via querystring params",
MemorySize: 128,
Timeout: 10,
}
A few items to note here:
- We’re providing a custom
LambdaFunctionOptions
in case the request to S3 to get item metadata exceeds the default 3 second timeout. - We also add a custom
iamDynamicRole.Privileges
entry to thePrivileges
slice that authorizes the lambda function to only access objects in a single bucket (resourceArn).- This bucket ARN is externally created and the ARN provided to this code.
- While the API will accept any bucketName value, it is only authorized to access a single bucket.
Create Resources
The next step is to associate a URL path with the sparta.LambdaAWSInfo
struct that represents the s3ItemInfo
function. This will be the relative path component used to reference our lambda function via the API Gateway.
apiGatewayResource, _ := api.NewResource("/info", s3ItemInfoLambdaFn)
method, err := apiGatewayResource.NewMethod("GET", http.StatusOK)
if err != nil {
return nil, err
}
Whitelist Input
The final step is to add the whitelisted parameters to the Method definition.
// Whitelist query string params
method.Parameters["method.request.querystring.keyName"] = true
method.Parameters["method.request.querystring.bucketName"] = true
Note that the keynames in the method.Parameters
map must be of the form: method.request.{location}.{name} where location is one of:
querystring
path
header
See the REST documentation for more information.
Provision
With everything configured, let’s provision the stack:
go run application.go --level debug provision --s3Bucket $S3_BUCKET
and check the results.
Querying
As this Sparta application includes an API Gateway definition, the stack Outputs
includes the API Gateway URL:
INFO[0113] Stack output Description=API Gateway URL Key=APIGatewayURL Value=https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1
INFO[0113] Stack output Description=Sparta Home Key=SpartaHome Value=https://github.com/mweagle/Sparta
INFO[0113] Stack output Description=Sparta Version Key=SpartaVersion Value=0.1.0
Let’s fetch an item we know exists:
curl -vs "https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1/info?keyName=gopher.png&bucketName=somebucket-log"
* Trying 54.192.70.158...
* Connected to 0ux556ho77.execute-api.us-west-2.amazonaws.com (54.192.70.158) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/info?keyName=gopher.png&bucketName=somebucket-log HTTP/1.1
> Host: 0ux556ho77.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 1584
< Connection: keep-alive
< Date: Sun, 06 Dec 2015 02:35:03 GMT
< x-amzn-RequestId: f333f4bb-9bc1-11e5-afde-61a428c89049
< X-Cache: Miss from cloudfront
< Via: 1.1 2f31d4850470c56c3b326946dc542a6b.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: 6rBpqjmi7DPax7XOHTbxDx8-FfFfvI04m2_K-PxLWfYFor7WtIcdxA==
<
* Connection #0 to host 0ux556ho77.execute-api.us-west-2.amazonaws.com left intact
{"code":200,"status":"OK","headers":{"content-type":"application/json","date":"Sun, 06 Dec 2015 02:35:03 GMT","content-length":"1468"},"results":{"S3":{"AcceptRanges":"bytes","Body":{},"CacheControl":null,"ContentDisposition":null,"ContentEncoding":null,"ContentLanguage":null,"ContentLength":70372,"ContentRange":null,"ContentType":"image/png","DeleteMarker":null,"ETag":"\"ca1f746d6f232f87fca4e4d94ef6f3ab\"","Expiration":null,"Expires":null,"LastModified":"2015-11-09T15:38:01Z","Metadata":{},"MissingMeta":null,"ReplicationStatus":null,"RequestCharged":null,"Restore":null,"SSECustomerAlgorithm":null,"SSECustomerKeyMD5":null,"SSEKMSKeyId":null,"ServerSideEncryption":null,"StorageClass":null,"VersionId":null,"WebsiteRedirectLocation":null},"URL":"https://somebucket-log.s3-us-west-2.amazonaws.com/gopher.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ5KB2P6SQ4E7IMMQ%2F20151206%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20151206T023503Z&X-Amz-Expires=300&X-Amz-Security-Token=AQoDYXdzEFQawAK7vrGb%2BH9lw%2FhEHpR9Yg1KwPmmFcvyMzF7ewFBmxpOkfEM7gLZirMcFFexcxpWv%2F5CVAxpqjRf5FznOYJZHHoBqgmUcKPQZOpYKSbQG768zH5gMNdOANWin1COZU8DyuABrkJYL1bdFpwV7oHgrDmRz2G6oZqqOnfesRHW8WcehSXMV%2BcQFaAcO7IaIMAkRINMIDfxQaa%2FP8i8dbrcOfsEy6UABeaLKL3YgdZIouxcUUKzXQ6Pr4Cgrf0TAyRDAO1t6bVXzv6UFa6j00%2Fm0PYElni7xs5844UFAav%2B1weO2kX65ETzwUxBacAAnuzt%2BmTVPWeikhzgRnjBFn8mQjkZLCJklJJb6QHBO8dph2CSQsh47yw7%2BnexGjAu1y106AA2%2Bfa0WFYC552Q%2FrVVhKU7dejy%2B3jz%2F4LyWdnva9IvmCDVvY6zBQ%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=7d0e6663e043317b5611ddf4ae9f7514aff8c484a31deba524906ba50cbc6a2f"}}
Pretty printing the response body:
{
"code": 200,
"status": "OK",
"headers": {
"content-type": "application/json",
"date": "Sun, 06 Dec 2015 02:35:03 GMT",
"content-length": "1468"
},
"results": {
"S3": {
"AcceptRanges": "bytes",
"Body": {},
"CacheControl": null,
"ContentDisposition": null,
"ContentEncoding": null,
"ContentLanguage": null,
"ContentLength": 70372,
"ContentRange": null,
"ContentType": "image/png",
"DeleteMarker": null,
"ETag": "\"ca1f746d6f232f87fca4e4d94ef6f3ab\"",
"Expiration": null,
"Expires": null,
"LastModified": "2015-11-09T15:38:01Z",
"Metadata": {},
"MissingMeta": null,
"ReplicationStatus": null,
"RequestCharged": null,
"Restore": null,
"SSECustomerAlgorithm": null,
"SSECustomerKeyMD5": null,
"SSEKMSKeyId": null,
"ServerSideEncryption": null,
"StorageClass": null,
"VersionId": null,
"WebsiteRedirectLocation": null
},
"URL": "https://somebucket-log.s3-us-west-2.amazonaws.com/gopher.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ5KB2P6SQ4E7IMMQ%2F20151206%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20151206T023503Z&X-Amz-Expires=300&X-Amz-Security-Token=AQoDYXdzEFQawAK7vrGb%2BH9lw%2FhEHpR9Yg1KwPmmFcvyMzF7ewFBmxpOkfEM7gLZirMcFFexcxpWv%2F5CVAxpqjRf5FznOYJZHHoBqgmUcKPQZOpYKSbQG768zH5gMNdOANWin1COZU8DyuABrkJYL1bdFpwV7oHgrDmRz2G6oZqqOnfesRHW8WcehSXMV%2BcQFaAcO7IaIMAkRINMIDfxQaa%2FP8i8dbrcOfsEy6UABeaLKL3YgdZIouxcUUKzXQ6Pr4Cgrf0TAyRDAO1t6bVXzv6UFa6j00%2Fm0PYElni7xs5844UFAav%2B1weO2kX65ETzwUxBacAAnuzt%2BmTVPWeikhzgRnjBFn8mQjkZLCJklJJb6QHBO8dph2CSQsh47yw7%2BnexGjAu1y106AA2%2Bfa0WFYC552Q%2FrVVhKU7dejy%2B3jz%2F4LyWdnva9IvmCDVvY6zBQ%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=7d0e6663e043317b5611ddf4ae9f7514aff8c484a31deba524906ba50cbc6a2f"
}
}
Please see the first example for more information on the code
, status
, and headers
keys.
What about an item that we know doesn’t exist, but is in the bucket our lambda function has privileges to access:
curl -vs "https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1/info?keyName=gopher42.png&bucketName=somebucket-log"
* Trying 54.230.71.213...
* Connected to 0ux556ho77.execute-api.us-west-2.amazonaws.com (54.230.71.213) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/info?keyName=gopher42.png&bucketName=somebucket-log HTTP/1.1
> Host: 0ux556ho77.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Content-Length: 524
< Connection: keep-alive
< Date: Sun, 06 Dec 2015 02:40:14 GMT
< x-amzn-RequestId: ad5d94eb-9bc2-11e5-8fad-476a6cacabce
< X-Cache: Error from cloudfront
< Via: 1.1 29bfa9b96f4ea66dc02526ee845ca6b0.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: XoVLBjm1dgozZsNAEGk8Vy_a5PXMYNWRD6eKJJBcVTXrtMgMhiLNyQ==
<
* Connection #0 to host 0ux556ho77.execute-api.us-west-2.amazonaws.com left intact
{"errorMessage":"{\"code\":500,\"status\":\"Internal Server Error\",\"headers\":{\"content-type\":\"text/plain; charset=utf-8\",\"x-content-type-options\":\"nosniff\",\"date\":\"Sun, 06 Dec 2015 02:40:14 GMT\",\"content-length\":\"60\"},\"error\":\"AccessDenied: Access Denied\\n\\tstatus code: 403, request id: \\n\"}","errorType":"Error","stackTrace":["IncomingMessage.<anonymous> (/var/task/index.js:68:53)","IncomingMessage.emit (events.js:117:20)","_stream_readable.js:944:16","process._tickCallback (node.js:442:13)"]}
And finally, what if we try to access a bucket that our lambda function isn’t authorized to access:
curl -vs "https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1/info?keyName=gopher.png&bucketName=weagle"
* Trying 54.192.70.129...
* Connected to 0ux556ho77.execute-api.us-west-2.amazonaws.com (54.192.70.129) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/info?keyName=gopher.png&bucketName=weagle HTTP/1.1
> Host: 0ux556ho77.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Content-Length: 524
< Connection: keep-alive
< Date: Sun, 06 Dec 2015 02:42:52 GMT
< x-amzn-RequestId: 0be0fc4f-9bc3-11e5-b827-81d99c02192f
< X-Cache: Error from cloudfront
< Via: 1.1 400bdbea4e851ce61e7df8252da93d3f.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: M_7pB1UsW63xzh_9g37-CqNYDXfXlec0B6DV4bdkq3tbCANCOrTY6Q==
<
* Connection #0 to host 0ux556ho77.execute-api.us-west-2.amazonaws.com left intact
{"errorMessage":"{\"code\":500,\"status\":\"Internal Server Error\",\"headers\":{\"content-type\":\"text/plain; charset=utf-8\",\"x-content-type-options\":\"nosniff\",\"date\":\"Sun, 06 Dec 2015 02:42:52 GMT\",\"content-length\":\"60\"},\"error\":\"AccessDenied: Access Denied\\n\\tstatus code: 403, request id: \\n\"}","errorType":"Error","stackTrace":["IncomingMessage.<anonymous> (/var/task/index.js:68:53)","IncomingMessage.emit (events.js:117:20)","_stream_readable.js:944:16","process._tickCallback (node.js:442:13)"]}
Cleaning Up
Before moving on, remember to decommission the service via:
go run application.go delete
Wrapping Up
With this example we’ve walked through a simple example that whitelists user input, uses IAM Roles to limit what S3 buckets a lambda function may access, and returns JSON data to the caller.