This example demonstrates how to accept user input (delivered as HTTP query params) and return an expiring S3 URL to fetch content. The source for this is the s3ItemInfo function defined as part of the SpartaApplication.
Our function will accept two params:
bucketName : The S3 bucket name storing the assetkeyName : The S3 item keyThose params will be passed as part of the URL query string. The function will fetch the item metadata, generate an expiring URL for public S3 access, and return a JSON response body with the item data.
Because s3ItemInfo is expected to be invoked by the API Gateway, we’ll use the AWS Lambda Go type in the function signature:
import (
spartaEvents "github.com/mweagle/Sparta/aws/events"
)
func s3ItemInfo(ctx context.Context, apigRequest spartaEvents.APIGatewayRequest) (*itemInfoResponse, error) {
logger, _ := ctx.Value(sparta.ContextKeyLogger).(*logrus.Logger)
lambdaContext, _ := awsLambdaContext.FromContext(ctx)
logger.WithFields(logrus.Fields{
"RequestID": lambdaContext.AwsRequestID,
}).Info("Request received")
getObjectInput := &s3.GetObjectInput{
Bucket: aws.String(apigRequest.QueryParams["bucketName"]),
Key: aws.String(apigRequest.QueryParams["keyName"]),
}
awsSession := spartaAWS.NewSession(logger)
svc := s3.New(awsSession)
result, err := svc.GetObject(getObjectInput)
if nil != err {
return nil, err
}
presignedReq, _ := svc.GetObjectRequest(getObjectInput)
url, err := presignedReq.Presign(5 * time.Minute)
if nil != err {
return nil, err
}
return &itemInfoResponse{
S3: result,
URL: url,
}, nil
}
The sparta.APIGatewayRequest fields correspond to the Integration Response Mapping template discussed in the previous example (see the full mapping template here).
Once the event is unmarshaled, we can use it to fetch the S3 item info:
getObjectInput := &s3.GetObjectInput{
Bucket: aws.String(lambdaEvent.QueryParams["bucketName"]),
Key: aws.String(lambdaEvent.QueryParams["keyName"]),
}
Assuming there are no errors (including the case where the item does not exist), the remainder of the function fetches the data, generates a presigned URL, and returns a JSON response:
awsSession := spartaAWS.NewSession(logger)
svc := s3.New(awsSession)
result, err := svc.GetObject(getObjectInput)
if nil != err {
return nil, err
}
presignedReq, _ := svc.GetObjectRequest(getObjectInput)
url, err := presignedReq.Presign(5 * time.Minute)
if nil != err {
return nil, err
}
return &itemInfoResponse{
S3: result,
URL: url,
}, nil
The next step is to create a new API instance via sparta.NewAPIGateway()
apiStage := sparta.NewStage("v1")
apiGateway := sparta.NewAPIGateway("SpartaImagerAPI", apiStage)
Next we create an sparta.LambdaAWSInfo struct that references the s3ItemInfo function:
var iamDynamicRole = sparta.IAMRoleDefinition{}
iamDynamicRole.Privileges = append(iamDynamicRole.Privileges,
sparta.IAMRolePrivilege{
Actions: []string{"s3:GetObject"},
Resource: resourceArn,
})
s3ItemInfoLambdaFn := sparta.HandleAWSLambda(sparta.LambdaName(s3ItemInfo),
s3ItemInfo,
iamDynamicRole)
s3ItemInfoOptions.Options = &sparta.LambdaFunctionOptions{
Description: "Get information about an item in S3 via querystring params",
MemorySize: 128,
Timeout: 10,
}
A few items to note here:
LambdaFunctionOptions in case the request to S3 to get item metadata exceeds the default 3 second timeout.iamDynamicRole.Privileges entry to the Privileges slice that authorizes the lambda function to only access objects in a single bucket (resourceArn).
The next step is to associate a URL path with the sparta.LambdaAWSInfo struct that represents the s3ItemInfo function. This will be the relative path component used to reference our lambda function via the API Gateway.
apiGatewayResource, _ := api.NewResource("/info", s3ItemInfoLambdaFn)
method, err := apiGatewayResource.NewMethod("GET", http.StatusOK)
if err != nil {
return nil, err
}
The final step is to add the whitelisted parameters to the Method definition.
// Whitelist query string params
method.Parameters["method.request.querystring.keyName"] = true
method.Parameters["method.request.querystring.bucketName"] = true
Note that the keynames in the method.Parameters map must be of the form: method.request.{location}.{name} where location is one of:
querystringpathheaderSee the REST documentation for more information.
With everything configured, let’s provision the stack:
go run application.go --level debug provision --s3Bucket $S3_BUCKET
and check the results.
As this Sparta application includes an API Gateway definition, the stack Outputs includes the API Gateway URL:
INFO[0113] Stack output Description=API Gateway URL Key=APIGatewayURL Value=https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1
INFO[0113] Stack output Description=Sparta Home Key=SpartaHome Value=https://github.com/mweagle/Sparta
INFO[0113] Stack output Description=Sparta Version Key=SpartaVersion Value=0.1.0
Let’s fetch an item we know exists:
curl -vs "https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1/info?keyName=gopher.png&bucketName=somebucket-log"
* Trying 54.192.70.158...
* Connected to 0ux556ho77.execute-api.us-west-2.amazonaws.com (54.192.70.158) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/info?keyName=gopher.png&bucketName=somebucket-log HTTP/1.1
> Host: 0ux556ho77.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 1584
< Connection: keep-alive
< Date: Sun, 06 Dec 2015 02:35:03 GMT
< x-amzn-RequestId: f333f4bb-9bc1-11e5-afde-61a428c89049
< X-Cache: Miss from cloudfront
< Via: 1.1 2f31d4850470c56c3b326946dc542a6b.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: 6rBpqjmi7DPax7XOHTbxDx8-FfFfvI04m2_K-PxLWfYFor7WtIcdxA==
<
* Connection #0 to host 0ux556ho77.execute-api.us-west-2.amazonaws.com left intact
{"code":200,"status":"OK","headers":{"content-type":"application/json","date":"Sun, 06 Dec 2015 02:35:03 GMT","content-length":"1468"},"results":{"S3":{"AcceptRanges":"bytes","Body":{},"CacheControl":null,"ContentDisposition":null,"ContentEncoding":null,"ContentLanguage":null,"ContentLength":70372,"ContentRange":null,"ContentType":"image/png","DeleteMarker":null,"ETag":"\"ca1f746d6f232f87fca4e4d94ef6f3ab\"","Expiration":null,"Expires":null,"LastModified":"2015-11-09T15:38:01Z","Metadata":{},"MissingMeta":null,"ReplicationStatus":null,"RequestCharged":null,"Restore":null,"SSECustomerAlgorithm":null,"SSECustomerKeyMD5":null,"SSEKMSKeyId":null,"ServerSideEncryption":null,"StorageClass":null,"VersionId":null,"WebsiteRedirectLocation":null},"URL":"https://somebucket-log.s3-us-west-2.amazonaws.com/gopher.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ5KB2P6SQ4E7IMMQ%2F20151206%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20151206T023503Z&X-Amz-Expires=300&X-Amz-Security-Token=AQoDYXdzEFQawAK7vrGb%2BH9lw%2FhEHpR9Yg1KwPmmFcvyMzF7ewFBmxpOkfEM7gLZirMcFFexcxpWv%2F5CVAxpqjRf5FznOYJZHHoBqgmUcKPQZOpYKSbQG768zH5gMNdOANWin1COZU8DyuABrkJYL1bdFpwV7oHgrDmRz2G6oZqqOnfesRHW8WcehSXMV%2BcQFaAcO7IaIMAkRINMIDfxQaa%2FP8i8dbrcOfsEy6UABeaLKL3YgdZIouxcUUKzXQ6Pr4Cgrf0TAyRDAO1t6bVXzv6UFa6j00%2Fm0PYElni7xs5844UFAav%2B1weO2kX65ETzwUxBacAAnuzt%2BmTVPWeikhzgRnjBFn8mQjkZLCJklJJb6QHBO8dph2CSQsh47yw7%2BnexGjAu1y106AA2%2Bfa0WFYC552Q%2FrVVhKU7dejy%2B3jz%2F4LyWdnva9IvmCDVvY6zBQ%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=7d0e6663e043317b5611ddf4ae9f7514aff8c484a31deba524906ba50cbc6a2f"}}
Pretty printing the response body:
{
"code": 200,
"status": "OK",
"headers": {
"content-type": "application/json",
"date": "Sun, 06 Dec 2015 02:35:03 GMT",
"content-length": "1468"
},
"results": {
"S3": {
"AcceptRanges": "bytes",
"Body": {},
"CacheControl": null,
"ContentDisposition": null,
"ContentEncoding": null,
"ContentLanguage": null,
"ContentLength": 70372,
"ContentRange": null,
"ContentType": "image/png",
"DeleteMarker": null,
"ETag": "\"ca1f746d6f232f87fca4e4d94ef6f3ab\"",
"Expiration": null,
"Expires": null,
"LastModified": "2015-11-09T15:38:01Z",
"Metadata": {},
"MissingMeta": null,
"ReplicationStatus": null,
"RequestCharged": null,
"Restore": null,
"SSECustomerAlgorithm": null,
"SSECustomerKeyMD5": null,
"SSEKMSKeyId": null,
"ServerSideEncryption": null,
"StorageClass": null,
"VersionId": null,
"WebsiteRedirectLocation": null
},
"URL": "https://somebucket-log.s3-us-west-2.amazonaws.com/gopher.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ5KB2P6SQ4E7IMMQ%2F20151206%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20151206T023503Z&X-Amz-Expires=300&X-Amz-Security-Token=AQoDYXdzEFQawAK7vrGb%2BH9lw%2FhEHpR9Yg1KwPmmFcvyMzF7ewFBmxpOkfEM7gLZirMcFFexcxpWv%2F5CVAxpqjRf5FznOYJZHHoBqgmUcKPQZOpYKSbQG768zH5gMNdOANWin1COZU8DyuABrkJYL1bdFpwV7oHgrDmRz2G6oZqqOnfesRHW8WcehSXMV%2BcQFaAcO7IaIMAkRINMIDfxQaa%2FP8i8dbrcOfsEy6UABeaLKL3YgdZIouxcUUKzXQ6Pr4Cgrf0TAyRDAO1t6bVXzv6UFa6j00%2Fm0PYElni7xs5844UFAav%2B1weO2kX65ETzwUxBacAAnuzt%2BmTVPWeikhzgRnjBFn8mQjkZLCJklJJb6QHBO8dph2CSQsh47yw7%2BnexGjAu1y106AA2%2Bfa0WFYC552Q%2FrVVhKU7dejy%2B3jz%2F4LyWdnva9IvmCDVvY6zBQ%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=7d0e6663e043317b5611ddf4ae9f7514aff8c484a31deba524906ba50cbc6a2f"
}
}
Please see the first example for more information on the code, status, and headers keys.
What about an item that we know doesn’t exist, but is in the bucket our lambda function has privileges to access:
curl -vs "https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1/info?keyName=gopher42.png&bucketName=somebucket-log"
* Trying 54.230.71.213...
* Connected to 0ux556ho77.execute-api.us-west-2.amazonaws.com (54.230.71.213) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/info?keyName=gopher42.png&bucketName=somebucket-log HTTP/1.1
> Host: 0ux556ho77.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Content-Length: 524
< Connection: keep-alive
< Date: Sun, 06 Dec 2015 02:40:14 GMT
< x-amzn-RequestId: ad5d94eb-9bc2-11e5-8fad-476a6cacabce
< X-Cache: Error from cloudfront
< Via: 1.1 29bfa9b96f4ea66dc02526ee845ca6b0.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: XoVLBjm1dgozZsNAEGk8Vy_a5PXMYNWRD6eKJJBcVTXrtMgMhiLNyQ==
<
* Connection #0 to host 0ux556ho77.execute-api.us-west-2.amazonaws.com left intact
{"errorMessage":"{\"code\":500,\"status\":\"Internal Server Error\",\"headers\":{\"content-type\":\"text/plain; charset=utf-8\",\"x-content-type-options\":\"nosniff\",\"date\":\"Sun, 06 Dec 2015 02:40:14 GMT\",\"content-length\":\"60\"},\"error\":\"AccessDenied: Access Denied\\n\\tstatus code: 403, request id: \\n\"}","errorType":"Error","stackTrace":["IncomingMessage.<anonymous> (/var/task/index.js:68:53)","IncomingMessage.emit (events.js:117:20)","_stream_readable.js:944:16","process._tickCallback (node.js:442:13)"]}
And finally, what if we try to access a bucket that our lambda function isn’t authorized to access:
curl -vs "https://0ux556ho77.execute-api.us-west-2.amazonaws.com/v1/info?keyName=gopher.png&bucketName=weagle"
* Trying 54.192.70.129...
* Connected to 0ux556ho77.execute-api.us-west-2.amazonaws.com (54.192.70.129) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-west-2.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> GET /v1/info?keyName=gopher.png&bucketName=weagle HTTP/1.1
> Host: 0ux556ho77.execute-api.us-west-2.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Content-Length: 524
< Connection: keep-alive
< Date: Sun, 06 Dec 2015 02:42:52 GMT
< x-amzn-RequestId: 0be0fc4f-9bc3-11e5-b827-81d99c02192f
< X-Cache: Error from cloudfront
< Via: 1.1 400bdbea4e851ce61e7df8252da93d3f.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: M_7pB1UsW63xzh_9g37-CqNYDXfXlec0B6DV4bdkq3tbCANCOrTY6Q==
<
* Connection #0 to host 0ux556ho77.execute-api.us-west-2.amazonaws.com left intact
{"errorMessage":"{\"code\":500,\"status\":\"Internal Server Error\",\"headers\":{\"content-type\":\"text/plain; charset=utf-8\",\"x-content-type-options\":\"nosniff\",\"date\":\"Sun, 06 Dec 2015 02:42:52 GMT\",\"content-length\":\"60\"},\"error\":\"AccessDenied: Access Denied\\n\\tstatus code: 403, request id: \\n\"}","errorType":"Error","stackTrace":["IncomingMessage.<anonymous> (/var/task/index.js:68:53)","IncomingMessage.emit (events.js:117:20)","_stream_readable.js:944:16","process._tickCallback (node.js:442:13)"]}
Before moving on, remember to decommission the service via:
go run application.go delete
With this example we’ve walked through a simple example that whitelists user input, uses IAM Roles to limit what S3 buckets a lambda function may access, and returns JSON data to the caller.