Overview

• Concepts
• Hello World
• AWS Event Sources
• API-Gateway
• Static Web Sites
• Performance
• Visualization/testing/developer experience
• Observability/monitoring
• Alternative topologies
• Conclusion: pros/cons

Opinions

Serverless is the PaaS endgame

Repo is the unit of collaboration

Repo commit is the unit of promotion

AWS Lambda is a viable execution platform

Responsibilities rather than Roles

Go is fantastic

Image courtesy of Ashley McNamara

Concepts

• ServiceName: Stable, logical service identifier
• AWS Go Function: func(...)
• Privileges: Lambda IAM Role Privilege
• Permissions: Configure AWS event publishers (eg: S3, Dynamo, Kinesis)
• Dynamic Resources: Other AWS infrastructure provisioned by a Sparta Application

Machine Setup

• AWS Credentials (Administrator privileges)
• Writable S3 bucket for code artifacts
• Environment variables:
  • AWS_REGION
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

Hello World

Sparta Style

$ go get -u -v github.com/mweagle/SpartaHelloWorld
$ cd $GOPATH/src/github.com/mweagle/SpartaHelloWorld
$ go get -u -v ./...
$ go run main.go help

Options

    $ go run main.go --help
    Simple Sparta application that demonstrates
    core functionality

    Usage:
        main [command]

    Available Commands:
        delete      Delete service
        describe    Describe service
        execute     Execute
        explore     Interactively explore service
        help        Help about any command
        provision   Provision service
        version     Sparta framework version

    Flags:
        -f, --format string Log format [text, json] (default "text")
        -h, --help          help for main
        --ldflags string    Go linker string definition flags (https://golang.org/cmd/link/)
        -l, --level string  Log level [panic, fatal, error, warn, info, debug] (default "info")
        -n, --noop          Dry-run behavior only (do not perform mutations)
        -t, --tags string   Optional build tags for conditional compilation

    Use "main [command] --help" for more information about a command.

Trust

`go run main.go provision --level info --s3Bucket $S3BUCKET`

Verify

• Login to AWS Console
• Navigate to Lambda view, test function
• View CloudFormation Log output

Workflow

go run flow

S3 Event Sources

$ go get -u -v github.com/mweagle/SpartaImager
$ cd $GOPATH/src/github.com/mweagle/SpartaImager
$ go run main.go provision --level info --s3Bucket $S3_BUCKET

IAM Role

// Provision an IAM::Role as part of this application
var iamRole = sparta.IAMRoleDefinition{}
// Setup the ARN that includes all child keys
resourceArn := fmt.Sprintf("%s/*", s3EventBroadcasterBucket)
iamRole.Privileges = append(iamRole.Privileges,
    sparta.IAMRolePrivilege{
        Actions: []string{"s3:GetObject", "s3:PutObject"},
        Resource: resourceArn,
    })

Lambda & Permissions

var lambdaFunctions []*sparta.LambdaAWSInfo
lambdaFn := sparta.HandleAWSLambda(
    sparta.LambdaName(transformImage),
    transformImage,
    iamRole)
// The default timeout is 3 seconds - increase that to 30
// so the transform function doesn't timeout
lambdaFn.Options = &sparta.LambdaFunctionOptions{
    Description: "Stamp assets in S3",
    MemorySize:  128,
    Timeout:     30,
}
// S3 notification configuration
lambdaFn.Permissions = append(lambdaFn.Permissions,
    sparta.S3Permission{
    BasePermission: sparta.BasePermission{
        SourceArn: s3EventBroadcasterBucket,
        },
        Events: []string{"s3:ObjectCreated:*",
                "s3:ObjectRemoved:*"},
    })
lambdaFunctions = append(lambdaFunctions, lambdaFn)

Walkthrough

API Gateway

$ go get -u -v github.com/mweagle/SpartaImager
$ cd $GOPATH/src/github.com/mweagle/SpartaImager
$ go run main.go provision --level info --s3Bucket $S3_BUCKET

API Gateway

    apiStage := sparta.NewStage("v1")
    apiGateway := sparta.NewAPIGateway("SpartaImagerAPI",
        apiStage)
    apiGateway.CORSEnabled = true

HTTPS Route

    s3ItemInfoLambdaFn := sparta.HandleAWSLambda(
        sparta.LambdaName(s3ItemInfo),
        s3ItemInfo,
        iamDynamicRole)

    // Register the function with the API Gateway
    apiGatewayResource, _ := api.NewResource("/info",
        s3ItemInfoLambdaFn)
    method, err := apiGatewayResource.NewMethod("GET",
        http.StatusOK)
    if err != nil {
        return nil, err
    }
    // Whitelist query string params
    method.Parameters["method.request.querystring.keyName"] = true
    method.Parameters["method.request.querystring.bucketName"] = true
    lambdaFunctions = append(lambdaFunctions, s3ItemInfoLambdaFn)

Walkthrough

Websites

• Provision a complete SPA from a repo
• Connect API-Gateway with Lambda
  • Lightweight discovery
  • Ben Kehoe on Serverless Service Discovery

$ go get -u -v github.com/mweagle/SpartaHTML
$ cd $GOPATH/src/github.com/mweagle/SpartaHTML
$ go get -u -v ./...
$ go run main.go provision --level info --s3Bucket $S3_BUCKET

Site + API-G

// Register the function with the API Gateway
apiStage := sparta.NewStage("v1")
apiGateway := sparta.NewAPIGateway("SpartaHTML", apiStage)
// Enable CORS s.t. the S3 site can access the resources
apiGateway.CORSEnabled = true

// Provision a new S3 bucket with the resources in the supplied subdirectory
s3Site, _ := sparta.NewS3Site("./resources")

// Deploy it
sparta.Main("SpartaHTML",
    fmt.Sprintf("Sparta app that provisions a CORS-enabled API Gateway together with an S3 site"),
    spartaLambdaFunctions(apiGateway),
    apiGateway,
    s3Site)

Making Connections

{
    "APIGatewayURL": {
            "Description": "API Gateway URL",
            "Value": "https://p0ns30kpnd.execute-api.us-west-2.amazonaws.com/v1"
    }
}

Manifest.json

Performance

• Developer experience

Developer Experience

• standard: Full CloudFormation UPDATE_STACK cycle.
• --inplace: When safe, concurrent code updates directly via Lambda API

Walkthrough

Testing

• Use go test to test your lambda functions.
  • See sparta_test.go
  • Mock arbitrary event input
  • Mock API Gateway requests

Observability

• CloudWatch Dashboard, Metrics, Logs
• Honeycomb.io
• IOPipe
• AWS Lambda XRay

Lambda measurement fundamentals

CI/CD

• Produce a CodePipeline package with an optional --provision option
• Build a CI/CD pipeline with:
  • CodePipeline
  • CodeBuild
  • CloudFormation
• See Serverless, Serverfull, and Weaving Pipelines

Security, Scalability

• IAMRoles
• SpartaVault
  • Use KMS EnvelopeEncryption to manage secrets
• Serverless Artillery
  • Lambda max 1000 functions
  • Memory & IO positively correlated
  • Related: hey, goad.io, vegeta

Alternative Topologies

Use TemplateDecorators and WorkflowHooks to customize and extend your service

• SpartaDocker
  • Lambda->SQS->ECS Worker Pool
  • Flexibility is a Virtue
• SpartaGrafana
  • Lambda->Grafana on EC2
  • Serverless Monitoring

🏆

Pros & Cons

Pros

• Serverless can be TTM, financial, and operational win
• Go enables developer confidence, team productivity
• Sparta supports application evolution: from Serverless, to Container, to VM
  • Leverage other AWS primitives when needed
  • Codebase is operationally aware by design

Cons

• Go isn't first class - although binaries are
• Serverless can be operationally opaque
  • AWS operational tool usability is uneven
  • CloudFormation can be verbose
• Lacking phased rollout strategies (blue/green, canary)
• API-Gateway configuration complexity & performance
• Easy to create complex λ call graphs
• Single vendor currently

Learn More

• GitHub: Sparta
  • Change Notes
• Example Repos

Sparta

@mweagle on Twitter, Serverless Slack and Gophers Slack

Sign Up for Serverless Slack Forum here

PDF

Export these slides to PDF