CloudMap Service Discovery

The CloudMapServiceDecorator allows your service to register a service instance for your application.

For example, an application that provisions a SQS queue and an AWS Lambda function that consumes messages from that queue may need a way for the Lambda function to discover the dynamically provisioned queue.

Sparta supports an environment-based discovery service but that discovery is limited to a single Service.

The CloudMapServiceDecorator leverages the CloudMap service to support intra- and inter-service resource discovery.

Definition

The first step is to create an instance of the CloudMapServiceDecorator type that can be used to register additional resources.

import (
  spartaDecorators "github.com/mweagle/Sparta/decorator"
)

func main() {

  ...
  cloudMapDecorator, cloudMapDecoratorErr := spartaDecorators.NewCloudMapServiceDecorator(gocf.String("SpartaServices"),
    gocf.String("SpartaSampleCloudMapService"))
...
}

The first argument is the Cloud Map Namespace ID value to which the service (MyService) will publish.

The decorator satisfies the ServiceDecoratorHookHandler. The instance should be provided as a WorkflowHooks.ServiceDecorators element to MainEx as in:

func main() {
  // ...
  cloudMapDecorator, cloudMapDecoratorErr := spartaDecorators.NewCloudMapServiceDecorator(gocf.String("SpartaServices"),
    gocf.String("SpartaSampleCloudMapService"))
    
  workflowHooks := &sparta.WorkflowHooks{
    ServiceDecorators: []sparta.ServiceDecoratorHookHandler{
      cloudMapDecorator,
    },
  }

  // ...
  err := sparta.MainEx(awsName,
    "Simple Sparta application that demonstrates core functionality",
    lambdaFunctions,
    nil,
    nil,
    workflowHooks,
    false)
}

Registering

The returned CloudMapServiceDecorator instance satisfies the ServiceDecoratorHookHandler interface. When invoked, it updates the content of you CloudFormation template with the resources and permissions as described below. CloudMapServiceDecorator implicitly creates a new AWS::ServiceDiscovery::Service resource to which your resources will be published.

Lambda Functions

The CloudMapServiceDecorator.PublishLambda function publishes Lambda function information to the (NamespaceID, ServiceID) pair.

lambdaFn, _ := sparta.NewAWSLambda("Hello World",
    helloWorld,
    sparta.IAMRoleDefinition{})
cloudMapDecorator.PublishLambda("lambdaDiscoveryName", lambdaFn, nil)

The default properties published include the Lambda Outputs and Type information:

{
    "Id": "CloudMapResbe2b7c536074312c-VuIPjfjuFaoc",
    "Attributes": {
        "Arn": "arn:aws:lambda:us-west-2:123412341234:function:MyHelloWorldStack-123412341234_Hello_World",
        "Name": "lambdaDiscoveryName",
        "Ref": "MyHelloWorldStack-123412341234_Hello_World",
        "Type": "AWS::Lambda::Function"
    }
}

Other Resources

The CloudMapServiceDecorator.PublishResource function publishes arbitrary CloudFormation resource outputs information to the (NamespaceID, ServiceID) pair.

For instance, to publish SQS information in the context of a standard ServiceDecorator

func createSQSResourceDecorator(cloudMapDecorator *spartaDecorators.CloudMapServiceDecorator) sparta.ServiceDecoratorHookHandler {
  return sparta.ServiceDecoratorHookFunc(func(context map[string]interface{},
    serviceName string,
    template *gocf.Template,
    S3Bucket string,
    S3Key string,
    buildID string,
    awsSession *session.Session,
    noop bool,
    logger *logrus.Logger) error {

    sqsResource := &gocf.SQSQueue{}
    template.AddResource("SQSResource", sqsResource)
    return cloudMapDecorator.PublishResource("SQSResource",
      "SQSResource",
      sqsResource,
      nil)
  })
}

The default properties published include the SQS Outputs and Type information:

{
    "Id": "CloudMapRes21cf275e8bbbe136-CqWZ27gdLHf8",
    "Attributes": {
        "Arn": "arn:aws:sqs:us-west-2:123412341234:MyHelloWorldStack-123412341234-SQSResource-S9DWKIFKP14U",
        "Name": "SQSResource",
        "QueueName": "MyHelloWorldStack-123412341234-SQSResource-S9DWKIFKP14U",
        "Ref": "https://sqs.us-west-2.amazonaws.com/123412341234/MyHelloWorldStack-123412341234-SQSResource-S9DWKIFKP14U",
        "Type": "AWS::SQS::Queue"
    }
}

Enabling

Publishing instances to CloudMap only makes them available for other services to discover them. Call the EnableDiscoverySupport with your *sparta.LambdaAWSInfo instance as the only argument. This function updates your Lambda function’s environment to include the provisioned ServiceInstance and also the IAM role privileges to authorize:

  • servicediscovery:DiscoverInstances
  • servicediscovery:GetNamespace
  • servicediscovery:ListInstances
  • servicediscovery:GetService

For instance:

func main() {
  // ...

  lambdaFn, _ := sparta.NewAWSLambda("Hello World",
      helloWorld,
      sparta.IAMRoleDefinition{})

  // ...

  cloudMapDecorator.EnableDiscoverySupport(lambdaFn)
  
  // ...
}

Invoking

With the resources published and the lambda role properly updated, the last step is to dynamically discover the provisioned resources via CloudMap. Call DiscoverInstancesWithContext with the the set of key-value pairs to use for discovery as below:


func helloWorld(ctx context.Context) (string, error) {
    // ...

  props := map[string]string{
    "Type": "AWS::SQS::Queue",
  }
  results, resultsErr := spartaDecorators.DiscoverInstancesWithContext(ctx,
    props,
    logger)
  logger.WithFields(logrus.Fields{
    "Instances": results,
    "Error":     resultsErr,
    }).Info("Discovered instances!")
    // ...
}

Given the previous example of a single lambda function and an SQS-queue provisioning decorator, the DiscoverInstancesWithContext would return the matching instance with data similar to:

{
  "Instances": [
    {
      "Attributes": {
        "Arn": "arn:aws:sqs:us-west-2:123412341234:MyHelloWorldStack-123412341234-SQSResource-S9DWKIFKP14U",
        "Name": "SQSResource",
        "QueueName": "MyHelloWorldStack-123412341234-SQSResource-S9DWKIFKP14U",
        "Ref": "https://sqs.us-west-2.amazonaws.com/123412341234/MyHelloWorldStack-123412341234-SQSResource-S9DWKIFKP14U",
        "Type": "AWS::SQS::Queue"
      },
      "HealthStatus": "HEALTHY",
      "InstanceId": "CloudMapResd1a507076543ccd0-Fln1ITi5cf0y",
      "NamespaceName": "SpartaServices",
      "ServiceName": "SpartaSampleCloudMapService"
    }
  ]
}