github.com/mweagle/Sparta@v1.15.0/docs_source/content/reference/eventsources/s3.md

---
date: 2016-03-09T19:56:50+01:00
title: S3
weight: 10
---


In this section we'll walkthrough how to trigger your lambda function in response to S3 events.  This overview is based on the [SpartaImager](https://github.com/mweagle/SpartaImager) sample code if you'd rather jump to the end result.

# Goal

Assume we have an S3 bucket that stores images.  You've been asked to write a service that creates a duplicate image that includes a characteristic stamp overlay and store it in the same S3 bucket.

## Getting Started

We'll start with an empty lambda function and build up the needed functionality.

```go
import (
  awsLambdaEvents "github.com/aws/aws-lambda-go/events"
	awsLambdaContext "github.com/aws/aws-lambda-go/lambdacontext"
)

type transformedResponse struct {
	Bucket string
	Key    string
}

func transformImage(ctx context.Context, event awsLambdaEvents.S3Event) ([]transformedResponse, error) {
	logger, _ := ctx.Value(sparta.ContextKeyLogger).(*logrus.Logger)
	lambdaContext, _ := awsLambdaContext.FromContext(ctx)
	logger.WithFields(logrus.Fields{
		"RequestID":   lambdaContext.AwsRequestID,
		"RecordCount": len(event.Records),
  }).Info("Request received 👍")

```


Since the `transformImage` is expected to be triggered by S3 event changes, we can transparently unmarshal
the incoming request into an [S3Event](https://github.com/aws/aws-lambda-go/blob/master/events/s3.go#L9)
defined by the AWS Go Lambda SDK.

S3 events are delivered in batches, via lists of [EventRecords](https://godoc.org/github.com/mweagle/Sparta/aws/s3#EventRecord), so we'll need to process each record.

```go
for _, eachRecord := range event.Records {
  // What happened?
  switch eachRecord.EventName {
  case "ObjectCreated:Put":
    {
      err = stampImage(eachRecord.S3.Bucket.Name, eachRecord.S3.Object.Key, logger)
    }
  case "s3:ObjectRemoved:Delete":
    {
      // Delete stamped image
    }
  default:
    {
      logger.Info("Unsupported event: ", eachRecord.EventName)
    }
  }

  //
  if err != nil {
    logger.Error("Failed to process event: ", err.Error())
    http.Error(w, err.Error(), http.StatusInternalServerError)
  }
}
```

The [stampImage](https://github.com/mweagle/SpartaImager/blob/master/application.go#L57) function does most of the work, fetching the S3 image to memory,
applying the stamp, and putting the transformed content back to S3 with a new name.  It uses a simple **xformed_** keyname prefix to identify
items which have already been stamped & prevents an "event-storm" from being triggered.  This simple approach is acceptable for an example,
but in production you should use a more durable approach.

## Sparta Integration

With the core of the `transformImage` complete, the next step is to integrate the **go** function with Sparta.  This is performed by the [imagerFunctions](https://github.com/mweagle/SpartaImager/blob/master/application.go#L200) source.

Our lambda function needs to both *Get* and *Put* items back to an S3 bucket, so we need an IAM Role that grants those privileges under which the function will execute:

```go
// Provision an IAM::Role as part of this application
var iamRole = sparta.IAMRoleDefinition{}

// Setup the ARN that includes all child keys
resourceArn := fmt.Sprintf("%s/*", s3EventBroadcasterBucket)
iamRole.Privileges = append(iamRole.Privileges, sparta.IAMRolePrivilege{
  Actions: []string{"s3:GetObject",
    "s3:PutObject",
  },
  Resource: resourceArn,
})
```

The `s3EventBroadcasterBucket` param is the ARN of the S3 bucket that will trigger your lambda function (eg: _arn:aws:s3:::MyImagingS3Bucket_).

With the IAM Role defined, we can create the Sparta lambda function for `transformImage`:

```go
// The default timeout is 3 seconds - increase that to 30 seconds s.t. the
// transform lambda doesn't fail early.
transformOptions := &sparta.LambdaFunctionOptions{
  Description: "Stamp assets in S3",
  MemorySize:  128,
  Timeout:     30,
}
lambdaFn, _ := sparta.NewAWSLambda(sparta.LambdaName(transformImage),
  transformImage,
  iamRole)
lambdaFn.Options = transformOptions
```

It typically takes more than 3 seconds to apply the transform, so we increase the execution timeout and provision a new
lambda function using the `iamRole` we defined earlier.

## Event Source Registration

If we were to deploy this Sparta application, the `transformImage` function would have the ability to *Get* and *Put* back
to the `s3EventBroadcasterBucket`, but would not be invoked in response to events triggered by that bucket.  To register
for state change events, we need to configure the lambda's [Permissions](http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html):

```go
//////////////////////////////////////////////////////////////////////////////
// S3 configuration
//
lambdaFn.Permissions = append(lambdaFn.Permissions, sparta.S3Permission{
  BasePermission: sparta.BasePermission{
    SourceArn: s3EventBroadcasterBucket,
  },
  Events: []string{"s3:ObjectCreated:*", "s3:ObjectRemoved:*"},
})
lambdaFunctions = append(lambdaFunctions, lambdaFn)
```

When `Sparta` generates the CloudFormation template, it scans for `Permission` configurations.
For [push based sources](http://docs.aws.amazon.com/lambda/latest/dg/intro-invocation-modes.html) like S3, Sparta uses that
service's APIs to register your lambda function as a publishing target for events.  This remote registration is handled
 automatically by CustomResources added to the CloudFormation template.

# Wrapping Up

With the `lambdaFn` fully defined, we can provide it to `sparta.Main()` and deploy our service.  The workflow below is shared by all S3-triggered lambda functions:

  * Define the lambda function (`transformImage`).
  * Implement the associated business logic  (`stampImage`).
  * If needed, create the required [IAMRoleDefinition](https://godoc.org/github.com/mweagle/Sparta*IAMRoleDefinition) with appropriate privileges.
  * Provide the lambda function & IAMRoleDefinition to `sparta.NewAWSLambda()`
  * Add the necessary [Permissions](https://godoc.org/github.com/mweagle/Sparta#LambdaAWSInfo) to the `LambdaAWSInfo` struct so that the lambda function is triggered.

The [SpartaImager](https://github.com/mweagle/SpartaImager) repo contains the full code, and includes [API Gateway](/reference/apigateway) support that allows you to publicly fetch the stamped image via an expiring S3 URL.

## Other Resources

  * The AWS docs have an excellent [S3 event source](http://docs.aws.amazon.com/lambda/latest/dg/getting-started-amazons3-events.html) walkthrough.