How to connect Mule to AWS Simple Storage Service API?

Learn how to connect Mule to AWS Simple Storage Service API

The Anypoint Amazon S3 Connector allows developers to connect and access Amazon S3 API. It provides an interface with Amazon S3 to create buckets, store objects, download and use data with other AWS services and build applications that call for internet storage – all from within MuleSoft Anypoint Platform.

 

What is Amazon S3?

AWS S3 (storage for the Internet) is a cloud base object storage built to store and retrieve any amount of data at any time, from anywhere on the web – web sites and mobile apps, corporate applications, and data from IoT sensors or devices.

This blog post explains how to use Mulesoft AWS S3 Connector to connect to AWS S3 and download a specific object and how to download all objects from a bucket.

 

Prerequisites

In order to use the AWS S3 connector, you must have the following:

  1. AWS account
  2. AWS access key and secret key (credentials in a form of IAM)
  3. Anypoint Studio Enterprise edition and Mule Anypoint Amazon Connector

 

Installing and Configuring

  • In Anypoint Studio, click Help->Install New Software in the Studio menu.
  • Search for the connector and click Install.
  • Follow the prompts to install the connector.

 

Figure 1 - Install S3 Connector

Figure 1 – Install S3 Connector

Implementation

The Mule Project below is used to download an object from an AWS S3 Bucket using AWS S3 Connector.

Below you can find an explanation of the steps needed for integrating AWS S3 with Mulesoft Application.

  • Create a new Anypoint Studio Project and Flow.
  • Create a global element for Amazon S3 configuration which can be reused in all other flows across the Mule project
  • Configure the Amazon S3 connector by providing the Access Key and Secret Key

 

Figure 2 - AWS S3 Configuration

Figure 2 – AWS S3 Configuration

 

  • Test the connectivity in order to confirm that the connector is able to connect to AWS S3.
  • Drag an HTTP endpoint and create a new HTTP Listener Configuration. The service will be available on the following URL: http://localhost:8081/get-object
Figure 3 - HTTP endpoint

Figure 3 – HTTP endpoint

 

  • Drag an Amazon S3 connector into the flow and configure it by providing the items written below in order to download the object from Amazon S3 bucket and store locally
    • Connector Configuration (Link to the previously created S3 Configuration)
    • Operation (Get object content)
    • Bucket Name
    • Key

 

Figure 4 - Configuration for Amazon S3 endpoint

Figure 4 – Configuration for Amazon S3 endpoint

 

Figure 5 - AWS S3 bucket

Figure 5 – AWS S3 bucket

 

  • Drag Object To Byte Array Transformer into the flow in order to serialize the object
  • Add File Endpoint in order to write file on the local file system

 

Figure 6 - Download file locally

Figure 6 – Download file locally

 

  • Drag Set Payload Transformer into the flow in order to set the payload

 

Figure 7 - Set payload

Figure 7 – Set payload

 

  • Run the Mule Application and trigger the URL via Postman
  • Confirm that the file was downloaded locally

 

Figure 8 - Invoke Get Object Service

Figure 8 – Invoke Get Object Service

 

Figure 9 - Confirmation for the downloaded file

Figure 9 – Confirmation for the downloaded file

 

Figure 10 - xml representation

Figure 10 – XML representation

 

The following example explains how to list all objects and how to download all of them.

 

Figure 11 - List and download all objects

Figure 11 – List and download all objects

 

Figure 12 - xml representation for the second flow

Figure 12 – XML representation for the second flow

 

Figure 13 - Invoke get all objects service

Figure 13 – Invoke get all objects service

 

Figure 14 - downloaded files from s3

Figure 14 – downloaded files from s3

 

Conclusion

From the examples, we can see that Amazon S3 connector facilitates easy integration of your Mule application with Amazon S3 API.