Skip to main content


S3 sink publishes events as Amazon AWS S3 buckets.


CREATE SINK <NAME> WITH (type="s3", map.type="<STRING>", credential.provider.class="<STRING>", aws.access.key="<STRING>", aws.secret.key="<STRING>","<STRING>", aws.region="<STRING>", versioning.enabled="<BOOL>", object.path="<STRING>", storage.class="<STRING>", content.type="<STRING>", bucket.acl="<STRING>","<STRING>")

Query Parameters

NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
credential.provider.classAWS credential provider class to be used. If blank along with the username and the password, default credential provider will be used.EMPTY_STRINGSTRINGYesNo
aws.access.keyAWS access key. This cannot be used along with the credential.provider.classEMPTY_STRINGSTRINGYesNo
aws.secret.keyAWS secret key. This cannot be used along with the credential.provider.classEMPTY_STRINGSTRINGYesNo
bucket.nameName of the S3 bucketSTRINGNoNo
aws.regionThe region to be used to create the bucket.EMPTY_STRINGSTRINGYesNo
versioning.enabledFlag to enable versioning support in the bucket.falseBOOLYesNo
object.pathPath for each S3 object.STRINGNoYes
storage.classAWS storage class.standardSTRINGYesNo
content.typeContent type of the event.application/octet-streamSTRINGYesYes
bucket.aclAccess control list for the bucket.EMPTY_STRINGSTRINGYesNo
node.idThe node ID of the current publisher. This needs to be unique for each publisher instance as it may cause object overwrites while uploading the objects to same S3 bucket from different publishers.EMPTY_STRINGSTRINGYesNo

Example 1

CREATE SINK UserStream WITH (type='s3','user-stream-bucket',object.path='bar/users', credential.provider='', flush.size='3', map.type='json', enclosing.element='$.user', map.payload=""""{"name": "{{name}}", "age": {{age}}}"""") (name string, age int);

This creates a S3 bucket named user-stream-bucket. Then this will collect three events together and create a JSON object and save that in S3.