Connector
Amazon S3
Last updated:
Visier can retrieve data through this connector if the following requirements are met.
Prerequisites
Create an integration user in Amazon S3.
Decide whether to use your own S3 bucket or send your files to Visier’s S3 bucket.
S3: When creating your Amazon S3 connector credentials in Visier, select S3 if you want to use your own S3 bucket.
CopyS3: When creating your Amazon S3 connector credentials in Visier, select CopyS3 if you want to send files to Visier’s S3 bucket. You can find Visier’s S3 access point during credential creation in Visier.
Integration user
Create an integration user in your source system and assign it the appropriate security access so that Visier can retrieve your source data through this user.
Access | Description |
|---|---|
If using your own S3 bucket, provide READ and LIST access | Assign Read Objects access to your files. Assign List Objects access to the S3 bucket you want Visier to retrieve. |
If pushing data to Visier’s S3 bucket, provide READ and WRITE access | Assign Read Objects access to your files. Assign Write Objects access to Visier’s S3 bucket. You can find Visier’s S3 access point during credential creation in Visier. |
Connector credentials
You need to provide the following credentials to Visier so that we can retrieve data from your source system.
If using your own S3 bucket, provide the following credentials.
Credential | Description | Example |
|---|---|---|
Bucket Name | The name of the S3 bucket to retrieve. | visier-data-bucket |
Bucket Region | The region in which the bucket is stored. For more information, see Managing AWS Regions. | us-west-2 |
Access Key | The integration user’s access key ID. | SPIA4BSBSDSNGG644POL |
Secret Key | The integration user’s secret access key. | xWSOf40JjoqRs9M7sRNe0lFNi3ilxge/Sbnx7J25 |
Path (Optional) | The path to the S3 bucket. If unspecified, Visier will retrieve data from the top level of the bucket. | s3-data/ |
If sending your files to Visier’s S3 bucket, provide the following credentials.
Note: You can find Visier’s S3 access point during credential creation in Visier.
Credential | Description | Example |
|---|---|---|
IAM Role | The integration user’s AWS Identity and Access Management (IAM) role. | arn:aws:iam::007816025659:role/myrole |
Data files
The Amazon S3 data connector only retrieves data stored in the accessible S3 bucket for use in Visier. The data in your S3 bucket must match Visier’s schema. The folder structure must be either path/date/filename.txt or path/filename_date.txt.
For other data, you must provide that data through alternative methods such as SFTP or Visier’s other data connectors.
How the connector loads data
Initially, data connectors retrieve a full history for each subject to generate an initial history of events. In each subsequent data retrieval, each connector handles the data load differently.
Full load: All data from the source system is loaded into Visier.
Delta load: Only changed data is loaded into Visier. A delta load may take either the entire history or a specific number of years of history for each changed record.
Snapshot load: All data at a particular point in time. A snapshot load does not contain historical records or show changes over time.
Partial load: A subset of data is loaded into Visier.
After the initial load, we recommend that you only provide data files for records that changed. However, you can send a full restatement if that's your preferred method.
Frequently asked questions
Do I need to authorize the integration user to access Visier’s S3 bucket?
If using your own S3 bucket, the integration user needs to be able to download data. If you’re pushing data to Visier’s S3 bucket, the integration user needs to be able to write data to Visier’s access point.
Does Visier remove my data files from Visier’s S3 bucket after the data is pulled into my Visier solution?
Yes. After a set amount of time, the data is vacuumed.
What time of user maintenance is required for the integration user?
None! If your organization has internal maintenance processes such as rotating keys on users, you will need to update the keys in the connector credentials you provided to Visier. You can update your credentials in the global workspace in the Data > Connector Credentials room.
Are Visier’s S3 buckets available in different regional Amazon Data Centers?
Yes.
If my AWS is in EMEA but I want to push data into Visier’s AWS Canada bucket, will I run into any issues?
Potentially. Because you’re pushing data across regions, your firewall may prevent you from pushing the data. If so, you might need to open a new pathway in your AWS instance.
Does Visier’s Amazon S3 data connector use S3’s webhook functionality?
No.