How to bulk ingest from AWS S3

Prev Next

Curator version supported from this article: 3.0


This guide explains how to create placeholder assets and ingest a large number of files from a single AWS S3 bucket (folder).

Bulk ingesting is helpful when you want to bring in all your assets at once and prefer to wait until everything is compiled in one location. This is especially useful for camera cards and multi-file assets.

Learn more about how to Prepare for AWS S3 for bulk ingest.

Ingesting the Manifest File

  1. From the Dashboard, open the S3 Bulk Ingest application.
    NOTE: If you do not see the application in your dashboard, please contact IPV support.

image.png

  1. Enter the required information, including both mandatory and optional metadata. Any optional fields can be completed if you wish. The metadata you provide will be applied to all assets created during the ingest.

If you’d like to change which metadata fields are available for tagging assets in an S3 Bulk Ingest, see Managing asset metadata.

S3 Bulk Ingest mandatory metadata:

  • S3 Bucket: Enter the bucket in which your manifest file was placed earlier.
  • S3 Manifest File Path: Enter the file name and extension of your Manifest file.

image.png

  1. When you’re ready, select Submit.
  2. Your placeholder assets will now be created.

Process Monitoring S3 Bulk Ingest

  1. Open the Process Engine application from the Dashboard.
  2. Navigate to the Processes tab.
  3. Search for the Plug-in - Send to GenericV4 workflow.
  4. Find the workflow that corresponds to your placeholder. Expanding the parent process using the drop-down arrows will display the child processes.

image.png

  1. Ensure that this matches the time you selected the Submit button.
  2. The process status will turn from yellow to green when complete.
  3. Once complete, navigate to the Clip Link application from the Dashboard. You should be able to see your assets in a placeholder state.
  4. Once a placeholder is created, a scavenge should gather the placeholder assets to process for the remaining ingest steps such as Proxy and Mediainfo.