Do I create an Import Job or an Export Job?

To get data into S3, you create an import job. Here are the details needed for an import job (configured via the console)

How do  I attach the snowball device to my network?

The device only attaches to a network interface. You want to attach it within the same subnet as the workload (your backup / data files) that need to be transferred. Basically, minimize the number of hops between the device and the backed up storage.

Job Details
Each import or export job for Snowball is defined by the details that you specify when it’s created. The following list describes all the details of a job.
  • Job name – A name for the job, containing alphanumeric characters, spaces, and any Unicode special characters.
  • Job type – The type of job, either import or export.
  • Job ID – A unique 39-character label that identifies your job. The job ID appears at the bottom of the shipping label that appears on the E Ink display, and in the name of a job’s manifest file.
  • Created date – The date that you created this job.
  • Shipping speed – Speed options are based on region. For more information, see Shipping Speeds.
  • IAM role ARN – This Amazon Resource Name (ARN) is the AWS Identity and Access Management (IAM) role that is created during job creation with write permissions for your Amazon S3 buckets. The creation process is automatic, and the IAM role that you allow Snowball to assume is only used to copy your data between your Amazon S3 buckets and the Snowball. For more information, see Creating an IAM Role for Snowball.
  • AWS KMS key – In Snowball, AWS Key Management Service (AWS KMS) encrypts the keys on each Snowball. When you create your job, you also choose or create an ARN for an AWS KMS encryption key that you own. For more information, see AWS Key Management Service in Snowball.
  • Snowball capacity – In the US regions, Snowballs come in two sizes: 50 TB (42 TB usable capacity) and 80 TB (72 TB usable capacity). All other regions have the 80 TB Snowballs only.
  • Storage service – The AWS storage service associated with this job, in this case Amazon S3.
  • Resources – The AWS storage service resources associated with your job. In this case, these are the Amazon S3 buckets that your data is transferred to or from.

Create a Snow Job within AWS Console.

  • Each job is either 50 TB or 80 TB (max)
  • Data Consistency (Checksum) checks are performed during data import by AWS
  • Pricing is $300 per job.
  • You get 10 days to complete your dump and send it over to AWS (included in the $300).
  • After that, it is $30 per additional day.  You CAN cancel the job and return the devices.

What can go wrong?

  • Data Consistency Checks, especially during Data dumps (on client side), can be an issue with low bandwidth transfers.  One should ensure that there is a HUGE transfer bandwidth ( 1 GBps  or 10  GBps recommended for TB sizes)

What is the nature of the workload that you are backing up?

  • If you require the S3 data to be managed via an on prem tool (e.g. a Veeam instance), then snowball to S3 will work, but you cannot manage the S3 storage from your Veeam instance.
  • File based workloads are not a problem. e.g. SQL  Database backups are based on files – and files to snowball to s3 is not problematic.
  • Tools such as Commvault can also manage the S3 objects directly.

File Transfer Time Calculator

E.g. for 112 TB

  • A 100 MBps pipe will take 14 days, 22 hours
  • 1 GBps will take 1 day, 11 hours
What about copying entire folder structures to S3?
The folder structure is retained in the Snowball device and onto s3. However, there is no ‘structure’, it is just prefixes to the s3 object names that serve as folders.
How do you SYNC new data with the S3 target?
You need to use the S3 sync API command (not the S3 copy command). The SYNC command can be executed via the CLI or via a tool like CloudBerry.
Best Practices for using Snowball to AWS S3
General Best Practices
docs.aws.amazon.com
Find best practices for working with the AWS Snowball Edge device, including recommendations for security, resource management, and performance.
Data Migration Best Practices
aws.amazon.com
AWS Snowball Edge enables petabyte-scale offline data migrations from on-premises storage (SAN, NAS, parallel file systems) or databases into Amazon S3. AWS Snowball Edge is part of the AWS Snow Family and the greater AWS portfolio of data transfer services that can be used for data migration. The AWS portfolio of data transfer services also […]

 


Need an experienced AWS/GCP/Azure Professional to help out with your Public Cloud Strategy? Set up a time with Anuj Varma.

Anuj holds professional certifications in Google Cloud, AWS as well as certifications in Docker and App Performance Tools such as New Relic. He specializes in Cloud Security, Data Encryption and Container Technologies.

Initial Consultation

Anuj Varma – who has written posts on Anuj Varma, Hands-On Technology Architect, Clean Air Activist.

 Twitter Linkedin