|
|
@@ -0,0 +1,86 @@ |
|
|
This is a mix between two sources: |
|
|
- http://www.alfielapeter.com/posts/8-transferring-s3-bucket-contents-between-accounts-with-s3cmd |
|
|
- and http://blog.techopsguru.com/2011/12/s3-bucket-copying-with-multiple-accounts.html |
|
|
|
|
|
basically the first resource is great but didn't work for me: I had to remove the trailing "/*" in the resource string to make it work. I also noticed that setting the policy on the source bucket was sufficient. |
|
|
In the end these are the exact steps I followed to copy data between two buckets on two accounts |
|
|
|
|
|
Basically the idea there is: |
|
|
- we allowe the *destination* account to read the source bucket (in the console for the *source* account) |
|
|
- we log as the *destination* and start the copy |
|
|
|
|
|
|
|
|
# Step 1 grab the user name for the *destination* account |
|
|
|
|
|
log into AWS with the *destination* account and go to "My Account" https://portal.aws.amazon.com/gp/aws/manageYourAccount |
|
|
The account number is on the top right below the search bar (under "Welcome XXX") and is like |
|
|
1234-1234-1234 (12 digits) |
|
|
|
|
|
For the rest I also assume you have and API key/password, if not: |
|
|
- go to the the console https://console.aws.amazon.com |
|
|
- click on your name on the top right > Security Credentials |
|
|
- Expand "Access Keys" and click on "Create New Access Key" |
|
|
You then obtain a file that looks like that: |
|
|
|
|
|
AWSAccessKeyId=AAAAAAAAAA |
|
|
AWSSecretKey=abababababababababababaabbabab |
|
|
|
|
|
The first value (AAAAAAAAAA) is the API key, the second (abababababababababababaabbabab) is the password. |
|
|
|
|
|
|
|
|
# Step 2 create the policy for the source bucket |
|
|
|
|
|
log into AWS with the *source* account and go to the AWS console for S3 |
|
|
|
|
|
select your bucket > Properties (on the right) > Permissions > Edit bucket policy |
|
|
You then see a dialog named "Bucket Policy Editor" |
|
|
|
|
|
on the bottom left of the dialog select "AWS policy generator". |
|
|
It will open a new page with a form, set the following values: |
|
|
|
|
|
- Select Type of Policy: S3 Bucket Policy |
|
|
- Effect: Allow |
|
|
- Principal: arn:aws:iam::123412341234:root (123412341234 is the *destination* account number without the dashes) |
|
|
- AWS Service: Amazon S3 |
|
|
- Actions: click "All Actions" |
|
|
- Amazon Resource Name: arn:aws:s3:::source-bucket (replace "source-bucket" with your *source* bucket name) |
|
|
|
|
|
The click "Add Statement" and then "Generate Policy" |
|
|
You then see a dialog with contents similar to: |
|
|
|
|
|
{ |
|
|
"Id": "Policy1383062241257", |
|
|
"Statement": [ |
|
|
{ |
|
|
"Sid": "Stmt1383062239775", |
|
|
"Action": "s3:*", |
|
|
"Effect": "Allow", |
|
|
"Resource": "arn:aws:s3:::source-bucket", |
|
|
"Principal": { |
|
|
"AWS": [ |
|
|
"arn:aws:iam::123412341234:root" |
|
|
] |
|
|
} |
|
|
} |
|
|
] |
|
|
} |
|
|
|
|
|
*cut and paste* the policy in the dialog of the previous page (the "Bucket Policy Editor") and click "Save" |
|
|
|
|
|
|
|
|
# Step 3 copy using s3cmd |
|
|
|
|
|
Install s3cmd, on the Mac: |
|
|
|
|
|
brew install s3cmd |
|
|
|
|
|
then configure your credentials for the *destination* account: |
|
|
|
|
|
s3cmd --configure |
|
|
|
|
|
It will ask for your API key and corresponding password, then a password to encode your credentials. Andswer yes (y) to |
|
|
test the connection and save the configuration. |
|
|
|
|
|
now you can copy: |
|
|
|
|
|
s3cmd sync --skip-existing --recursive s3://source-bucket s3://destination-bucket |
ushu revised this gist