Community

27 Dec

s3fox does not create valid export manifest files

S3fox provides a handy utility for creating import manifest and signature files; however, it doesn’t seem to properly create export manifests.  In fact, it seems like it only creates import manifest files.  The version I used was 0.4.8.

Here’s a screenshot of the s3fox create manifest interface:

s3fox configuration screen

Note there is nowhere in the GUI to specify export or import.  Note also in the manifest preview on the right that the exportBucket token is not visible.  Looking at the spec for an export  the exportBucket token is required for AWS to know where to get your data.  Without it, you’ll get a note from Amazon asking what you’re trying to do.

So, if you want to export data from an S3 bucket to a USB drive, it’s best to manually create a manifest file according to the AWS spec.

If you manually create an export manifest, you’ll also need to manually create a signature file.  The procedure for manually creating signature files was not immediately obvious (to me, anyway) on the AWS site, but I found it on Jeorge Escobar’s blog (thanks, Jeorge!)

You’ll need to download the signature tool then in a command line terminal, change to the signature tool directory and run the following command:

java -jar lib/AWSImportClientSignatureTool-1.0.jar <manifest file> <job ID> <SIGNATURE file>

Once you have the manifest and signature file, you can simply follow the documented procedure and expect to see your USB drive in a few days.

Related posts from the blog:

  1. Oracle Correction to Export/Import
    If you don’t get alerts regularly from Oracle Support you may have missed this notice: If you conducted...
  2. Recovering a Schema From InnoDB .frm Files
    Sometimes you find yourself in a bad situation where your only hope of recovering your InnoDB data lies...
  3. EC2 persistent boots with pivot root
    Amazon recently allowed Elastic Block Store to boot persistent images. However, there are two concerns I have with...
  4. Using Checksums to Ensure Table Consistency in MySQL
    If you’re using MySQL replication, chances are your master and slave databases aren’t entirely consistent. There are a...
  5. Exploring Oracle 11g Tablespace Encryption
    Tablespace encryption encrypts data at the datafile level to keep people from being able to peek at the...

3 Responses to “s3fox does not create valid export manifest files”

  1. cedwards 18. Jan, 2010 at 1:00 pm #

    Sure – here’s one that worked fine:

    manifestVersion:1.2
    accessKeyId:*********************
    deviceId:963×0-4rbd1-305
    returnAddress:
    name:Charles Edwards
    street1:1408 4th Ave
    street2:Suite 200
    city:Seattle
    stateOrProvince:WA
    postalCode:98103
    phoneNumber:(206) 508-4760
    country:USA
    operations:
    - exportBucket: mybucket
    targetDirectory: /
    logPrefix:mylogs/
    fileSystem: FAT32

    If you follow the manual instructions on Amazon’s documentation, it works OK. S3 just doesn’t do a very good job of translating.

  2. mbell 18. Jan, 2010 at 1:08 pm #

    Thanks for the tips. Do you have an example of your export manifest. Most of the documentation applies to Importing. I want to do a backup also and looking for example.

  3. Andy 24. Feb, 2010 at 12:49 pm #

    You can also use CloudBerry Explorer to create import/ export manifests . It is freeware.

Leave a Reply