What does “extended metadata backup” mean?

One of the features we’ve mentioned several times in the new backup engine that powers Jungle Disk Server Edition (and is also present in Desktop/Workgroup 3.0) is “extended filesystem metadata backup”. But what exactly does that mean?

The previous versions of Jungle Disk focused on backing up your file data, which is the most important part of your files. On Mac, we also backed up resource forks which are a method of storing additional data alongside a file. However on modern filesystems, there is a lot more than just file data on your filesystem. There is lots of metadata information that goes along with your files and can be important to backup and restore in many situations. One example is security information – most desktop users don’t really know about file ownership, security information, and ACLs, but on servers that type of information is important and if lost it could lead to a security leak or permission issues. On multi-user Desktop systems it can be important as well to prevent other users from getting access to your files.

The new Backup Vaults in Jungle Disk 3.0 were designed from the outset to be able to backup and restore virtually any type of filesystem metadata information. Among the metadata that we can backup and restore:

On Windows:

  • NTFS alternate streams
  • File creation, modification, and access time
  • File attributes like compression, hidden, and system files
  • File and directory ACLs and ownership information
  • Junction points, symlinks, and hardlinks

On Linux:

  • File and directory owner, group, and permissions
  • Symlinks and hardlinks
  • File modification, and access time
  • xattr attributes, including ACLs
  • Devices nodes and other special files

And on Mac? Well, let’s say that Mac is among the most difficult systems to backup and restore correctly. Fortunately there is a great tool called Backup Bouncer that can be used to verify metadata backup and restore for Mac backup software. I’ll let the output of that tool speak for itself:

Legacy backup and restore from Jungle Disk 2.62:

Verifying:    basic-permissions ... FAIL (Critical)
Verifying:           timestamps ... FAIL (Critical)
Verifying:             symlinks ... FAIL (Critical)
Verifying:    symlink-ownership ... FAIL
Verifying:            hardlinks ... FAIL (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... FAIL (Critical)
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... FAIL
Verifying:       extended-attrs ...
   Sub-test:             on files ... FAIL (Important)
   Sub-test:       on directories ... FAIL (Important)
   Sub-test:          on symlinks ... FAIL
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL (Important)
   Sub-test:              on dirs ... FAIL (Important)
Test dir '/Users/mrhodes/Desktop/bbouncer/backup-bouncer-0.2.0/Src/90-fifo' does not exist
Test dir '/Users/mrhodes/Desktop/bbouncer/backup-bouncer-0.2.0/Src/95-devices' does not exist
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... FAIL
   Sub-test:     lots of metadata ... FAIL

Backup Vault backup and restore from Jungle Disk 3.0:

Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... ok (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... ok
Verifying:        creation-date ... ok
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok
Verifying: access-control-lists ...
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... ok

That’s a bit better! There are only a few Mac local backup products that can pass the full Backup Bouncer test (Time Machine doesn’t even pass it). To my knowledge, Jungle Disk 3.0 is the only online backup product that passes every test, something the Jungle Disk Mac team can be proud of.

Now when you backup with Jungle Disk, you know you’re getting the whole enchilada, not just the filling!

8 Comments

  1. Robert Kosara said,

    November 13, 2009 @ 9:24 am

    Very cool! Do you have any pointers to benchmarks using Backup Bouncer that show what Time Machine is missing?

  2. Michael said,

    November 13, 2009 @ 10:03 am

    Awesome job guys! Thanks for your hard work! :)

  3. Jungle Dave said,

    November 13, 2009 @ 11:49 am

    Robert – there is a blog post on that here: http://www.n8gray.org/blog/2008/06/09/backup-bouncer-and-time-machine/

  4. Marco said,

    November 13, 2009 @ 8:50 pm

    Many thanks and congratulations on this new feature. I was evaluating JD against Dropbox for a multiple computer sync solution and backup.

    This clearly settles the question. Now, if only I could get better speeds to/from Rackspace server from here in Italy…

  5. Robert Kosara said,

    November 14, 2009 @ 8:49 am

    Thanks for the link, Dave!

  6. Vincent Pelletier said,

    March 9, 2010 @ 6:01 pm

    Tested in workgroup:
    Preservation of openmeta tags:
    - worked with backup, well done!
    - but it doesn’t work with sync, too bad :-(
    Any chance jungledisk sync will be compatible with openmeta tags in the future?

  7. MacMacken said,

    August 7, 2010 @ 6:50 pm

    Arq is the only other Mac online backup software that can pass all the Backup Bouncer tests:

    http://www.haystacksoftware.com/arq/

    Arq is more Mac-like than Jungle Disk but less feature-rich, i.e. most Jungle Disk users might probably want to stay with Jungle Disk.

  8. MacMacken said,

    November 7, 2010 @ 5:01 pm

    Could you please explain step-by-step how you performed your Backup Bouncer test?

    My own Backup Bouncer test with Jungle Disk has just miserably failed …

RSS feed for comments on this post · TrackBack URI

Leave a Reply