Over the next few weeks I’m planning on posting a series of blog updates about new features and changes coming in the next major release of Jungle Disk (tentatively: Jungle Disk 2.0). This release will bring a host of highly requested features while also laying the groundwork for many others to come.
One of the main changes in Jungle Disk 2.0 has to do with buckets. For those not hip to the lingo, “buckets” are a way of segmenting the storage in your Amazon S3 account to keep some files completely separate from others. In a way you can think of them like separate hard drives. Amazon allows for up to 100 buckets per account, but most users don’t need anywhere near that many. One bucket is all most people need, especially since Jungle Disk lets you share a single bucket for backup across multiple machines.
Amazon S3 buckets are used to store “objects” – primarily files, but also other filesystem objects such as directories. Amazon S3 itself doesn’t have a built-in notion of directories or folders at all. Because of this, it is up to each application that uses S3 to decide how to use buckets to store files and folders – there is no single standard. Jungle Disk was one of the first Amazon S3 applications and created it’s own object naming scheme (that we’ll call Jungle Disk 1.0 Buckets). The naming scheme was unique in that it allowed for enumeration of individual directory listings, something that was not supported originally by Amazon S3 but has since been added. We’ve documented the naming scheme, and released source code that demonstrates object name encoding/decoding on our downloads page. A few developers have added support for this scheme to their applications, however most other S3 applications now use a simpler scheme that uses object names that resemble standard URLs, made possible by a feature called delimiters added to Amazon S3 last year. Some folks have called the Jungle Disk naming scheme “proprietary” because it’s not compatible with most other S3 apps, but it’s important to point out that there is no standard, and each app implements object names slightly differently – particularly when it comes to things like representing empty directories. That said, we realize there is significant demand for Jungle Disk to support URL-style object naming, both to ease compatibility with other applications and to allow it to manage content on S3 for web hosting.
In Jungle Disk 2.0 we’ll be supporting several different types of object naming for buckets – traditional Jungle Disk 1.0 buckets, URL-style “compatibility” buckets, and a new object naming standard we’re calling Jungle Disk 2.0 buckets. Jungle Disk 2.0 buckets will enable us to break free of many of the limits imposed by Amazon S3 and add features that would otherwise be impossible. A few key examples:
- Instant renaming and moving of files and entire directories within the bucket
- Optional encryption of file and directory names
- Ability to change the encryption key for existing files, without re-uploading
- Support for individual files larger than 5GB (may not be available in the first release)
As with Jungle Disk 1.0 buckets, we’ll be releasing documentation and source code for other developers who want to support these features as well. For existing Jungle Disk users who want to take advantage of these new features we’ll be supporting conversion from Jungle Disk 1.0 buckets (without needing to re-upload all your files). The table below summarizes the features offered by each bucket type:
| Compatibility | Jungle Disk 1.0 | Jungle Disk 2.0 | |
| Maximum number of buckets | 100 | 100 | Unlimited |
| Maximum file size | 5GB | 5GB | TBD |
| Instant file / directory renaming | X | ||
| Instant archiving of previous versions | X | ||
| Preserves file modification times | X | X | |
| File data encryption | X | X | |
| File / directory name encryption | X | ||
| Change encryption key for existing files | X | ||
| Requires globally unique bucket name | X | ||
| Compatible with other S3 applications | X | ||
| Suitable for web file hosting | X |
Jungle Disk 2.0 buckets will also allow us to offer the option of using the European S3 endpoint for European customers while maintaining the ability to easily rename and move files. Jungle Disk Plus features won’t be available in Europe until a European based version of EC2 is available.
19 Comments
RSS feed for comments on this post · TrackBack URI
Subscribe to our feed
David said,
February 19, 2008 @ 4:39 pm
Great news! How much is JD 2.0 going to cost for existing users?
Re. European buckets, do you know if there’re really faster than their American cousins? From a legal perspective, an American bucket remains interesting to me since European authorities cannot access them that easily I guess.
Jungle Dave said,
February 19, 2008 @ 4:43 pm
As with all upgrades, Jungle Disk 2.0 will be free for existing users!
The performance difference for European users depends mostly on your connection – if you can max it out or come close on the US servers, the European ones won’t be any faster, but for users on very high bandwidth connections it should help the transfer speeds. Of course when it comes to backup, some users value price over raw speed, and the storage costs for the US-based locations is currently cheaper.
David said,
February 19, 2008 @ 5:00 pm
Great news again! Thank you, Jungle Dave! You truly deserve the success of Jungle Disk!
Mike said,
February 20, 2008 @ 1:13 am
Are there any plans to be able to use Jungle Disk as a service? Run it in the background so the drive is always mounted without having to run a client application?
I have seen many places shouting for this feature, it would be the single greatest reason to get JD in my opinion.
Thanks for all of your work, I was turned on to your product though several mentions of it on the Twit podcast network. I love it (and would love it a LOT more if it could be run as a service
Robert said,
February 20, 2008 @ 11:49 am
Great news, I love how this program is evolving! Will it be possible to mix types of buckets, i.e., one in JD2 mode for secure backups and at the same time one in Compatibility mode to administer stuff that should be public? Will it be possible to have them mounted at the same time as separate drives?
Jungle Dave said,
February 20, 2008 @ 11:52 am
Yes, we’re looking to offer both those features.
Brad C. said,
February 20, 2008 @ 6:05 pm
Can you explain a little more how some tricky bucket work allows re-encryption of a file without a re-upload? It seems that unless the unencrypted version leaves your computer, there’s no way to change the key (and thus, the encrypted file) on Amazon S3.
Jungle Dave said,
February 20, 2008 @ 6:11 pm
The trick to the re-encryption is that files and filenames are actually encrypted using a random 256-bit “seed” key. This seed key is then itself encrypted and stored using your encryption password. When you change your password only the seed key file itself needs to be re-encrypted rather than all the files. The unencrypted seed key never exists anywhere except in-memory on your machine.
Brad C. said,
February 20, 2008 @ 6:36 pm
Brilliant! This will make managing my backups a lot easier and is definitely a design change for the better.
Looking forward to the 2.0 release!
Rob Lewis said,
February 20, 2008 @ 7:59 pm
In response to the request to run JD as a service (which is a nice idea): could this be done via a FUSE module?
Max said,
February 22, 2008 @ 9:23 am
Hello, I’m a mac user and I’m not very happy with the look & feel of JungleDiskMonitor in MacOS X.
Will there be a more convenient look in JD2 for MacOS X?
Thanks and greets form Germany!
Pat Ritche said,
February 22, 2008 @ 8:28 pm
great stuff. When should we expect this new version? Looking forward to better encryption
MacMacken said,
February 24, 2008 @ 8:19 am
I agree with Max – I hope you are going to spend some ressources for a more Mac-like look & feel of Jungle Disk.
Jungle Disk » Blog Archive » UI Changes in Jungle Disk 2.0 said,
February 25, 2008 @ 11:20 am
[...] the series of posts on Jungle Disk 2.0 started last week. I’d like to talk about some of the user-interface changes coming in Jungle Disk 2.0 as well [...]
Nils said,
February 25, 2008 @ 1:19 pm
Jungle Dave, regarding re-encryption, does this hold for files encrypted with old versions of Jungle Disk as well?
Jungle Dave said,
February 25, 2008 @ 3:33 pm
@Nils – the key changing feature only works with JD2.0 buckets, so you’d need to convert the existing bucket first, but after that was done, then yes.
Jungle Disk » Blog Archive » Jungle Disk 2.0 Beta Released! said,
May 2, 2008 @ 5:44 pm
[...] completely reworked both the internal engine and user interface for Jungle Disk. We now support multiple bucket types, there is an entirely new user interface, and a whole host of new features including support for [...]
Chuck Juhl said,
June 15, 2008 @ 1:53 pm
How do i convert 1.0 buckets to 2.0 buckets?
Jungle Dave said,
June 15, 2008 @ 7:32 pm
Jungle Disk 2.0 includes a feature to upgrade your bucket in the configuration menu.