Well, if you read some of my other yammering, you know I’ve been doing a lot of publishing to S3 lately. I tried a number of tools, but none of them did exactly what I wanted:

  • recursively upload files
  • push corrent mime type
  • make everything world readable
  • ignore CVS directories

I do a lot of Java mess, so Java seemed like a fitting choice… I had the s3-example-libraries-1.0.0.jar and the whole gestalt is kinda Java-ey and why am I apologizing?


Java is a perfectly respectable language. It’s not like I wrote it in perl!

jar-czar-publisher expect to find a file called s3.properties that looks something like:

key    = <put your key here>
secret = <put you secret here>
bucket = <name of your bucket>
email  = <your email address>
source = publish

To make things easy, you can use bin/run.sh to kick everything off after running “mvn package“.

The application will recurse all the files under publish and push them up to your bucket. For example:

                         xsl/headers.xsl ->                  publish/xsl/headers.xsl
                         xsl/footers.xsl ->                  publish/xsl/footers.xsl
                            xsl/user.xsl ->                     publish/xsl/user.xsl
                          xsl/search.xsl ->                   publish/xsl/search.xsl

Now the url “http://mybucket.s3.amazonaws.com/xsl/headers.xsl&#8221; will pull back the file from “publish/xsl/headers.xsl”.

To keep things easy-peasy, I leave the stuff where ever it is on disk and link it’s directory in publish ala:

% mkdir publish
% cd publish
% ln -s ${HOME}/tmp/somejunk 
% cd -
% mvn package
% vim s3.properties
% ./bin/run.sh

using that zip

That jar-czar-publisher.zip contains the source code… Speaking more accurately, it contains the CVS directory. In order to use it you’ll need to setup a local cvs repository.

Just in case you haven’t done this, shame on you! Everything you write should go into CVS!


These instructions are for bash… Running ’em in other shells is left as an excercise for the reader… 😛

% mkdir -p /tmp/cvs/a-go-go
% cd ${_}
% export CVSROOT=${PWD}
% cvs init 
% wget http://brianin3d.googlepages.com/jar-czar-publisher.zip
% unzip jar-czar-publisher.zip
% mkdir -p /tmp/some/more/dumb/junk/from/brian
% cd ${_}
% cvs co jar-czar-publisher
% cd ${_}
% echo come on.. I think you know what to do from here...

If you don’t…

% mkdir -p ${HOME}/.m2/repository/com/amazon/s3/s3-example-libraries/1.0.0/
% cd ${_}
% wget http://brianin3d.googlepages.com/s3-example-libraries-1.0.0.jar
% cd -
% mvn package
% echo see above

a few things about a few things

OK, I hate to explain a joke, but let me say something about some of that… “${_}” is a bash-ism that means “whatever the last argument of the previous command.” Got it off a good friend of mine name of John Taylor.

The other “cd -” just means change directory back to wherever I just came from.

a word about mime types

I really don’t like mime types. I think they are dumb. Ever use the file command? That’s smart! A list of magic keys to identify file type. Heck, even guessing by file extensions is smarter!

I always wondered why there was no Java equivalent of the file command, but I have things to do, so I went with the still pretty dumb solution of file extension.

It’s just a property file lookup. I made if from /etc/mime.types like so:

% grep -v '#' /etc/mime.types | awk '{ \
    for ( i = 2 ; i <= NF ; i++ ) printf( "%-40s = %s\n", $i, $1 ); \
}' > mimes.properties
% wc -l src/main/resources/mimes.properties

mimes.properties seems pretty handy….

Yup, just like that. I used to think I was pretty good at shell scripting, but it always ends up with being pretty ok with awk. I’ve written some fancier awk, believe you me.

If you really care:

public abstract class Util {
    public static final Properties MIME = new Properties();
    static {
        try {
            MIME.load( new FileInputStream( "src/main/resources/mimes.properties" ) );
        } catch( Exception e ) {

    public static String mime( String file ) {
        String mime = MIME.getProperty( file.replaceAll( ".*\\.", ""  ) );
        return null == mime ? DEFAULT_MIME_TYPE : mime;

    public static String mime( File file ) {
        return Util.mime( file.toString() );

bit of s3 naming advice

I really wish someone had pointed this out earlier. If you want to use S3 to host stuff for your website:

name your bucket something from your domain

For example: ” xml.jar-czar.com” and then point a cname at “xml.jar-czar.com.s3.amazonaws.com”. That way, when you want to give people a URL, you can give them http://xml.jar-czar.com/browse/a/a.page00000035.xml instead of http://xml.jar-czar.com.s3.amazonaws.com/browse/a/a.page00000035.xml


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: