A simple build system

April 11, 2007

Nowadays it seems that whenever you’re working on something, you, at some point, have to start releasing stuff. Typically you can’t just take the directory of your app, compress it and upload somewhere. And even if you could, doing it manually would be extremely munday at best.

In a typical webapp project, creating a “build” means: a) cleaning up our configuration of any sensitive information b) removing any data that should not be included in the distribution c) archiving and compressing the build d) id each archive (typically using a timestamp) the build e) uploading the archive somewhere.

Looking back at my previous projects, I think that those steps are pretty universal. So what I want is actually a tool that would do all of those for us. Something non-interactive that we could could be run both on-demand or as part of some scheduling system. And I’m convinced there are a lot of fancy solutions out there. SVN would probably work (with possibly a bit of hand-coding added), so would probably Ant and a gazillion other SCM systems. Or we could just code one ourselves with a little bit of BASH scripting. This solution is neither particularly elegant nor very easily reusable, but it has worked very well for my current project and handles all the steps I mentioned before. And it doubles very nicely as a little backup scheme.

archive=`date +%y%m%d-%H%M-collective`.tar

config='./collective-build/lib/collective.yaml'



echo -n " > Copying..."; rsync -a ./collective/* ./collective-build/ --exclude "data/*/*" && echo " [OK]" && \

echo -n " > Cleaning config..."

sed -e 's/username: [a-z]*/username: username/' -e 's/password: [a-zA-Z]*/password: password/' $config > config.tmp && \

mv config.tmp $config && echo " [OK]"

echo -n " > Creating archive ..."

tar -cf $archive ./collective-build/ && gzip -c $archive > $archive.gz && echo " [OK]"

echo -n " > Uploading $archive.gz..."

curl -u user:pass -T $archive.gz 'ftp://server.com/' && echo " [OK]"

echo -n " > Cleaning up..."

rm -r $archive $archive.gz ./collective-build && echo " [OK]"

exit 0

I chose that timestamp format because it gives me nice granularity, at a stage when fixes and maybe even new features might be added every hour, I can just run this at any time. And the nice thing about this also that it’s very flexible, for example the DB schema might be very important in some other project and so you could just drop a mysqldump in there too.