New project: backup-to-dropbox - Release 0.2.0

The version 0.2.0 of the backup-to-dropbox tool has been released and can be downloaded through PyPI.

The changes introduced in 0.2.0 are:

• Feature: GPG encryption of backup
• Feature: Installation via PyPI or using setup.py from the repo. The backup-to-dropbox.py executable will be installed in the PATH.
• Code Quality: Tests have been added and code has been refactored into modules and submodules.

A little bit of background

I've recently been becoming more service independent: hosting my own personal services in VPS and managing them using Ansible.
The reason behind that will probably lead to a different blogpost, though.
Because of this, the need for doing recurrent backups became a necessity.

I use Dropbox and have a lot of free space in it, using it as my backup hosting seemed a no-brainer. Also as I keep my home server and my home PC synced to my account, I would have at least one "physical" copy of my backups automatically.

What I need for doing backups is something as simple as tar-gzipping a set of files and folders and upload it to a folder in my Dropbox. Plus, I wanted it to do some cleanup so as not to bloat my account but also be able to keep more than a single backup.

I looked to see if I could find something already made that I could use but, after a bit, I was not convinced with what I could find and decided to write one myself.

That's how backup-to-dropbox came to exist.

The initial version

The first fucntional version: 0.1.1 (yeah, I had a bug in 0.1.0 and had to patch it) was a single-file python3 script with 158 lines of code with no tests. But it had the basic functionality:

• Generate a tar-gzip file and upload it to a Dropbox account
• Have the ability to keep the last N backups and delete the oldest ones

In order to properly install it on my server, I had to create an ansible playbook that would checkout the repo, create a new virtualenv and install the dependencies there.
It would also create a bash script that activated the venv and run the script forwarding the arguments.

It was a bit of a pain but it served it purpose and I was able to actually get value from it.

After using it for a couple of days, I realized that I needed to add some more things to actually make it useful.

Version 0.2.0 - Encryption and path to maturity

The first thing I realized as I started using the script, was that there were some things I wanted to backup that contained sensitive information. Pushing them to Dropbox as-is, wouldn't have been prudent: encryption was needed.

I decided to go with GPG encryption mainly because it makes it easy to manage the keys. There's not something else I have to remember on the decryption side and I can have the encryption key in my server without having to worry much about protecting it.

Implementing it was fairly straight-forward using the pretty-bad-protocol python library.
Just by giving it the key's fingerprint and the location of the gpg home and public keyring, the application encrypts the backup for that key owner prior uploading it.

The next thing that became imperative to do for a second version was improving the installation mechanism: writing a setup.py to generate a python distribution and upload it to PyPI.
This was my first time doing this and I must admit the process was really fluid with no hiccups at all just by following the instructions.

Now installing the applications is as simply as runing python3 -m pip install backup-to-dropbox. This will install the backup-to-dropbox.py executable in a location that is in the PATH so it can be used without hassle.
This change led to be able to remove the playbook from my Ansible repo and just add it to the default packages to install through pip: so simple!

Finally, I wanted to add tests to the codebase. I wanted to make sure I had a way of (1) reproducing any bugs that I find, (2) make sure that the basic use-cases were covered and (3) that any future changes I make do not break what is currently working.

As part of the effort of adding tests, I decided to break the single-file script into a module with a couple of submodules leading to a bit more mature codebase.