Monthly Archives: February 2016

Flask App Server Setup

This is a detailed log of the steps I went through to configure a Digital Ocean Droplet to run my Flask app.

Creating the Droplet

From the Digital Ocean control panel, choose to create a new Droplet with the latest version of Ubuntu (15.10 as of this writing).

Digital Ocean recommends using the 32-bit version for servers with less than 3 GB of RAM:

A 32-bit operating system is recommended for cloud servers with less than 3 GB of RAM — this is especially true for servers with 1 GB, or less, of RAM. Processes can require significantly more memory on the 64-bit architecture. On servers with a limited amount of RAM, any performance benefits that one might gain from a 64-bit operating system would be diluted by having less memory available for buffers and caching.

Set the rest of the options up as desired. For this server, I chose:

  • The $5/month plan
  • No private networking (this will run on a single server)
  • No Backups (these instructions will get me up and running again)
  • IPv6 (since it’s harder to set it up later)1
  • No User Data (this is helpful for scripting server setup, which I’m not yet doing)

Since I already have SSH keys setup with Digital Ocean, I’ve added them to this new server. If you don’t currently have SSH keys setup, you can follow this guide - and be sure to disable password login!

Initial Configuration

Digital Ocean provides great configuration instructions.

First, create a new user with super user powers:

  • Log in:
    ssh root@droplet_ip_address
  • Create a new user:
    adduser username
  • Give new user sudo privileges:
    gpasswd -a username sudo
  • Disable root login via ssh:
    nano /etc/ssh/sshd_config
    • In that file, replace PermitRootLogin yes with PermitRootLogin no
  • Restart ssh:
    service ssh restart
  • In a new shell, verify the connection works before closing the open connection to root:
    ssh username@SERVER_IP_ADDRESS

Then, copy your ssh public key to the new user account on the server. From my Mac, I ran:
ssh-copy-id username@SERVER_IP_ADDRESS

You can view the installed keys by sshing back into the server and inspecting the ~/.ssh/authorized_keys file. The root user ssh keys live in /root/.ssh/authorized_keys.

Additional Setup

Digital Ocean has also published additional security steps that should be performed on a new Ubuntu server.

First, setup a firewall (additional details):

  • Allow SSH sessions:
    sudo ufw allow ssh
  • Allow web server sessions:
    sudo ufw allow 80/tcp
  • Allow SSL/TLS:
    sudo ufw allow 443/tcp
  • Enable the firewall:
    sudo ufw enable

Next, configure NTP and timezone data to maintain your server’s clock:

  • Configure the time zone by running this command and selecting your country and city:
    sudo dpkg-reconfigure tzdata
  • Install NTP:
    sudo apt-get update sudo apt-get install ntp
  • … that’s it. NTP will be running after the installation.

If they aren’t already configured, set up automatic updates for the server (this was setup by default for me on Ubuntu 15.10):

  • Install the package: sudo apt-get install unattended-upgrades
  • Edit the configuration: nano /etc/apt/apt.conf.d/50unattended-upgrades

    Unattended-Upgrade::Allowed-Origins {
            "${distro_id}:${distro_codename}-security";
    //      "${distro_id}:${distro_codename}-updates";
    //      "${distro_id}:${distro_codename}-proposed";
    //      "${distro_id}:${distro_codename}-backports";
    };
    
  • Enable the updates daily: nano /etc/apt/apt.conf.d/10periodic

    APT::Periodic::Update-Package-Lists "1";
    APT::Periodic::Download-Upgradeable-Packages "1";
    APT::Periodic::AutocleanInterval "7";
    APT::Periodic::Unattended-Upgrade "1";
    

Install Apache (or a complete LAMP stack using these instructions): sudo apt-get install apache2

Setup git:

  • Install git:
    sudo apt-get install git
  • Set user name2:
    git config --global user.name "Your Name"
  • Set user email:
    git config --global user.email "youremail@domain.com"

Flask App

File Structure for App

I’ll go into the details of the Flask app in a later post, but it’s important to understand the directory and file configuration before we get to deploying the app (as many of the commands are specific to this setup). I’ve taken a cue from these two Digital Ocean guides, and structured the app this way:

~/repoName
    |-- appName.conf        # Apache config file, to be copied after deployment
    |-- appName.wsgi        # The wsgi script Apache will call into
    |-- requirements.txt    # Virtual Environment configuration
    |__ /venv               # Virtual Environment (ignored by repo)
    |__ /appName            # Our Application Module
         |-- __init__.py    # The main logic
         |__ /templates
         |__ /static
         |__ ..
         |__ .
    |__ ..
    |__ .

Everything needed on the server is contained here; the steps below walk through cloning the repo, installing the necessary tools, and configuring the server to run the app.

Flask Deployment

At this point you have a basic webserver up and running; navigating to http://SERVER_IP_ADDRESS/ in your browser should take you to the Apache default page.

To deploy my Flask app, I’m using mod_wsgi. First, that needs to be installed:
sudo apt-get install libapache2-mod-wsgi

Install pip for managing Python packages:
sudo apt-get install python-pip

Install Virtualenv for managing the Python dependencies (as on the development machine):
sudo apt-get install python-virtualenv

Move to /var/www/ and clone the app git repo:

  • Move to the directory: cd /var/www/
  • Clone git repo: sudo git clone GIT_REPO_URL
  • For some reason, the repo comes down from BitBucket with lowercase name: sudo mv reponame/ repoName/

Setup the virtual environment:

  • Move to cloned repo: cd repoName
  • Create environment:sudo virtualenv --no-site-packages --distribute venv
  • Install modules: sudo venv/bin/pip install -r requirements.txt

Copy the site configuration for Apache, enable site, and restart Apache:

  • Copy the configuration: sudo cp appName.conf /etc/apache2/sites-available/
  • Enable the site: sudo a2ensite appName
  • Restart Apache: sudo service apache2 restart

HTTPS (added on 2016-03-05)

These instructions (as I discussed in this post) provide great guidance for configuring HTTPS with a certificate from Let’s Encrypt. First, install the Let’s Encrypt tool: sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

Install a certificate: cd /opt/letsencrypt
./letsencrypt-auto --apache -d example.com

Install the le-renew script for auto-renewal:

sudo curl -L -o /usr/local/sbin/le-renew https://gist.githubusercontent.com/jeffvautin/5d98b4f7d42ab29463e2/raw/6a4b01a4caba2efd1e3dbc97a33d2ef1f80ecf26/le-renew.sh
sudo chmod +x /usr/local/sbin/le-renew

Edit the cron configuration: sudo crontab -e

And add this line, to run the renewal script weekly: 30 2 * * 1 /usr/local/sbin/le-renew example.com >> /var/log/le-renew.log

Other stuff

There’s still a bit to do, but I’ll circle back to these items after I go through the mechanics of the app in my next post:

In the future, I should script some of this3, but getting it written down is a good first step.


  1. I’m not worrying about this at the moment, but Digital Ocean provides instruction for configuring applications to use IPv6

  2. To see these settings on another machine, or to make sure you’ve set them correctly, you can use the command git config --list

  3. This article on Droplet Metadata may be helpful. 

Choosing a Language

In my last post I decided I’d try to write a small web app to serve as a relay for URLs called from my app, Snow Day. The first decision I needed to make was: what language should I use?

I wasn’t the only person thinking about this. I saw this conversation between Chris Adamson and Marco Arment:

Like Chris, I found the idea of pursuing Swift interesting, but thought it was a bit too premature at the moment. Python seemed interesting to me for a few reasons:

  • Many of my colleagues have begun to look at Python as an alternative to MATLAB for data set processing, and have been raving about tools such as Anaconda.
  • Blogs I read (like Dr. Drang’s site) often recommend Python for basic scripting tasks.
  • There are a few stable, well documented frameworks available for web services.
  • There are great tools for working with Python on iOS, like Pythonista.

Based on all of this, Python seemed like a reasonable choice for getting started.

Anaconda (or not)

I started by installing Anaconda 3.5, which was a mistake for at least one reason. The Python community is split into two camps: those still using Python 2.7, and those who are using Python 3. It looks like Python 2.7 is the right place to jump in, since both the Flask framework and Pythonista still require Python 2. My goal is to work with Python 2, while writing code that is forward compatible with Python 3.

The second reason installing Anaconda may have been a mistake is that it’s just more overhead than I need as I’m getting started. I was immediately trying to learn both a new language and a new tool.

So I uninstalled Anaconda, and fell back to using the Python version that shipped with OS X 10.11. I began working through the tutorials at Python Programming Language, to get a grasp of the language.

Flask

Based on this Stack Overflow comment, it seemed like Flask would be a good framework for trying to build my URL redirection service. I worked through the installation instructions on the Flask website. They recommend using Virtualenv to handle the Python configuration, and setup was easy on the Mac:

  • Install Virtualenv1: sudo easy_install virtualenv
  • Create a new environment: virtualenv venv
  • Activate the environment: . venv/bin/activate
  • Finally, install Flask in the new environment: pip install Flask

With Flask installed, I started working through the Quickstart guide.

Next Steps

In future posts I’ll detail the application I’ve built so far, and after that I’ll examine deploying it to an Ubuntu VPS.


  1. On an Ubuntu server, this step is: sudo apt-get install python-virtualenv 

Getting Up to Speed on Web Services

I was listening to the latest episode of Core Intuition, and around 42 minutes 30 seconds into the episode, I heard my name mentioned. It caught me off guard - when I replied to Manton on Twitter, I didn’t expect such a direct response on the program. Thank you, Manton and Daniel, for addressing my question:

My inquiry was in response to something Manton wrote:

I’ve always advocated for iOS developers to also be good at web services. Customers expect sync everywhere now, and you can do things with your own server that will give you an advantage over competitors who have a simpler, standalone iOS app. But being forced to migrate server data isn’t fun, especially on someone else’s schedule.

Web infrastructure largely remains a mystery to me. I’ve learned a bit about servers by migrating this site from Scriptogr.am to a self-hosted WordPress installation at Digital Ocean, but I don’t know any server-side scripting languages, and I’m intimidated by the prospect of server security. I want to learn more, but it’s tough to even sort out how to start:

  • How do I choose a language to tackle?
  • What’s the best way to manage servers, to be able to consistently spin them up quickly and securely?
  • How do you manage development versus production environments?
  • What about backups?
  • How do you monitor traffic and system status?

Daniel and Manton had a great high-level discussion about both using and building web services, with some solid advice. In particular, Daniel, around 54 minutes in, suggested taking on a small project as a first step. The example he gave was of a small trampoline-type URL redirection service he built to manage some of the web requests his apps generate.

This was an idea that had been kicking around in my mind for a while, too. Snow Day, my very simple weather app, makes direct requests of the Forecast.io API. This requires embedding my API token in the app, which puts me at risk of having someone extract it. I would greatly prefer having the app make a call to a server I run, which could then make the API request on the behalf of the app. This would allow me to keep the token private1, and it seems like a great starter project.

I think that’s what I’ll tackle. And I’ll try to document the process here as I move forward. So again, thank you, Manton and Daniel, for taking the time to respond to my question. I really appreciate the advice!


  1. …But requires me to figure out how to identify valid requests from the app… 

Messed Up Apple Music Metadata

Kirk McElhearn wrote about the messed up state of Apple Music metadata, and he used the Ultimate Sinatra collection as his example:

When I first added the Ultimate Sinatra album to my library, it was a single album, but it changed after a while. I surmise that iTunes matched the tracks and found that many of them were on different albums. Just as iTunes often matches tracks from studio albums to “best of” albums, here, it’s matched some tracks to the album I downloaded, and other tracks to whatever iTunes found first. But, of course, iTunes shouldn’t match Apple Music tracks; it only matches your music, that you’ve ripped or downloaded, and added to your iTunes library.

I actually added this album to my iTunes library about two months ago. My process for importing music from CD is:

  1. Rip the CD to Apple Lossless format using XLD.
  2. Update the metadata of these files using MusicBrainz Picard.
  3. Import these files into my iTunes library.

Once I read Kirk’s article, after the panic subsided, I opened up iTunes to see what Apple Music might have done to the tags on my copy of this album. To my surprise, the metadata seems to have been unaltered since I imported it into iTunes:

Frank Sinatra's *Ultimate Sinatra* in my iTunes library

It’s the unpredictability of Apple Music that most scares me. That Kirk and I can have such wildly varying experiences (with the same album!) does not bode well for the back-end of Apple Music. I haven’t done it yet, but every month I consider canceling my Apple Music subscription out of fear that it will destroy my music library.

If there was an easy way to use iTunes Match with my main library, and run Apple Music in a separate library, I would probably do that. Since they’ve merged the back-ends of those two features as iCloud Music Library, I don’t believe that’s possible. So I make regular backups, and I’m hoping I’ll still have a good backup around once I figure out which of my 19,454 songs have been corrupted by Apple Music.