Faux Pas

Release Process Outside the Mac App Store

Ali Rantakari
March 27, 2015

As you might know, Faux Pas is sold outside of the Mac App Store (via FastSpring). This is by necessity: Apple requires sandboxing for all apps sold through their App Store, and due to its nature Faux Pas is extremely difficult (or impossible) to sandbox.

This post isn’t about releasing outside the Mac App Store in general, though — it’s about the process I use for handling releases. When you’re selling through Apple’s store, the release process is defined by how their system (iTunes Connect and Xcode) works, but the rest of us out here in the wilderness must learn how to fend for ourselves.

I’ve tried to keep the release process simple, and to automate as much as is reasonable. I won’t share the actual scripts I use (because they’re not generic enough to be useful to others) but I will display the important parts below as we go along.


Like with any other project, before each release I run some tests.


Since Faux Pas is an Xcode project for an error detection tool for Xcode projects, it can be used to check itself.

Whenever I fix some issue that I found by checking Faux Pas with itself, I try to remember to mention the word “dogfooding” in the commit message:

$ git log --grep "[Dd]ogfooding" --format=oneline | wc -l

Finding issues in the app this way makes me feel both good and bad at the same time… which I think is good?

Unit tests

I use the default XCTest framework for unit tests — nothing fancy.

At the time of this writing, there are ~16 KSLOC (thousand source lines of code) of tests, as well as ~8 KSLOC of test data code — that is, code that the tests apply some of Faux Pas’ rules on to check for false positives and false negatives.

The unit tests sometimes catch regressions after refactoring sessions.

Clang Static Analyzer

I’ve configured Xcode to automatically run “deep” static analysis whenever I make a release build:


So far the static analyzer hasn’t caught any issues in Faux Pas.

Semiautomatic diagnostics output regression test

I have a bunch of open-source iOS and Mac app projects’ repositories checked out in a folder, and a script that checks all of them with both the latest released version of Faux Pas, and the current in-development version.

First of all, the script shows the checking time difference between the two versions of Faux Pas, which gives me a ballpark estimate on whether the new version might be slower or faster than the previous one. Since this is just a single run, it can’t be considered a proper benchmark, but it’s useful as a safeguard for obvious speed regressions.

Running /usr/local/bin/fauxpas SomeApp/SomeApp.xcodeproj
Writing temp.diff/SomeApp.old.json
Running ./fauxpas SomeApp/SomeApp.xcodeproj
Writing temp.diff/SomeApp.new.json
Checking time was 3.1 sec faster (0.87x — old 23.8, new 20.7)

The primary purpose, though, is to dump the diagnostics output from these checks into JSON files on disk, and show me if there are any differences. If the JSON files differ at all, I can have the script automatically open a GUI diff tool to display them to me. If anything looks suspicious I can then investigate further.

diagnostics diff

These tests have often caught regressions.

Since Faux Pas can emit a lot of diagnostics for some of the projects I use for these tests, the JSON files can be quite large (almost 10 MB). Most of the GUI diff tools I tried to use for this purpose choked quite badly (in some cases, becoming completely unusable). These included at least FileMerge, Kaleidoscope, and DeltaWalker.

I ended up with Beyond Compare, which loads up these large JSON diffs at a reasonable speed, and doesn’t lock up the UI thread while it’s doing it. It’s not the most beautiful app, but works well for diffing large files such as these.

Organizing Past and Future Releases

In the Faux Pas Git repository I have a folder called releases, which looks like this:

release folders

Each released version of the app has a version folder that contains a release_notes.md file, detailing the release notes for that version.

The latest two version folders contain the app archive of that version (whenever I publish a new release I delete the oldest archive from this directory tree).

The version folder for the current in-development release is prefixed with an underscore so that it can be easily detected by both humans and scripts. I update the release notes of the current in-development release as I go along: whenever I commit something that warrants a mention in the release notes, I put it there right away.

This directory tree is the canonical representation of the Faux Pas release history, and is referenced when generating some other content, which we’ll look at a bit later.

Keeping the release archives under Git version control like this is a somewhat controversial practice, however. For me in this particular case it’s convenient, which I feel outweighs the cons (primarily, that it bloats the size of the repo). If you are interested in using this kind of a setup for your app, I invite you to please consider whether keeping the release archives under version control makes sense in your case.

Making a New Release

In order to create a new release I run a script that does the following:

Archives the project (creating an .xcarchive):

xcodebuild \
-workspace "${ROOT_DIR}/FauxPas.xcodeproj/project.xcworkspace" \
-scheme FauxPas \
archive \
-archivePath "${XCARCHIVE_OUT_PATH}"

Exports a signed app bundle from the Xcode archive into the version folder under releases, using the same signing identity as was used in the previous step (i.e. whatever is set in the target build settings for CODE_SIGN_IDENTITY):


xcodebuild \
-exportArchive \
-exportFormat app \
-archivePath "${XCARCHIVE_OUT_PATH}" \
-exportPath "${RELEASE_DIR_PATH}/FauxPas.app" \

Compresses the .app bundle into a .tar.bz2:

tar \
-jcvf \
"${RELEASE_DIR_PATH}/FauxPas.tar.bz2" \

trash "${RELEASE_DIR_PATH}/FauxPas.app"

So I end up with releases/1.4/FauxPas.tar.bz2, which contains a correctly signed app bundle.

I store the .xcarchive in Dropbox — the debugging symbols it contains are required if I need to symbolicate any crash reports I receive from users.

Publishing the New Release

Faux Pas uses Sparkle, a library that makes it easy to update the application from within itself:


Sparkle checks for updates by downloading an Appcast, which is an XML file that declares the latest available versions of the app. It looks like this:

<?xml version="1.0" encoding="utf-8"?>
<rss xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle"
     xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">
    <title>Faux Pas</title>
    <description>Most recent changes with links to updates.</description>
      <title>Version 1.3</title>
      <pubDate>Mon, 12 Jan 2015 21:43:57</pubDate>
          length="12049698" />

The latest version of the Faux Pas Appcast is at http://api.fauxpasapp.com/appcast.

When I’m ready to publish the latest release, I run the release-publishing script. This script inspects the state of the releases/ directory tree and does the following:

  • Generates the appcast.xml
  • Generates the releasenotes.html (for Sparkle — the Appcast file links to this)
  • Validates the latest app archive
  • Uploads the two generated files, as well as all of the available app archives, into Amazon S3.

Let’s look at all of these in turn.

Sparkle appcast and release notes generation

The release notes HTML file generation is pretty straightforward: just render all of the release_notes.md Markdown files into HTML, concatenate them, and slap in some CSS.

The Appcast is a bit more tricky, however, because it needs the <sparkle:minimumSystemVersion> element for each version. This information, of course, is encoded in each release’s app bundle’s Info.plist file, which means that we need to break open the .tar.bz2, grab FauxPas.app/Contents/Info.plist from there, and read the value of the LSMinimumSystemVersion key from the property list file:

# Simplified version of actual Python script
# for illustration purposes.

import plistlib
import sh

def get_min_os_version(app_tar_bz2_path):
    info_plist = plistlib.readPlistFromString(
        str(sh.tar('-Oxf', app_tar_bz2_path,
    return info_plist.get('LSMinimumSystemVersion')

App archive validation

The script checks that the app archive is a valid .tar.bz2 archive, and that the app bundle contained in that archive is correctly code-signed.

Craig Hockenberry has written an excellent post about the steps needed to correctly check a Mac app bundle for code signing errors, so I won’t repeat that here: furbo.org: Code Signing and Mavericks. This is essentially what my script does.

Uploading to Amazon S3

The script creates a temporary folder, puts all of the files there, and then sends the contents of that folder to S3 using s3cmd:

# Control the order in which we send the files:
s3cmd --acl-public put \
  "${TEMPDIR_S3}/FauxPas-${LATEST_VERSION}.tar.bz2" \
  "${TEMPDIR_S3}/releasenotes.html" \
  "${TEMPDIR_S3}/appcast.xml" \

# Ensure old archives are deleted from S3 (--delete-removed):
s3cmd --acl-public --delete-removed \
  sync "${TEMPDIR_S3}/" "s3://${S3_FILES_BUCKET}/"

It’s important to upload the app archive and release notes HTML first, and only then the Appcast, so that as soon as the new Appcast is available for download, the files it links to will be available and updated as well.

Updating the Website

The Faux Pas website is a static site generated with Middleman, and hosted on Amazon S3.

The primary lesson of this section is DRY (Don’t Repeat Yourself): everything on the website should be updated automatically to reflect the new state of the world now that a new release of the app has been published.

Dynamic data in website content

Some parts of the website refer to things that often change whenever new releases are made. For example:

  • The “Release notes” page lists all releases and their release notes
  • The “All rules” page lists all the available rules in the latest version, and their metadata
  • The front page refers to many rules by name and mentions the total number of rules in the latest version.

In order to ensure that all these parts of the website are automatically updated, these references are made dynamic in the content templates:

  %h3 Choose from #{data.rules.length} different rules
    Faux Pas comes with #{data.rules.length} rules, categorized
    using __tags__ like _Resources_, _Style_, or _Config_.

In order for this to work, then, we need to somehow update the data these references in the website content depend on. Before every build of the static website, a script runs and regenerates this data.

The canonical source for the release information is the releases directory tree, so the script simply gets the necessary information from there and writes it into a JSON file in the Middleman data folder.

The canonical source for the available rules is the latest build of the app itself, so I’ve made a “private API” into the Faux Pas command-line interface that can be used to query for this information. The script extracts the app bundle from the latest .tar.bz2 archive, and executes the command-line interface with specific arguments, redirecting the JSON output into files in the Middleman data folder.

With this kind of a setup, different parts of the website can refer to changing aspects of the app, and I won’t have to worry about keeping them up to date.

The URL to download the archive for the latest version of Faux Pas is http://api.fauxpasapp.com/download_latest.

This is implemented as a simple web app that downloads appcast.xml, interprets it in order to determine the actual download URL for the latest version, and then redirects to the actual .tar.bz2 in S3.

This may be a bit more complicated than you’d expect, but this way the download_latest endpoint can remain completely stateless. When I make a new release I don’t have to worry about updating some piece of state there to mark the latest release version: I just have to update the single canonical representation of the “latest version” metadata — the Appcast.

An alternative way to achieve this could be to simply name the latest release package the same way (e.g. FauxPas.tar.bz2 rather than FauxPas-v1.4.tar.bz2) but I do like having the version number explicitly in the filename.

Validating the Release

Now that the release has been published, it’s good to verify that everything looks good from the users’ point of view. For this purpose, I have another script that performs the following checks:

  • The download_latest endpoint correctly redirects to the .tar.bz2
  • An HTTP HEAD request to the app .tar.bz2 archive URL responds with 200 OK and the correct Content-Length
  • HTTP HEAD requests to the appcast.xml and releasenotes.html URLs respond with 200 OK
  • The XML returned from the appcast.xml URL contains an <enclosure> element for the latest version that points to the correct .tar.bz2 archive URL.

Last Manual Steps

Tag the release in Git:

$ git tag -a v1.4
$ git push --tags



That was a lot of words for a release process that in the end feels quite simple.

This is what the commands in the shell session for all of the above would roughly look like:

$ # (Run unit tests via Xcode GUI)
$ # (Check Faux Pas with itself via its GUI)
$ tools/difftest.py # diagnostic output regression diff test
$ mv releases/_1.4 releases/1.4
$ tools/build_release.sh
$ trash releases/1.2/FauxPas.tar.bz2 # Remove oldest release archive
$ git commit -am "Add v1.4 release" # (I don't always commit like this,
$                                   #  but when I do, it's for
$                                   #  illustration purposes only)
$ tools/publish_release.sh
$ web/site/deploy.sh
$ tools/sanity_check_websites.sh
$ git push
$ git tag -a v1.4 -m "Version 1.4"
$ git push --tags

More things could be automated, but I’m pretty happy with this.