How to automate the build of bottles on your Homebrew tap
========================================================

Note on Oct 4, 2018: due to a [change](https://github.com/Homebrew/brew/commit/59c858a8e8362e7cdc063e107d0360be9dc85d0e#diff-04e700045721a197c725bc15b7f2a173) in Homebrew's `brew test-bot` behaviour, the user must set `HOMEBREW_TRAVIS_CI` and `HOMEBREW_TRAVIS_SUDO` appropriately (it was previously using Travis-CI-provided `TRAVIS` and `TRAVIS_SUDO`).

This tutorial is a follow-up to the discussion we had on https://github.com/davidchall/homebrew-hep/pull/114.
It relies on a fork of the `test-bot` provided by davidchall; you can get it with `brew tap maelvalais/test-bot`.
First:

1. the Github project must be of the form `https://github.com/<user>/homebrew-<tap>` with the following tree
   (I give the example of one of my formulas, _touist_):
   
       .
       ├── touist.rb
       └── .travis.yml
   

2. the Bintray project must be of the form `https://bintray.com/<user>/bottles-<tap>`
3. the S3 bucket can have any name

## Use Travis CI to test and bottle your formulae

The idea is to use _staged build_ in Travis CI, where the two stages will be
1) the **testing stage** where the bottles are built and tested for the different architectures with
    
       brew test-bot
   
   At the end of this stage, each job must upload its bottle.tar.gz and bottle.json to somewhere (e.g.,
   free AWS S3 bucket).
2) the **deploy stage** (only on master) where we fetch the bottle.json and bottle.tar.gz,
   upload the bottle.tar.gz to Bintray and merge the multiple `bottle.json` into a commit that
   contains the bottle DSL with the command
   
       brew test-bot --ci-upload

![Screenshot of the staged build status on Travis CI](https://user-images.githubusercontent.com/2195781/32568431-83a25464-c4be-11e7-8c9f-31202d07fd79.png)

You can see a detailed example of such a configuration in the [`.travis.yml` (this gist)](#file-travis-yml),
where bottles are built for linux, sierra and el\_capitan.
You can also see an example of tap, [touist/homebrew-touist](https://github.com/touist/homebrew-touist),
that uses it and its [`.travis.yml` (homebrew-touist repo)](https://github.com/touist/homebrew-touist/blob/master/.travis.yml). 

## Workflow for updating formulas

The Homebrew/homebrew-core has the following workflow:

[![homebrew-core workflow](https://www.lucidchart.com/publicSegments/view/525e9c0f-3a81-49e6-9879-97da4d543c0f/image.png)](https://www.lucidchart.com/documents/view/d6a4384f-2faf-4fef-b9c2-ceb1dd417037)

Instead of pushing `pr-1234` (containing the updated DSL commit) to a fork (in core's case, the fork is BrewTestBot/homebrew-core),
I propose to use the same repo. Don't worry, the `pr-1234` tags won't show in people's clones. Here is the workflow:

[![homebrew-touist workflow, a tap](https://www.lucidchart.com/publicSegments/view/0ae31518-840c-4207-9276-4e3717a4bc6c/image.png)](https://www.lucidchart.com/documents/view/d8a214f4-d978-4be5-9f28-85086dbd26c8)

<p align="right">Drawings made using Lucidchart (not free).</p>

When a PR is opened and the formula needs a bottle, Travis CI will build the bottle (`brew test-bot`) and then
run `brew test-bot --ci-upload` which will:

- upload the bottle as "unpublished".
- commit and push the DSL to BrewTestBot/Homebrew-core with the pr-1234 tag.

Then a Homebrew maintainer (me) will do

    brew pull --bottle --bintray-org=touist --test-bot-user=touist https://github.com/touist/homebrew-touist/pull/5

which will do two things:

- switch the bottle in bintray from "unpublished" to "published"
- fetch the `pr-1234` tag, rebase it on top of Homebrew/homebrew-core

then the mainainer push the merged PR to Homebrew/homebrew-core.

### Drawbacks w.r.t. bottles when maintaining a tap relying on core formulae

On the homebrew-core repo, whenever a formula on which some other formulae
are relying on is updated, we also must rebuild the bottles of these dependent
formulae (this is why we have the revision number `_1` after the actual version
number). This seems to be automatically taken care of on the homebrew-core
tap.

On your own tap, this revision bump whenever a dependency bottle gets rebuilt
can lead to broken bottles. This problem has not been solved yet (see
https://github.com/Homebrew/brew/issues/3346 and
https://github.com/Homebrew/brew/issues/2572). But we can:

1) run a cron job that tests the bottles every night to check
   if the bottle still works, using `brew test *.rb` and `brew linkage --test *.rb`;
2) or we could subscribe to the changes on the formulae we depend on and
   run a script on a Travis CI build (I could not find a way to do that yet).

### What you need

1) the name of the Github repo for your tap must be `<user>/homebrew-<tap>`
2) you need a Github OAuth2 token that you can get in your Github settings
   (secret variable `GITHUB_TOKEN` in `.travis.yml`);
2) the name of the Bintray repo must be `<bintray-user>/bottles-<tap>`
   (secret variable `HOMEBREW_BINTRAY_KEY` in `.travis.yml`);
3) you also need an AWS S3 bucket, it is free and takes 1 minute to create;
   then you need to create a token+password and set them as secret variables in 
   the travis-ci settings (secret variables `AWS_ACCESS_KEY_ID` and
   `AWS_SECRET_ACCESS_KEY` in `.travis.yml`).
   
### Tricks I want to remember when I was setting up Travis CI + test-bot

1) when using `brew test-bot`, **never use** `--ci-pr`, `--ci-master`,
   `--ci-testing` on your mac as it is going to remove everything installed.
   You can still use `brew test-bot` and `brew test-bot --ci-upload`; what I would do is
   to make sure everything runs correctly locally (except for `brew doctor` which generally always
   fails on my mac) using:
   
       brew test-bot touist.rb
   
2) On Travis CI, mac builds are costly (in time). For trying to make the
   `.travis.yml` work, I really recommend to use `os: linux` instead of a mac
   image.
3) Also, when you find yourself stuck when trying to make the CI work, I
   recommend to re-run your job in [debug mode](https://docs.travis-ci.com/user/running-build-in-debug-mode/)
   in order to see what is going wrong. Again, as mac images are slow,
   I recommend to use the linux alternative.
4) if you see that `brew test-bot` is not building/testing any bottle, check
   that you have made the symlink (in travis-ci) at `$(brew --repo touist/touist)`
   that should target `$TRAVIS_BUILD_DIR`. It is important because `brew test-bot`
   is testing the tap inside `$(brew --repo)` but the actual repo cloned by travis
   is in `$TRAVIS_BUILD_DIR`.
5) the `--keep-old` option when uploading seems to be a good idea (= keeps bottles you
   previously built but that you don't produce anymore) but it keeps failing on stupid
   errors all the time; I don't use it anymore.
7) adding "revision 1" to a formula does not seem to be appropriate for forcing a bottle to be rebuild
8) the tests/building of the bottle only happens when you push a series of commit on ONE single formula;
   if you include some edits to .travis.yml for example, the push won't trigger build of the bottle. This
   caused me a lot of time lost on try and errors.
9) when creating a new formula, you may get these kind of errors during the build (on PR, branch and master):
   
       * New formulae should not require patches to build. Patches should be submitted and accepted upstream first.
       * GitHub fork (not canonical repository)
       * GitHub repository too new (<30 days old)
   
   First, make sure the bottles build without errors and that the only step failing is `brew audit --online`.
   Then, accept the PR and make a small change to the formula to force the rebuild of the bottle. The errors
   will disappear (as they are only there when the formula is new).

### Note on wierd `prefix:` and `cellar:` in bottle DSL

WARNING: in my solution, in the deploy state, I do on a linux build:

- `brew test-bot --ci-upload`

which packs all bottles at the same time and I get weird `prefix: /usr/local` and
`cellar: :any_skip_relocation`. I fixed this with a little trick: change every
`/usr/local` with `/home/linuxbrew/.linuxbrew` so that `brew test-bot --ci-upload`
removes the `cellar:` field when it is the default cellar.

Example of such problem:
MacOS `lingeling-151109.el_capitan.bottle.json` and `lingeling-151109.sierra.bottle.json` have:
```json
{
    "prefix": "/usr/local",
    "cellar": "any_skip_relocation",
}
```
but the linux bottle `lingeling-151109.x86_64_linux.bottle.json` has:
```json
{
    "prefix": "/home/linuxbrew/.linuxbrew",
    "cellar": "/home/linuxbrew/.linuxbrew/Cellar",
}
```
and it wrongly became the following bottle DSL, probably because the linux `bottle.json`
is the last to be read, so it takes precedence over the previous ones:

```ruby
  bottle do
    root_url "https://dl.bintray.com/touist/bottles-touist"
    sha256 "141f132d5ed58f930ada22fbe81d63e4115b9f78b8fb2ca3fa266e30e5fdf0a3" => :sierra
    sha256 "be1e33155185e36cbfbc48a743efd14d65edbfc174d2e31e4386414ea2153233" => :el_capitan
    sha256 "b24481b3ac4a7b02114ea5d627aff7cd6e2bb1551e4b4f073d3aa1d580f787c8" => :x86_64_linux
  end
```
It should have been
```ruby
  bottle do
    root_url "https://dl.bintray.com/touist/bottles-touist"
    cellar :any_skip_relocation if OS.mac?
    sha256 "141f132d5ed58f930ada22fbe81d63e4115b9f78b8fb2ca3fa266e30e5fdf0a3" => :sierra
    sha256 "be1e33155185e36cbfbc48a743efd14d65edbfc174d2e31e4386414ea2153233" => :el_capitan
    sha256 "b24481b3ac4a7b02114ea5d627aff7cd6e2bb1551e4b4f073d3aa1d580f787c8" => :x86_64_linux
  end
```
### Notes on my discoveries of Linuxbrew
- When a formula is linuxbrew-only, it has the comment `# tag "linuxbrew"` in it.

### What to do when some dependencies are updated?

As mentionned by Steven Peters [here](https://github.com/Homebrew/brew/issues/3346#issuecomment-339059005), when a dependency is updated, we must also update the formula that depends on it. In 'core', this is done by incrementing the _revision_ number for all formulas that rely on the updated formula.

In this tap, a cron is daily testing every formula (`brew install <formula> && brew linkage --test <formula>`). If a linkage breakage is detected, it automatically increments the revision number. It has been working fine for now, the only problem is to have travis-ci working every single day (I am getting better and better at that).