Building a Nix Package (The C&C++ Version)
------------------------------------------

Nix can be used to build any kind of package. But here I'm just going to focus on the simple C&C++ case.

Firstly we have to know that the final built packages will located inside `/nix/store`. Which is globally readable directory of all build inputs and build outputs of the Nix system. The emphasis is on _readable_, not _writable_, that is `/nix/store` is meant to be modified by the user or programs except for the Nix system utilities. This centralises the management of packages, and keeps our packages and package configuration consistent.

So what exactly are we trying to build. Our goal is to build a directory that will be located in `/nix/store/*-package-version/`, where `*` is the hash of the package. Preferably a `version` is also available, but some C&C++ packages don't have versions, so in that case, there's only `/nix/store/*-package/`.

What will be inside this directory? It follows the GNU Coding Standards described here: https://www.gnu.org/prep/standards/html_node/Directory-Variables.html

Here's an imaginary package (with the `/nix/store/` omitted):

```
*-package-version/bin/                               # executables
*-package-version/lib/                               # shared libraries
*-package-version/include/                           # header files
*-package-version/etc/                               # configuration files
*-package-version/share/                             # architecture independent data files
*-package-version/share/man                          # man pages
*-package-version/share/man/man1                     # man for general commands
*-package-version/share/man/man6                     # man for games and screensavers
*-package-version/share/man/man8                     # man for system administration commands and daemons
*-package-version/share/info                         # info pages
*-package-version/share/doc/package-version/         # arbitrary documentation
*-package-version/share/doc/package-version/pdf/     # pdf documentation
*-package-version/share/doc/package-version/ps/      # ps docuumentation
*-package-version/share/doc/package-version/html/    # html documentation
*-package-version/share/doc/package-version/html/en/ # language specific html documentation
```

Notice we don't really have `/var` or `/run` or `/com` inside the `/nix/store`, any programs requiring these, will be patched or configured to use locations outside of `/nix/store`.

When such a package will is installed, the relevant directories and files will be symlinked to either `~/.nix-profile/`, or in `/run/current-system/sw/`, allowing you access the man pages, the shared libraries, their configuration and other documentation.

The package we are going to try to build is the [TraceFileGen](https://github.com/GarCoSim/TraceFileGen) and [TraceFileSim](https://github.com/GarCoSim/TraceFileSim) from the [GarCoSim](https://github.com/GarCoSim) project.

For these 2 packages, we shall create them in `nixpkgs/pkgs/development/tools/analysis/garcosim/tracefilegen` and `nixpkgs/pkgs/development/tools/analysis/garcosim/tracefilesim`. We will also modify the `nixpkgs/pkgs/top-level/all-packages.nix` to add the top-level aliases for the 2 packages that we are creating.

The key tools that help us create these 2 packages are:

* `nix-repl` from the `nix-repl` package
* `nix-build`
* `nix-prefetch-git` from the `nix-prefetch-scripts` package

Before we even get to developing our package build and deployment scripts, we need to first content address our dependencies. We'll use `nix-prefetch-git` for this, as the 2 packages are exported via github.com. Remember to use a revision SHA256 so as to make our package deterministic. Even if you want a release tag, it's still better to get the content address associated to the release tag, as the tag is mutable (can be changed willy-nilly by the package author), but the content address isn't. We can however use a corresponding release tag as a mutable semantic label, designed for human readable aliases.

```
> nix-prefetch-git --url "https://github.com/GarCoSim/TraceFileGen.git" --rev "4acf75b142683cc475c6b1c841a221db0753b404"
Initialized empty Git repository in /tmp/git-checkout-tmp-rmmqIKdj/git-export/.git/
remote: Counting objects: 161, done.
remote: Compressing objects: 100% (111/111), done.
remote: Total 161 (delta 73), reused 111 (delta 46), pack-reused 0
Receiving objects: 100% (161/161), 163.40 KiB | 101.00 KiB/s, done.
Resolving deltas: 100% (73/73), done.
From https://github.com/GarCoSim/TraceFileGen
 * branch            HEAD       -> FETCH_HEAD
Switched to a new branch 'fetchgit'
git revision is 4acf75b142683cc475c6b1c841a221db0753b404
git human-readable version is -- none --
Commit date is 2015-11-13 11:13:13 -0400
removing `.git'...
hash is 69b056298cf570debd3718b2e2cb7e63ad9465919c8190cf38043791ce61d0d6
path is /nix/store/wda0pgx1m01aza4d1mhh35yp6di97bcl-git-export
69b056298cf570debd3718b2e2cb7e63ad9465919c8190cf38043791ce61d0d6
```

Now we know the package hash that we shall use for content addressing the upstream package. Repeat for TraceFileSim.

We can now develop the package build and deploy scripts.

For TraceFileGen, we actually have 2 files, this is because its build phase is non-standard.

Here's our `default.nix` a Nix expression that will be deriving the package:

```nix
{ stdenv, fetchgit, cmake }:

stdenv.mkDerivation rec {

  name = "tracefilegen";

  src = fetchgit {
    url = "https://github.com/GarCoSim/TraceFileGen.git";
    rev = "4acf75b142683cc475c6b1c841a221db0753b404";
    sha256 = "69b056298cf570debd3718b2e2cb7e63ad9465919c8190cf38043791ce61d0d6";
  };

  buildInputs = [ cmake ];

  builder = ./builder.sh;
  
  meta = with stdenv.lib; {
    description = "Automatically generate all types of basic memory management operations and write into trace files";
    homepage = "https://github.com/GarCoSim"; 
    maintainers = [ maintainers.cmcdragonkai ];
    license = licenses.gpl2;
    platforms = platforms.linux;
  };

}
```

And here's the `builder.sh`:

```sh
source "$stdenv"/setup

cp --recursive "$src" ./

chmod --recursive u=rwx ./"$(basename "$src")"

cd ./"$(basename "$src")"

cmake ./ 

make

mkdir --parents "$out"/bin
cp ./TraceFileGen "$out"/bin

mkdir --parents "$out"/share/doc/"$name"/html
cp --recursive ./Documentation/html/* "$out/share/doc/$name/html/"
```

While for TraceFileSim, it's very standardised build phase, but its install phase is not standardised, so we just have to override the `installPhase`:

```nix
{ stdenv, fetchgit }:

stdenv.mkDerivation {

  name = "tracefilesim";
  
  src = fetchgit {
    url = "https://github.com/GarCoSim/TraceFileSim.git";
    rev = "368aa6b1d6560e7ecbd16fca47000c8f528f3da2";
    sha256 = "22dfb60d1680ce6c98d60d12c0d0950073f02359605fcdef625e3049bca07809";
  };
  
  installPhase = ''
    mkdir --parents "$out/bin"
    cp ./traceFileSim "$out/bin"
  '';

  meta = with stdenv.lib; {
    description = "Ease the analysis of existing memory management techniques, as well as the prototyping of new memory management techniques.";
    homepage = "https://github.com/GarCoSim";
    maintainers = [ maintainers.cmcdragonkai ];
    licenses = licenses.gpl2;
    platforms = platforms.linux;
  };

}
```

Before we modify `nixpkgs/pkgs/top-level/all-packages.nix`, we need to test if our package works locally, this can easily be done using `nix-build`.

```
nix-build --keep-failed --expr 'with import <nixpkgs> {}; callPackage ./default.nix {}'
```

When it fails, because it will probably do so before you get your build scripts correct, you can find the build directory inside `/tmp`. The state of the directory will be left at exactly when the build was considered to be failed. This can help you iteratively build the build and deploy scripts.

Repeat for both packages.

When a build succeeds, it will place a `./result` symlink that goes to the built package in `/nix/store`. Here you can use it to test the binaries and documentation. It will not be "installed" on your system, so once you remove the `./result` symlink, the build artifacts can be garbage collected by Nix.

Once that is done, we can now modify the `all-packages.nix` to include:

```
...

  tracefilegen = callPackage ../development/tools/analysis/garcosim/tracefilegen { };

  tracefilesim = callPackage ../development/tools/analysis/garcosim/tracefilesim { };

...
```

It appears that there isn't much of an order to `all-packages.nix`, so I just placed it near other packages that had `trace` in their name.

Note that it is possible to imperatively compile something. Just do it at the user level.

```sh
nix-env -iA nixos.gcc
nix-env -iA nixos.gnumake
nix-env -iA nixos.cmake
```

You can now, run `make` and `cmake` in user space.

Nix's `mkDerivation` is quite advanced and can deal with most standardised `configure && make && make install` packages. It even detects `cmake` packages and appropriately configures them. But this time, the 2 packages did not respect the usual configuration parameters of `cmake` that allows one to specify the output directory. Instead a manual copy of the binaries and documentation was required.

Note that the `src` attribute can be specified to a local directory. However be aware that `./.` is valid for the surrounding directory, but not `./` nor `.`.

Once you have submitted a PR, or are looking at somebody else's PR for a package or change to NixOS or nixpkgs.

It's easy to check whether the commit has been applied to your branches if you're using pinned OS nixpkgs.

You just do `git branch --contains <commit-id> 2>/dev/null`. You just need to use the PR commit hash for this.

The command will list all the branches which contain that commit.

---

Some gnome packages require `wrapGAppsHook` to be put into their `nativeBuildInputs` especially when dealing with Gnome settings error.

You should be putting `pkgconfig` in the `nativeBuildInputs` instead of `buildInputs`. Cause it's generally only required at build time, and not runtime.

Even statically linked libraries are still considered `buildInputs` since the code that gets generated is still running at runtime.

Sometimes deps you need isn't available.

So use things like `nix-shell` to find things. Use `:l <nixpkgs>` to load global nixpkgs. Use `pkgs = import (fetchTarball ....tar.gz) {}` to load other kinds of nixpgks. I think you can also do `:l ~/Projects/nixpkgs` to nixpkgs as directories as well.

Finally remember once you have checked out your own branch, you can do `nix-env -f ~/Projects/nixpkgs -i gpredict` to install a package that only exists on your dev branch.

---

    shellHook = ''
      echo 'Entering ShelfTrend API Environment'
      . ./.env
      set -v

      alias mysql="\mysql --socket='./.mysql/mysql.sock' "$SHELFTREND_DB_DATABASE""
      alias mysqladmin="\mysqladmin --socket='./.mysql/mysql.sock'"
      alias mysqld="\mysqld \
        --datadir="$(pwd)/.mysql" \
        --socket="$(pwd)/.mysql/mysql.sock" \
        --bind-address="$SHELFTREND_DB_HOST" \
        --port="$SHELFTREND_DB_PORT""

      alias flyway="\flyway \
        -user="$SHELFTREND_DB_USERNAME" \
        -password="$SHELFTREND_DB_PASSWORD" \
        -url="jdbc:mysql://$SHELFTREND_DB_HOST:$SHELFTREND_DB_PORT/$SHELFTREND_DB_DATABASE" \
        -locations=filesystem:$(pwd)/migrations";

      set +v
    '';