Skip to content

Instantly share code, notes, and snippets.

@CodingKoopa
Last active October 25, 2025 06:20
Show Gist options
  • Save CodingKoopa/3b30afe8c91e3950f6b124cd2abe3b6b to your computer and use it in GitHub Desktop.
Save CodingKoopa/3b30afe8c91e3950f6b124cd2abe3b6b to your computer and use it in GitHub Desktop.

Revisions

  1. CodingKoopa revised this gist Aug 6, 2025. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -387,7 +387,7 @@ As for distribution upgrades, Ubuntu has an [upgrade GUI](https://ubuntu.com/tut
    ## Conclusion
    I started these notes to ascertain how Ubuntu's update channels work. The software update interface makes it seem like, when you only enable the security update channel, you only get security updates. However, after sifting through all the documentation, this isn't always the case: once a non-security bug fix has been pushed, future security bug fixes will be based on that latest package. Only using the security update channel does not control *which- updates (out of `<release>-security`, `<release-updates>`, and `<release-proposed>` by proxy) you get, but rather *when- you get them. The same goes for Debian.
    I started these notes to ascertain how Ubuntu's update channels work. The software update interface makes it seem like, when you only enable the security update channel, you only get security updates. However, after sifting through all the documentation, this isn't always the case: once a non-security bug fix has been pushed, future security bug fixes will be based on that latest package. Only using the security update channel does not control *which* updates (out of `<release>-security`, `<release-updates>`, and `<release-proposed>` by proxy) you get, but rather *when* you get them. The same goes for Debian.
    This info is useful to know when configuring unattended-upgrades, as it defaults to only using the security channel. It's my opinion that most users and system administrators are not given the precise information they need to make this decision, something which I think plagues the landscape of system administration on Ubuntu. I polished this into a longer form of writing to help combat cargo-cult administration by encouraging curiosity about the underlying systems.
  2. CodingKoopa revised this gist Aug 6, 2025. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -217,7 +217,7 @@ Ubuntu has the following repositories:
    - As with Debian's stable updates, this update channel is used for desirable but non-security bug fixes. It may be looser, with Ubuntu permitting the publishing of features in non-critical packages, and even entire new versions, granted the changes fall within the SRU qualifications.
    - Ubuntu utilizes [phased updates](https://web.archive.org/web/20170719231238/http://www.murraytwins.com/blog/?p=127) to deploy a update while checking for regressions.
    - For each of the stable releases, an additional suite for stable proposed updates, `<release>-proposed`.
    - This is the staging area for SRUs. Unlike Debian's proposed updates, *everything- that is pushed to the SRU suite is intended to be promoted to `<release>-updates` once tested.
    - This is the staging area for SRUs. Unlike Debian's proposed updates, *everything* that is pushed to the SRU suite is intended to be promoted to `<release>-updates` once tested.
    - Unlike Debian Testing, this *is- used for `devel`, because there is no equivalent of Debian Unstable.
    - For each of the stable releases, an additional suite for [backports](https://help.ubuntu.com/community/UbuntuBackports). Called "Unsupported updates" in the GUI/
    - As with Debian backports, these are provided without any security guarantee, and require pinning or explicit installation to use..
  3. CodingKoopa revised this gist Aug 12, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -134,7 +134,7 @@ Debian has the following repositories [https://wiki.debian.org/SourcesList]:
    - [Debian Ports](https://www.ports.debian.org/), hosted at https://deb.debian.org/debian-ports/. This providing packages for unofficial architectures, as non-stable releases. [https://www.debian.org/ports/]
    - [Debian Archives](https://www.debian.org/distrib/archive), hosted at http://archive.debian.org/debian/. This provides source packages (and binary packages for some) for older stable releases that are no longer supported.
    Of these sources, Debian systems tend to default to `<release>` + `<release-security>` +`<release-updates>` (where there shouldn't be any significant tie breaking done by the order). `<release>` + `<release-security>` is also well-supported. Backports has to be manually enabled, and has less support guarantees. It's possible to use the `stable` aliases rather than the specific codenames, but not recommended as this will perform a distribution upgrade, seemingly randomly once released.
    Of these sources, Debian systems tend to default to `<release>` + `<release-security>` +`<release-updates>` (where there shouldn't be any significant tie breaking done by the order) from the current releases. `<release>` + `<release-security>` is also well-supported. Backports has to be manually enabled, and has less support guarantees. It's possible to use the `stable` aliases rather than the specific codenames, but not recommended as this will perform a distribution upgrade, seemingly randomly once released.
    To help solidify what the pieces do, the lifespan of a package is very roughly like this:
    - Upstream publishes source code.
  4. CodingKoopa revised this gist Aug 12, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -24,7 +24,7 @@ Within a repository is a organizational unit for each release of the distributio
    - Tooling and manpages tends to call this the **suite**, so this is my preference. Examples are [Debian's repository documentation](https://wiki.debian.org/DebianRepository/Format) and the manpage for `sources.list`.
    - Debian calls this the **release** when it's being used in the form of a release codename.
    - Debian calls this the **suite** when it's being used as an alias for a release, such as `stable`. On the server, this is implemented using symlinks that are transparent to the client.
    - Debian calls this the **distribution** when referring to either a suite or release as defined in their terms.
    - Debian calls this the **distribution** when referring to either a suite or release.
    - Debian used to call this the **archive**. There are still references to "archive root" that refer to paths relative to the suite.
    - Ubuntu calls this the **pocket**, the **repository**, [https://help.ubuntu.com/community/Repositories] and sometimes a **channel** when talking about updates.
    - unattended-upgrades calls this the **archive**.
  5. CodingKoopa revised this gist Aug 12, 2022. 1 changed file with 4 additions and 4 deletions.
    8 changes: 4 additions & 4 deletions apt.md
    Original file line number Diff line number Diff line change
    @@ -22,11 +22,11 @@ Packages within this repository will have an `Origin` field indicating where the
    Within a repository is a organizational unit for each release of the distribution. This unit has a few names depending on the context: [https://wiki.debian.org/DebianRepository/Format]

    - Tooling and manpages tends to call this the **suite**, so this is my preference. Examples are [Debian's repository documentation](https://wiki.debian.org/DebianRepository/Format) and the manpage for `sources.list`.
    - Debian calls this the **release*- when it's being used in the form of a release codename.
    - Debian calls this the **suite*- when it's being used as an alias for a release, such as `stable`. On the server, this is implemented using symlinks that are transparent to the client.
    - Debian calls this the **distribution*- when referring to either a suite or release as defined in their terms.
    - Debian calls this the **release** when it's being used in the form of a release codename.
    - Debian calls this the **suite** when it's being used as an alias for a release, such as `stable`. On the server, this is implemented using symlinks that are transparent to the client.
    - Debian calls this the **distribution** when referring to either a suite or release as defined in their terms.
    - Debian used to call this the **archive**. There are still references to "archive root" that refer to paths relative to the suite.
    - Ubuntu calls this the **pocket**, the **repository**, [https://help.ubuntu.com/community/Repositories] and sometimes a **channel*- when talking about updates.
    - Ubuntu calls this the **pocket**, the **repository**, [https://help.ubuntu.com/community/Repositories] and sometimes a **channel** when talking about updates.
    - unattended-upgrades calls this the **archive**.
    - This is not to be confused with the [Debian Archives](https://www.debian.org/distrib/archive) (http://archive.debian.org/debian/) or the Ubuntu Archive (http://archive.ubuntu.com/ubuntu/), which are first-party hosts of distribution packages. The latter is referred to by the Ubuntu Package Search (https://packages.ubuntu.com/), whereas the [Debian Package search](https://www.debian.org/distrib/packages) (https://packages.debian.org/) refers to https://deb.debian.org/ for downloads, which dispatches to an offsite CDN.
    - unattended-upgrades also uses the word "origin" to refer to the combination of a package `Origin` field and the suite it comes from [https://github.com/mvo5/unattended-upgrades/blob/2.9.1/unattended-upgrade#L607] :
  6. CodingKoopa revised this gist Aug 12, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -6,7 +6,7 @@ These notes go into the downloading part, as I think it's easy to get overwhelme

    ## Repository

    A **repository* is where the packages come from. It provides binary packages for an arbitrary selection of architectures as well as source packages. Repositories are APT's data source. [https://wiki.debian.org/DebianRepository]
    A **repository** is where the packages come from. It provides binary packages for an arbitrary selection of architectures as well as source packages. Repositories are APT's data source. [https://wiki.debian.org/DebianRepository]

    APT data sources can take different forms such as [CD](https://en.wikipedia.org/wiki/Compact_disc), but the one you likely use is [HTTP(S)](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol), for downloading packages from a repository on the internet. The sources are configured in `/etc/apt/sources.list` in a one-line-style format in a format that we'll disect later.

  7. CodingKoopa revised this gist Aug 12, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -6,7 +6,7 @@ These notes go into the downloading part, as I think it's easy to get overwhelme

    ## Repository

    A **repository*- is where the packages come from. It provides binary packages for an arbitrary selection of architectures as well as source packages. Repositories are APT's data source. [https://wiki.debian.org/DebianRepository]
    A **repository* is where the packages come from. It provides binary packages for an arbitrary selection of architectures as well as source packages. Repositories are APT's data source. [https://wiki.debian.org/DebianRepository]

    APT data sources can take different forms such as [CD](https://en.wikipedia.org/wiki/Compact_disc), but the one you likely use is [HTTP(S)](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol), for downloading packages from a repository on the internet. The sources are configured in `/etc/apt/sources.list` in a one-line-style format in a format that we'll disect later.

  8. CodingKoopa revised this gist Aug 12, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion apt.md
    Original file line number Diff line number Diff line change
    @@ -2,7 +2,7 @@

    To install and update packages on Debian, Ubuntu, or most derived distributions, you use [APT](https://en.wikipedia.org/wiki/APT_(software)), the Advanced packaging tool, to download the packages and their dependencies, which uses dpkg to install them.

    These notes go into the downloading part, as I think it's easy to get overwhelmed by how the repositories work.
    These notes go into the downloading part, as I think it's easy to get overwhelmed by how the repositories work. I will eventually turn this into a page on my website.

    ## Repository

  9. CodingKoopa created this gist Aug 12, 2022.
    405 changes: 405 additions & 0 deletions apt.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,405 @@
    ## Intro

    To install and update packages on Debian, Ubuntu, or most derived distributions, you use [APT](https://en.wikipedia.org/wiki/APT_(software)), the Advanced packaging tool, to download the packages and their dependencies, which uses dpkg to install them.

    These notes go into the downloading part, as I think it's easy to get overwhelmed by how the repositories work.

    ## Repository

    A **repository*- is where the packages come from. It provides binary packages for an arbitrary selection of architectures as well as source packages. Repositories are APT's data source. [https://wiki.debian.org/DebianRepository]

    APT data sources can take different forms such as [CD](https://en.wikipedia.org/wiki/Compact_disc), but the one you likely use is [HTTP(S)](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol), for downloading packages from a repository on the internet. The sources are configured in `/etc/apt/sources.list` in a one-line-style format in a format that we'll disect later.

    A given server has one repository servicing the needs of a particular distribution, across releases. Since a mirror server may be doing mirroring for multiple distributions, it is likely (but not required) that the repository is located in a subdirectory of the server such as `/debian/` or `/ubuntu/`.

    ## Origin

    Packages within this repository will have an `Origin` field indicating where the packages are coming from - not referring to this mirror server, but rather who put together the package. This is `Debian` for official packages, or the distributor otherwise [https://debian-handbook.info/browse/stable/sect.apt-get.html, "6.2.5. Managing Package Priorities
    "]. Though this article attempts to go through the hierarchy starting from the broader parts, it's worth noting this individual package detail because unattended-updates uses it for broadly categorizing packages.

    ## Suite

    Within a repository is a organizational unit for each release of the distribution. This unit has a few names depending on the context: [https://wiki.debian.org/DebianRepository/Format]

    - Tooling and manpages tends to call this the **suite**, so this is my preference. Examples are [Debian's repository documentation](https://wiki.debian.org/DebianRepository/Format) and the manpage for `sources.list`.
    - Debian calls this the **release*- when it's being used in the form of a release codename.
    - Debian calls this the **suite*- when it's being used as an alias for a release, such as `stable`. On the server, this is implemented using symlinks that are transparent to the client.
    - Debian calls this the **distribution*- when referring to either a suite or release as defined in their terms.
    - Debian used to call this the **archive**. There are still references to "archive root" that refer to paths relative to the suite.
    - Ubuntu calls this the **pocket**, the **repository**, [https://help.ubuntu.com/community/Repositories] and sometimes a **channel*- when talking about updates.
    - unattended-upgrades calls this the **archive**.
    - This is not to be confused with the [Debian Archives](https://www.debian.org/distrib/archive) (http://archive.debian.org/debian/) or the Ubuntu Archive (http://archive.ubuntu.com/ubuntu/), which are first-party hosts of distribution packages. The latter is referred to by the Ubuntu Package Search (https://packages.ubuntu.com/), whereas the [Debian Package search](https://www.debian.org/distrib/packages) (https://packages.debian.org/) refers to https://deb.debian.org/ for downloads, which dispatches to an offsite CDN.
    - unattended-upgrades also uses the word "origin" to refer to the combination of a package `Origin` field and the suite it comes from [https://github.com/mvo5/unattended-upgrades/blob/2.9.1/unattended-upgrade#L607] :
    ```
    Allowed origins are: o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic, o=UbuntuESM,a=bionic-infra-security
    ```
    In the repository, the suites are located within the `dists/` directory, e.g. `/debian/dists/stable`. With some Ubuntu-related exceptions, this organizational unit warrants its own section in UIs such as package search. That is, in navigating what a repository has from a website, they will usually list each suite that it has.
    In practice, there are more than one suite for a distribution release. We will explore this once we have solidified the foundations of how APT works.
    ## Component
    A suite is broken up into different components, used for categorizing packages based on their libre purity and/or support guarantees.
    The components are located in `dists/<suite>/`, e.g. `/debian/dists/stable/main`. However, unlike suites, these may not have their own section in UIs. In both Debian Package Search and Ubuntu Packages Search, packages from all components are pooled together, and packages that are not from `main` are labeled as such.
    "Components" is a very universal term, though Debian can call them [archive areas](https://www.debian.org/doc/debian-policy/ch-archive.html#archive-areas), and Ubuntu can (unfortunately) call them [repositories, or channels](https://help.ubuntu.com/community/Repositories/Ubuntu#The_Four_Main_Repositories). Unlike the different names for suites, though, these alternate terms are rare.
    ## Release and Package Index
    Within each suite is a release file, signed with GnuPG. `Release` is accomodated with `Release.gpg`, and the `InRelease` has the signature inside the file instead. The release file is comprised of hashes for the package indices, for verifying their integrity.
    Within each component is another directory corresponding either with a particular architecture, or is `source` (or is one of the other exceptions that are out-of-scope). Along the directories themselves may be content indices, containing the files provided by packages.
    Each architecture has a binary package index. The source directory has its own mechnanism. The package index contains metadata for every package in the component. The set of info for each package is a superset of the binary package control file, as described in the [Debian Policy Manual](https://www.debian.org/doc/debian-policy).
    There are two ways to get to the binary package index:
    - `dists/<suite>/<component>/<architecture>/Packages.<ext>`.
    - `dists/<suite>/<component>/<architecture>/by-hash/<hash_type>/<hash>`, using the hash from the release file.
    In either case, the checksum of the package index will be verified by the package manager. The package index contains a `Filename` field containing a path within the archive (what we've been calling the suite), indicating where the actual package can be downloaded.
    ## APT Process
    The process of discovering and installing software starts with one of many package management tools. When it comes to frontends to APT, [Debian suggests](https://wiki.debian.org/PackageManagementTools) `apt` and `aptitude` for CLI, and Synapse for GUI. [GNOME Software](https://wiki.gnome.org/Apps/Software) and [Discover](https://apps.kde.org/discover/) are DE-specifc GUI offerings. Ubuntu, on the other hand, provides the [Ubuntu Software Center](https://wiki.ubuntu.com/SoftwareCenter). Ubuntu also ships a [GUI for configuring and reviewing updates](https://wiki.ubuntu.com/SoftwareUpdates).
    These tools all use the lower-level `apt-` programs to install and update programs. The relationship between `apt` and `apt-get` is similar to that of [adduser](https://manpages.debian.org/latest/adduser/adduser.8.html) and [useradd](https://manpages.debian.org/latest/passwd/useradd.en.html). In truth, the higher level frontends also could be using libapt directly, but the former assumption usually isn't harmful to make. [https://manpages.debian.org/stable/apt/apt.8.html] This article will proceed with the simplification that, by discussing `apt-get`, we are covering all of our bases. `apt` and related tools are all developed at https://salsa.debian.org/apt-team/apt.
    It's the case for just about any Linux package manager that the package index has to be synchronized before packages can be downloaded. The "human-friendly" interface to do this is `apt update`. `apt-get update` is the lower level tool.
    `apt-get update` goes through each of the APT data sources, in order. The sources are configured in `/etc/apt/sources.list` in a one-line-style format including: [https://manpages.debian.org/stable/apt/sources.list.5.html]
    - The type of the source, either `deb` (binary packages for the current architecture) and `deb-src` (source packages).
    - Optional repository options, such as a signing key.
    - The URI of the repository.
    - The suite within the repository to use.
    - Zero or more components within the suite to use.
    The suite and components may contain slashes. What matters is that there is a `Release` file after concatenating the repository, suite, and each component.
    For each of the data sources, `apt-get update` downloads the package index to `/var/lib/apt/lists/`. This directory comprises APT's package cache, and is its source of truth for downloading packages. [https://debian-handbook.info/browse/stable/sect.apt-get.html, "6.2.1. Initialization
    "] This database can be queried using `apt search`, `apt show`, and in more detail with `apt-cache` [https://debian-handbook.info/browse/stable/sect.apt-cache.html, https://manpages.debian.org/stable/apt/apt-cache.8.html]
    A package can exist in more than one APT data source, potentially even of the same version. To automatically decide which one to use, APT prioritizes, in order:
    - The package's priority value, determined by APT. This is entirely unrelated to the `Policy` [control field used in the package itself](https://www.debian.org/doc/debian-policy/ch-archive.html#priorities). This value can be influenced by the system administrator in the APT preferences file, as to have a final say as to which package is installed. Specifically, they may "pin" the priority of a suite or just a package, to make APT use it. [https://wiki.debian.org/AptConfiguration, https://manpages.debian.org/stable/apt/apt_preferences.5.html, https://help.ubuntu.com/community/PinningHowto] Without this manual intervention, installed packages default to a priority of `100`, and uninstalled packages to `500`. ["6.2.5. Managing Package Priorities"](https://debian-handbook.info/browse/stable/sect.apt-get.html) of the Debian Handbook best explains all of the consequences of this.
    - The package's version.
    - Ordering within `sources.list`.
    See the apt_preferences manpage for a more rigorous definition. You can run `apt-cache policy <package>` to see what APT sees, for a particular package. This includes each source for the package, and the priorities and versions.
    Additionally, you can manually select a data source as described in the apt_preference manpage. When the "target release" is explicitly given, and a package from it is installed, it defaults to a priority of `990`. When it comes to getting it installed in the frist place, though, this "target release" overrides general priorities, but not specific package priorities.
    Overall, having added multiple APT data sources, there are several ways to influence how APT treats a package:
    - Configuring a target release to make APT prioritize that distribution over others (withstanding package pins).
    - Pinning a package or a suite.
    - Holding a package, preventing it from being upgraded.
    - Installing a specific package version by running `apt-get install <package_name>=<package_version>`. Couples well with holding, to prevent automatic upgrades.
    - Installing a package from a specific suite by running `apt-get install <package_name>/<suite>-backports`. [https://help.ubuntu.com/community/UbuntuBackports#Manually_Installing_Packages_from_Backports].
    For most of these techniques (all but the first), even if a non-default version of a package is installed, the default versions of the dependencies will be used. That is, if you install backport without pinning, the non-backport dependencies will be used if possible.
    Once APT has decided which package to install, it then has to resolve its dependencies, and then use dpkg to install them. dpkg will resolve any conflicts that arise with existing files or modified configuration files. This is all outside of the scope of this article, though.
    Having established how APT interacts with repositories, we can go into the specifics of how Debian and Ubuntu uses them.
    ## APT in the Wild
    ### Debian Repositories
    Debian has the following repositories [https://wiki.debian.org/SourcesList]:
    - Current Releases, hosted at https://deb.debian.org/debian (first-party global CDN mirror), http://ftp.debian.org/debian/ (legacy first-party mirror), hxxp://ftp.CC.debian.org/debian/ ([third-party mirrors by country](https://wiki.debian.org/ftp.debian.org)), and [other third-party mirrors](https://www.debian.org/mirror/list). Has the following suites:
    - A suite (and codename alias) for the latest point releases of each of:
    - The current [stable release](https://wiki.debian.org/DebianStable), `stable`.
    - The past 3 stable releases, up to `oldoldstable`.
    - An additional suite for [stable updates](https://wiki.debian.org/StableUpdates) for each of the current stable releases, `<release>-updates`.
    - Since the point release suites are frozen until the next release, this is the mechanism for important non-security bug fixes to be deployed to users.
    - A suite for the [next stable release](https://wiki.debian.org/DebianTesting), `testing`.
    - There is not a notion of point releases for this release; this gets updates semi-regularly until it is frozen right before releasing as `stable`.
    - Don't be fooled by `testing-updates` - that is an empty suite. Debian Testing does not have an update suite, since it itself is regularly updated.
    - For each of the stable releases, an additional suite for [stable proposed updates](https://wiki.debian.org/StableProposedUpdates), `<release>-proposed-updates`.
    - This is the staging area for the next *point release*, though updates do get pulled into `<release>-updates` as needed. [https://wiki.debian.org/DebianReleases/PointReleases]
    - This is empty/meaningless for `testing`, because Debian Unstable serves as the the staging area for Testing.
    - For each of the stable releases, an additional two suites for [backports](https://wiki.debian.org/DebianBackports). This is `<release>-backports` for newer software than `<release>` or `<release>-updates`, and `<release>-backports-sloppy` for even newer software, newer than what the stable release after `<release>` comes with.
    - These have to be manually added to the `sources.list`, and their packages have a default priority of `100`.
    - This is empty/meaningless for `testing`.
    - These packages do not receive security fixes from the Security Team. [https://backports.debian.org/FAQ/]
    - A [rolling-release suite](https://wiki.debian.org/DebianUnstable) for new packages, `unstable` (codename Sid - I don't have to worry about that aging poorly!). Debian Unstable does not have separate update suites.
    - [Debian Security](https://www.debian.org/security/), hosted at http://security.debian.org/debian-security/ (and aliased at https://deb.debian.org/debian-security).
    - This does not use mirrors or CDNs in the same way that https://deb.debian.org does. [https://www.debian.org/security/faq#mirror]
    - This has a `<release>-security` suite for each release that the Security Team is providing security updates for. The Security Team endeavors "to support a stable distribution for about one year after the next stable distribution has been released," with the `contrib` and `nonfree` components generally being out of scope. [https://www.debian.org/security/faq#lifespan]
    - [Automatic Debug Packages](https://wiki.debian.org/AutomaticDebugPackages), hosted at http://deb.debian.org/debian-debug/. This has the same layout as the current releases, but the packages are all built with debug symbols.
    - [Debian Ports](https://www.ports.debian.org/), hosted at https://deb.debian.org/debian-ports/. This providing packages for unofficial architectures, as non-stable releases. [https://www.debian.org/ports/]
    - [Debian Archives](https://www.debian.org/distrib/archive), hosted at http://archive.debian.org/debian/. This provides source packages (and binary packages for some) for older stable releases that are no longer supported.
    Of these sources, Debian systems tend to default to `<release>` + `<release-security>` +`<release-updates>` (where there shouldn't be any significant tie breaking done by the order). `<release>` + `<release-security>` is also well-supported. Backports has to be manually enabled, and has less support guarantees. It's possible to use the `stable` aliases rather than the specific codenames, but not recommended as this will perform a distribution upgrade, seemingly randomly once released.
    To help solidify what the pieces do, the lifespan of a package is very roughly like this:
    - Upstream publishes source code.
    - Debian packager creates package.
    - Debian package uploads package to Debian Unstable.
    - If security updates occur here, they are addressed directly through package updates.
    - Package is promoted to next stable (Debian Testing).
    - If security issues occur here, the Debian Security Team should be patching the Testing package and uploading it to the `testing-security` suite. However, this is the slowest security suite, so users are encouraged to pull package updates from `unstable` if needed. [https://www.debian.org/security/faq#testing, https://www.debian.org/doc/manuals/securing-debian-manual/ch10.en.html#security-support-testing].
    - Package is promoted to `stable`. Initial point release is made.
    - [Stable updates](https://wiki.debian.org/DebianReleases/PointReleases) are pulled from upstream and pushed to `stable-proposed-updates`.
    - Security updates pushed to `stable-security` are automatically uploaded here. [https://www.debian.org/releases/proposed-updates, https://www.debian.org/security/faq#ppu]
    - Some of proposed updates are pulled into `stable-updates`.
    - If a security issue occur here, the Debian Security Team patches the package and uploads it to the Debian Security repository.
    - The pacakge version chosen to be patched is the latest out of `stable`, `stable-security`, `stable-updates`, and `stable-proposed-updates`. This is necessary so that:
    - Users with `stable-updates` do not experience regressions in non-security bugs.
    - The patched package can be integrated into the next point release without any changes.
    - The security team does not have the burden of maintaining patches for every suite.
    - With this said, ["The most important guideline when making a new package that fixes a security problem is to make as few changes as possible."](https://www.debian.org/security/faq#oldversion).
    - The package updated by the Security Team in `stable-security` is automatically uploaded to `stable-proposed-updates`.
    - When the time comes for a new point release, everything from `stable-proposed-updates` becomes the new `stable`.
    For the purposes of the changelog, the Debian Security Team signs off the distribution as `<release>-security` for security fixes. When it comes to `stable-updates`, though, since they originate as proposed updates destined for the next point release, in the changelog they are simply `<release>`. In other words, you will never see `-updates` in a changelog file.
    ### Debian Package Examples
    Let's look at some examples of what APT sees, on my Debian Buster virtual machine with the suites `buster`, `buster-updates`, and `buster-security`. Starting with a simple one:
    ```
    $ apt-cache policy vim
    vim:
    Installed: (none)
    Candidate: 2:8.1.0875-5+deb10u2
    Version table:
    2:8.1.0875-5+deb10u2 500
    500 http://deb.debian.org/debian buster/main amd64 Packages
    ```
    Vim has not had any stable updates or security fixes, so there is no Vim entry in the package index for `buster-updates`. There is only the Vim version that shipped with the last Buster point release.
    When it comes to stable updates, a nice example is antivirus data:
    ```
    $ apt-cache policy clamav
    clamav:
    Installed: (none)
    Candidate: 0.103.6+dfsg-0+deb10u1
    Version table:
    0.103.6+dfsg-0+deb10u1 500
    500 http://deb.debian.org/debian buster-updates/main amd64 Packages
    0.103.5+dfsg-0+deb10u1 500
    500 http://deb.debian.org/debian buster/main amd64 Packages
    ```
    There is the version that was shipped by the point release, and then the newer package that is in the update suite. APT will prefer the package with the higher version.
    Thunderbird, on the other hand, exemplifies how extreme the security suite can be:
    ```
    $ apt-cache policy thunderbird
    thunderbird:
    Installed: (none)
    Candidate: 1:91.12.0-1~deb10u1
    Version table:
    1:91.12.0-1~deb10u1 500
    500 http://security.debian.org/debian-security buster/updates/main amd64 Packages
    1:78.14.0-1~deb10u1 500
    500 http://deb.debian.org/debian buster/main amd64 Packages
    ```
    It seems that, in order to fix the exploits, the Security Team decided to execute a huge version bump.
    ### Ubuntu Repositories
    Ubuntu has the following repositories:
    - Ubuntu Archive, hosted at http://archive.ubuntu.com/ubuntu/ (first-party mirror), hxxp://CC.archive.ubuntu.com/ubuntu/ ([third-party country mirrors](https://wiki.ubuntu.com/Mirrors)), and [other third-party mirrors](https://launchpad.net/ubuntu/+archivemirrors). Has the following suites:
    - A suite for the latest point releases of each of the [releases in standard support](https://wiki.ubuntu.com/Releases).
    - This suite is held to high standards but shouldn't be used alone because it is frozen most of the time for point releases, like Debian stable releases.
    - Unlike Debian, there are no aliases such as `stable`.
    - A suite for the [next stable release](https://ubuntuforums.org/forumdisplay.php?f=427) (Ubuntu Development Version), `devel` [https://ubuntu.com/server/docs/upgrade-introduction#main-content].
    - This is one place where there is an alias - `devel` is aliased to the release's codename.
    - As with Debian Testing, `devel-updates` is empty/meaningless since `devel` is never finalized as a point release.
    - An additional suite for [stable release updates](https://wiki.ubuntu.com/StableReleaseUpdates) (SRUs) for each of the current stable releases, `<release>-updates`. Called "Recommended updates" in the GUI. [https://help.ubuntu.com/community/Repositories/Ubuntu#Install_updates_from:]
    - As with Debian's stable updates, this update channel is used for desirable but non-security bug fixes. It may be looser, with Ubuntu permitting the publishing of features in non-critical packages, and even entire new versions, granted the changes fall within the SRU qualifications.
    - Ubuntu utilizes [phased updates](https://web.archive.org/web/20170719231238/http://www.murraytwins.com/blog/?p=127) to deploy a update while checking for regressions.
    - For each of the stable releases, an additional suite for stable proposed updates, `<release>-proposed`.
    - This is the staging area for SRUs. Unlike Debian's proposed updates, *everything- that is pushed to the SRU suite is intended to be promoted to `<release>-updates` once tested.
    - Unlike Debian Testing, this *is- used for `devel`, because there is no equivalent of Debian Unstable.
    - For each of the stable releases, an additional suite for [backports](https://help.ubuntu.com/community/UbuntuBackports). Called "Unsupported updates" in the GUI/
    - As with Debian backports, these are provided without any security guarantee, and require pinning or explicit installation to use..
    - The preference is for backports to come from the the current Ubuntu stable release. [https://wiki.ubuntu.com/UbuntuBackports]
    - This is empty/meaningless for `devel`.
    - [Ubuntu Security](https://wiki.ubuntu.com/SecurityTeam), hosted at http://security.ubuntu.com/ubuntu/. Called "Security updates" in the GUI.
    - The Ubuntu Security team services the `<release>-security` suite, delivering [security updates](https://wiki.ubuntu.com/SecurityTeam/UpdateProcedures).
    - This suite is pushed to for as long as the release is not EoL.
    - These are based on the latest version of a package, out of everything except the Backports. [https://wiki.ubuntu.com/SecurityTeam/FAQ#How_are_the_.22-updates.22_and_.22-security.22_pockets_different.3F, https://wiki.ubuntu.com/SecurityTeam/UpdatePreparation]
    - The Ubuntu security repository is not meant to be mirrored from by a third party.
    - Like with Debian Security, the intent is for users to get these updates as quickly as possible.
    - Despite this, the Ubuntu Security repository and the Ubuntu Archive repository [appear to be one and the same](https://askubuntu.com/a/185444/497651), and as such the `<release>-security` suite exists on mirrors. It remains that the official Ubuntu Security repository is the first to update, and is recommended to stay in your `sources.list`.
    - After a short amount of time (less than an hour), security package updates get copied to `<release>-updates` for proper mirroring. [https://wiki.ubuntu.com/SecurityTeam/UpdatePublication#Announce_Publication]
    `release` + `release-security` is considered the minimum for a supported Ubuntu system. `release` + `release-security` + `release-updates` is the default for Ubuntu systems.
    Like we did for Ubuntu, let's go through the life of a package:
    - Upstream publishes source code.
    - Debian packager creates package.
    - A new release suite is created from the latest versions, and aliased to `devel`. Ubuntu Developers upgrade their packages to a new upstream release. [https://wiki.ubuntu.com/UbuntuDevelopment/ReleaseProcess]
    - By default (for most packages), this will be synced from Debian. [https://wiki.ubuntu.com/UbuntuDevelopment/PackageArchive#Syncing_and_Merging]
    - This happens automatically until the [Debian import freeze](https://wiki.ubuntu.com/DebianImportFreeze).
    - Otherwise, if Ubuntu modified the package (with `ubuntu` in the version), either:
    - The package is gotten back in sync.
    - A merge is performed and the divergence remains.
    - "This phase can be said to end when all packages have been brought up to date at least once, and the result has been sufficiently stabilized to produce the first milestone, with installable CD images."
    - The development release undergoes several freezes, until it becomes a release candidate. [https://wiki.ubuntu.com/UbuntuDevelopment/ReleaseProcess#Stabilization_.28Freeze.29]
    - Unlike Debian, Ubuntu has regular biannual releases. As a result, many freezes are [time-based](https://wiki.ubuntu.com/TimeBasedReleases).
    - The final release is made, and the first point release is made.
    - Stable release updates are made and pushed to `<release>-proposed`.
    - If a security issue occurs here, it's necessarily the maintainer's resposibility to handle it. [https://wiki.ubuntu.com/SecurityTeam/FAQ#How_are_components_and_pockets_used_in_the_builds.2C_and_how_do_they_affect_security_updates.3F]
    - After sufficient testing, the proposed updates are pulled into `<release>-updates` in phases, without rebuilding.
    - If a security issue occurs here or in the point release, the Security Team will patch the latest package from `<release>` + `<release>-security` + `<release>-updates`, and upload it to `<release>-security`.
    - It was mentioned before, but: After about a half hour, this will be mirrored to `<release>-updates` for better availability.
    - When a point release is scheduled:
    - Pushing updates to `<release>-proposed` is discouraged to allow all the existing updates to be flushed for `<release>-updates`.
    - The point release is published with the latest packages from `<release>` + `<release>-security` + `<release>-updates`.
    ### Ubuntu Package Examples
    Let's go through some `apt-cache` examples on Ubuntu. These will be more complicated than Debians', for less redundancy.
    First, a stable release update:
    ```
    $ apt-cache policy unattended-upgrades
    unattended-upgrades:
    Installed: 1.1ubuntu1.18.04.14
    Candidate: 1.1ubuntu1.18.04.14
    Version table:
    *** 1.1ubuntu1.18.04.14 500
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/main amd64 Packages
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/main i386 Packages
    100 /var/lib/dpkg/status
    1.1ubuntu1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic/main amd64 Packages
    500 http://mirrors.digitalocean.com/ubuntu bionic/main i386 Packages
    ```
    There's not much new here, other than this is a case where I actually have the package installed. Accordingly, there is a source for the package with priority `100`; that's the one that I have installed.
    We've seen ClamAV antivirutes updates be delivered as stable, non-security updates. In this case, though, there has been a security update issued:
    ```
    $ apt-cache policy clamav
    clamav:
    Installed: (none)
    Candidate: 0.103.6+dfsg-0ubuntu0.18.04.1
    Version table:
    0.103.6+dfsg-0ubuntu0.18.04.1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/main amd64 Packages
    500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
    0.99.4+addedllvm-0ubuntu1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic/main amd64 Packages
    ```
    We know that `bionic-security` gets mirrored to `bionic-updates` after a little bit. So, we can draw from this that:
    - The current Ubuntu Bionic Beaver point release ISO comes with ClamAV 0.99.4.
    - This is neither upstream nor Debian's ClamAV 0.99.4, as indicated by the `ubuntu` string in the version.
    - A [security issue](https://ubuntu.com/security/notices/USN-5423-1) was discovered, which affects ClamAV 0.99.4.
    - This [does affect Debian](https://security-tracker.debian.org/tracker/CVE-2022-20792) too, but it seems to have not been delegated to the Security Team.
    - To find this, I just looked up "ubuntu clamav cve" to get to the security notice.
    - The security team pushed ClamAV `0.103.6+dfsg-0ubuntu0.18.04.1` to `bionic-security`.
    - The updated package was pushed to `bionic-updates`, propagating to mirrors including the one I am using in this example.
    I can also tell that this package last got a security update by looking at the Ubuntu Packages Search, noticing that [there is a security version](https://web.archive.org/web/20220806220733/https://packages.ubuntu.com/bionic/clamav), and [it is the same version that is in the update suite](https://web.archive.org/web/20220806221059/https://packages.ubuntu.com/bionic-updates/clamav).
    Now, a mixed example:
    ```
    $ apt-cache policy librte-vhost17.11
    librte-vhost17.11:
    Installed: (none)
    Candidate: 17.11.10-0ubuntu0.1
    Version table:
    17.11.10-0ubuntu0.1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/main amd64 Packages
    17.11.9-0ubuntu18.04.2 500
    500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
    17.11.1-6 500
    500 http://mirrors.digitalocean.com/ubuntu bionic/main amd64 Packages
    ```
    This package received a security fix, and then an update fix that will be based on that. If this package gets another security fix, it will look something like this:
    ```
    17.11.11 500
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/main amd64 Packages
    500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages
    17.11.1-6 500
    500 http://mirrors.digitalocean.com/ubuntu bionic/main amd64 Packages
    ```
    The release version stays the same, and the security version is arbitrarily higher so that it will be used. `17.11.11` is a version with the security fix that was based on `17.11.10-0ubuntu0.1` from `bionic-updates`, because that was the latest package update.
    Now, a weirder one:
    ```
    libgnat-8-dbg-hppa-cross:
    Installed: (none)
    Candidate: 8.4.0-1ubuntu1~18.04cross1
    Version table:
    8.4.0-1ubuntu1~18.04cross1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/universe amd64 Packages
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/universe i386 Packages
    500 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages
    500 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages
    8.2.0-1ubuntu2~18.04cross1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/universe amd64 Packages
    500 http://mirrors.digitalocean.com/ubuntu bionic-updates/universe i386 Packages
    8-20180414-1ubuntu2cross1 500
    500 http://mirrors.digitalocean.com/ubuntu bionic/universe amd64 Packages
    500 http://mirrors.digitalocean.com/ubuntu bionic/universe i386 Packages
    ```
    There is an older version of this package in `bionic-updates` that will not be defaulted to in any circumstances. I guess that old packages aren't removed immediately from the repo.
    ## Debian and Ubuntu Components
    Now for something a little easier. Within the official suites, Debian has the following components: [https://www.debian.org/doc/debian-policy/ch-archive.html#archive-areas, https://wiki.debian.org/SourcesList#Example_sources.list]
    - `main`, for free (as in freedom) software that only requires other `main` packages, as determined by the [Debian Free Software Guidelines](https://wiki.debian.org/DFSGLicenses).
    - `contrib`, for free software with dependencies not in `main` or `contrib` (either as a package dependency, or by being an accessory).
    - `non-free`, for non-free software.
    `contrib` and `non-free` are not official parts of the Debian distribution, and are not maintained by the Security Team. [https://www.debian.org/security/faq#contrib]
    Ubuntu has the following components: [https://ubuntu.com/about/release-cycle, https://wiki.ubuntu.com/SecurityTeam/FAQ#Standard_Support, https://help.ubuntu.com/community/Repositories#Components, https://help.ubuntu.com/community/Repositories/Ubuntu#The_Four_Main_Repositories]
    - `main`, Canonical-maintained free packages.
    - `restricted`, Canonical-maintained non-free packages (mostly just proprietary drivers).
    - `universe`, community-maintained free packages.
    - `multiverse`, community-maintained non-free packages.
    The Security Team only maintains `main` and `restricted`, and for the duration of the standard support. [https://wiki.ubuntu.com/SecurityTeam/FAQ#Standard_Support]
    ## Upgrades
    https://help.ubuntu.com/community/AutoWeeklyUpdateHowTo
    [unattended-programs](https://github.com/mvo5/unattended-upgrades) is a Python program that uses APT to install upgrades without user intervention. By default, once enabled it will only upgrade packages from the `<release>-security` suite. Like with `apt upgrade`, it will not remove packages to satisfy a transaction. This is documented by Debian at https://wiki.debian.org/UnattendedUpgrades, and Ubuntu at https://help.ubuntu.com/community/AutomaticSecurityUpdates. This supersedes the older technique of running a cron job. [https://help.ubuntu.com/community/AutoWeeklyUpdateHowTo]
    When it comes to regular package upgrades, Ubuntu has its [Software Update GUI](https://launchpad.net/ubuntu/+source/update-manager).
    As for distribution upgrades, Ubuntu has an [upgrade GUI](https://ubuntu.com/tutorials/upgrading-ubuntu-desktop) and upgrade script [do-release-upgrade](https://ubuntu.com/server/docs/upgrade-introduction), both backed by [update-manager](https://launchpad.net/ubuntu/+source/update-manager) [https://askubuntu.com/a/409556/497651]. Debian provides [manual instructions](https://wiki.debian.org/DebianUpgrade), but a Debian user [suggests](https://wiki.debian.org/AutomatedUpgrade) that automation is possible, perhaps by adopting Ubuntu's work.
    ## Conclusion
    I started these notes to ascertain how Ubuntu's update channels work. The software update interface makes it seem like, when you only enable the security update channel, you only get security updates. However, after sifting through all the documentation, this isn't always the case: once a non-security bug fix has been pushed, future security bug fixes will be based on that latest package. Only using the security update channel does not control *which- updates (out of `<release>-security`, `<release-updates>`, and `<release-proposed>` by proxy) you get, but rather *when- you get them. The same goes for Debian.
    This info is useful to know when configuring unattended-upgrades, as it defaults to only using the security channel. It's my opinion that most users and system administrators are not given the precise information they need to make this decision, something which I think plagues the landscape of system administration on Ubuntu. I polished this into a longer form of writing to help combat cargo-cult administration by encouraging curiosity about the underlying systems.
    ## More info
    I have linked to the specific resources that I have used, but there are some more general pages that provide a good overview, including some links that I did not use:
    - <https://wiki.debian.org/PackageManagement>: Home page for package management techniques on Debian.
    - <https://wiki.ubuntu.com/SoftwareHandling>: Home page for package management techniques on Ubuntu.
    - <https://wiki.debian.org/Software>: Debian's end-user homepage for managing software.
    - <https://help.ubuntu.com/lts/ubuntu-help/addremove.html.en>: Ubuntu's end-user homepage for managing software.
    - <https://help.ubuntu.com/community/AptGet/Howto> end-user apt-get instructions.
    - <https://www.debian.org/devel/>: Debian developer resources.
    - <https://wiki.ubuntu.com/UbuntuDevelopers>: Ubuntu developer resources.
    - <http://www.ubuntu.com/testing/>: Ubuntu Development Version info.