DOCUMENTATION.md

   1 Get Started
   2 ===========
   3
   4 - You may run ./nixops/scripts/setup to setup the directory and variables
   5   The script won’t do anything without asking first, you may stop at any
   6   step.
   7
   8 Nix snippets
   9 ============
  10
  11 - Evaluate nixos build:
  12   `nix eval -f '<nixpkgs/nixos>' options.virtualisation.anbox.image.value.outPath`
  13
  14 Helpers / External documentation
  15 ================================
  16
  17 - How to write good options in a module (not too much to ensure it’s maintainable, ...):
  18   https://github.com/NixOS/rfcs/blob/e719102ace17c2b4a8b98efe0f08e344a7713dec/rfcs/0042-config-option.md
  19 - SSL configuration: https://ssl-config.mozilla.org/
  20   And check: https://www.ssllabs.com/
  21
  22 Channels
  23 ========
  24
  25 - [channel](https://nixos.org/channels/nixos-20.03)
  26 - nixpkgs-unstable: last version of nixpkgs
  27 - nixos-unstable: last one guaranteed to build a correct system
  28
  29 The LDAP directory
  30 ==================
  31
  32 The LDAP directory has been hand-tuned and may not have the same layout
  33 as "regular" LDAP directories.
  34
  35 Sections
  36 --------
  37 The directory is divided in several sections:
  38 - The "service" section (ou=services,dc=immae,dc=eu) contains all the
  39   services. Each service usually has a list of dn (can be real user or
  40   hosts or anything) that can access it. Conversely, a service usually
  41   cannot list users he doesn’t have access to.
  42 - The "hosts" section (ou=hosts,dc=immae,dc=eu) contains the identities
  43   of hosts. They tend to be less used now. But sometimes they need to
  44   have an identity (mostly to be able to send e-mails) The subsection
  45   "roles" (ou=roles,ou=hosts,dc=immae,dc=eu) was from a Puppet age and
  46   is deprecated. Only one host remains handled by Puppet (caldance) and
  47   should be replaced with an internal VM.
  48 - The "groups" section (ou=groups,dc=immae,dc=eu) contains the generic
  49   groups of users not associated to a service.
  50 - The "group_users" and "users" sections contain the users (usually with
  51   password) of the infrastructure. The "users" section usally contains a
  52   single "physical" person, while the "group_user" represents a shared
  53   identity (plus one legacy "buildbot" system identity).
  54
  55 How does nixpkgs resolve
  56 ========================
  57
  58 To build nixops machines
  59 ------------------------
  60
  61 The `NIX_PATH` environment variable is built in nixops/Makefile and
  62 contains three paths: nixpkgs, nixpkgsNext, nixpkgsPrevious. Only the
  63 first one is actually used most of the time. Derivations that need
  64 pinned nixpkgs should declare it in `nix/sources.json` (it’s the case
  65 for some buildbot scripts for instance).
  66
  67 The config and overlays in standard directories will be read
  68 (~/.config/nixpkgs) and parsed, but should not have any consequence on
  69 the result.
  70
  71 In shells
  72 ---------
  73
  74 When calling nix tools from shells, the environment variable `NIX_PATH`
  75 will determine the `<nixpkgs>` version that will be used. In addition,
  76 `~/.config/nixpkgs/overlays.nix` will add an overlay to this nixpkgs. In
  77 its current state, it resolves to
  78     builtins.attrValues (import "${thisRepository}/overlays")
  79
  80 To build Home-manager
  81 ---------------------
  82
  83 Home-manager will use the same mechanisms as the shell and use the
  84 configuration from ~/.config/nixpkgs/home.nix . In the current state,
  85 pkgs will refer to the `<nixpkgs>` content, that is nixos-unstable
  86 channel. The exception is `home.packages` (which will populate
  87 ~/.nix-profile), which is built partly by importing the
  88 `myEnvironments.immae-eu.packages` key of this repository’s
  89 `default.nix`. This file uses the nix/sources.json "nixpkgs" key by
  90 default, which may differ from `<nixpkgs>`.
  91 Consistency needs to be ensured either by using nix/sources.json
  92 everywhere or by using the unstable channel.
  93 default.nix was changed to accept pkgs, making in effect use of the
  94 unstable channel everywhere.
  95
  96 To build NUR
  97 ------------
  98
  99 The NUR bot will evaluate default.nix with `pkgs` argument set with its
 100 own nixpkgs version. It will not permit fetching urls related to nixpkgs
 101 during the evaluation.
 102
 103 HTTP
 104 ====
 105
 106 [Reference](https://infosec.mozilla.org/guidelines/web_security)
 107 Protections that can be implemented:
 108 - `Header always set Strict-Transport-Security "max-age=31536000"`
 109   Force https on this domain for the age duration
 110 - Public-Key-Pins -> dangerous if acme private key changes
 111 - Always load script/style by https (don’t use `//`)
 112 - Content-Security-Policy
 113   Either as `<meta http-equiv="Content-Security-Policy" content="...">` (must be first one!)
 114   Or as Content-Security-Policy header
 115 - Restrictive referrer status
 116
 117 Packaging issues
 118 ================
 119
 120 Yarn
 121 ----
 122
 123 - Sometimes yarn will silently ignore package.json: the build will succeed but output will be empty. In that case it means that some mandatory fields are missing (mainly: `name`, `version`)
 124 - If yarn complains about `TypeError: Cannot read property 'lang' of undefined`:
 125   make sure that all package names in yarn-packages.nix finish in .tar.gz where due
 126 - If yarn complains about `error Couldn't find the binary git`:
 127   It’s related to the previous error:
 128   - the initial yarn.lock and package.json contain some reference to github repositories.
 129   - They need to be changed to a fixed version, and the resolved url needs to point to a tar.gz file.
 130   - Example (the diff should be saved as a patch and applied to the yarn derivation build):
 131         --- a/package.json
 132         +++ b/package.json
 133         @@ -2,9 +2,9 @@
 134                 "name": "foo",
 135                 "private": true,
 136                 "dependencies": {
 137         -               "@danielfarrell/bootstrap-combobox": "https://github.com/berrnd/bootstrap-combobox.git#master",
 138         +               "@danielfarrell/bootstrap-combobox": "^1.1.8",
 139                         "@fortawesome/fontawesome-free": "^5.12.1",
 140                         "animate.css": "^3.7.2",
 141                         "bootbox": "^5.3.2",
 142                         "bootstrap": "^4.3.1",
 143         --- a/yarn.lock
 144         +++ b/yarn.lock
 145         @@ -2,18 +2,18 @@
 146          # yarn lockfile v1
 147         -"@danielfarrell/bootstrap-combobox@https://github.com/berrnd/bootstrap-combobox.git#master":
 148         +"@danielfarrell/bootstrap-combobox@^1.1.8":
 149            version "1.1.8"
 150         -  resolved "https://github.com/berrnd/bootstrap-combobox.git#fcf0110146f4daab94888234c57d198b4ca5f129"
 151         +  resolved "https://github.com/berrnd/bootstrap-combobox/archive/fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz"
 152   - Also the yarn-packages.nix needs to be changed accordingly: the name of the package in that file needs to match the one computed by yarn2nix (special chars replaced with underscores), and the url needs to point to the correct url too
 153   - Example:
 154         --- a/yarn-packages.nix
 155         +++ b/yarn-packages.nix
 156         @@ -3,11 +3,11 @@
 157              {
 158         -      name = "https___github.com_berrnd_bootstrap_combobox.git";
 159         +      name = "https___github.com_berrnd_bootstrap_combobox_archive_fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
 160                path = fetchurl {
 161         -        name = "https___github.com_berrnd_bootstrap_combobox.git";
 162         -        url  = "https://github.com/berrnd/bootstrap-combobox.git";
 163         -        sha1 = "fcf0110146f4daab94888234c57d198b4ca5f129";
 164         +        name = "https___github.com_berrnd_bootstrap_combobox_archive_fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
 165         +        url  = "https://github.com/berrnd/bootstrap-combobox/archive/fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
 166         +        sha1 = "0nvnkr1w9nnn7r2h63zmkjpphawwmfca";
 167                };
 168              }
 169
 170 NUR
 171 ===
 172
 173 [Link to the CI latest job](https://travis-ci.com/github/nix-community/NUR)
 174
 175 Upgrades
 176 ========
 177
 178 Things to look at during upgrades:
 179
 180 Upgrade to latest unstable
 181 -------------------
 182
 183 - Nothing in particular yet
 184
 185 Etherpad-lite
 186 -------------
 187
 188 When upgrading etherpad-lite modules, make sure that possible hacks are
 189 carried along (usually as preBuild hook). Run the following command in
 190 the module directory:
 191
 192     node2nix -i node-packages.json
 193
 194 Nodejs
 195 ------
 196
 197 - The nodeHeaders will change at each bump, making previous hash
 198   incorrect (current at unstable: 1df3yhlwlvai0m9kvjyknjg11hnw0kj0rnhyzbwvsfjnmr6z8r76)
 199 - At runtime, peertube may complain about mismatching NODE_MODULE_VERSION for bcrypt. Check the url to make sure that it contains the same module version:
 200   [](https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.2/bcrypt_lib-v3.0.2-node-v64-linux-x64-glibc.tar.gz)
 201   change the v64 to whichever value corresponds in
 202   [](https://nodejs.org/en/download/releases/)
 203
 204 PHP/Mysql/PAM
 205 -------------
 206
 207 adminer installation requires pam module (it’s the only one, since pam
 208 is only used in php environment by "humans" account). It currently
 209 doesn’t work, I couldn’t find a way to properly make php find the
 210 mariadb include files. It seems like php74 might solve the issue
 211 https://stackoverflow.com/questions/50026939/php-mysqli-connect-authentication-method-unknown-to-the-client-caching-sha2-pa
 212
 213 Postgresql
 214 ----------
 215
 216 Postgresql is linked to glibc version. Any change in this version may
 217 corrupt the indexes:
 218 [](https://wiki.postgresql.org/wiki/Locale_data_changes#What_to_do)
 219
 220 For postgresql major upgrade:
 221 - Change the postgresql overlay to define `postgresql_next`
 222 - import `modules/private/databases/postgresql_upgrade.nix` and adjust the header to mark the cluster to upgrade.
 223 - Deploy it
 224 - run `upgrade-pg-cluster` as root
 225 - Add an overlay for the server:
 226   ```
 227   nixpkgs.overlays = [ (self: super: {
 228     postgresql = self.postgresql_next;
 229   }) ];
 230   ```
 231 - Deploy (make sure that the new datadir is the one that gets used)
 232 - If everyone has upgraded, the per-server overlay may be removed and
 233   the global one modified
 234
 235 Nextcloud
 236 ---------
 237
 238 - Do not skip major versions!
 239 - Put nextcloud in maintenance mode :
 240   ```
 241   nextcloud-occ maintenance:mode --on
 242   ```
 243 - Do a backup :
 244   ```
 245   sudo -u postgres pg_dump -n owncloud webapps > nextcloud.sql
 246   ```
 247 - Upgrade
 248 - Run the upgrade task :
 249   ```
 250   nextcloud-occ upgrade
 251   ```
 252  - Stop maintenance mode :
 253   ```
 254   nextcloud-occ maintenance:mode --off
 255   ```
 256 - Issues :
 257   https://docs.nextcloud.com/server/16/admin_manual/maintenance/manual_upgrade.html