Upgrade syden peertube to flake
[perso/Immae/Config/Nix.git] / DOCUMENTATION.md
1 Get Started
2 ===========
3
4 - You may run ./nixops/scripts/setup to setup the directory and variables
5 The script won’t do anything without asking first, you may stop at any
6 step.
7
8 Nix snippets
9 ============
10
11 - Evaluate nixos build:
12 `nix eval -f '<nixpkgs/nixos>' options.virtualisation.anbox.image.value.outPath`
13
14 Helpers / External documentation
15 ================================
16
17 - How to write good options in a module (not too much to ensure it’s maintainable, ...):
18 https://github.com/NixOS/rfcs/blob/e719102ace17c2b4a8b98efe0f08e344a7713dec/rfcs/0042-config-option.md
19 - SSL configuration: https://ssl-config.mozilla.org/
20 And check: https://www.ssllabs.com/
21
22 Channels
23 ========
24
25 - [channel](https://nixos.org/channels/nixos-20.03)
26 - nixpkgs-unstable: last version of nixpkgs
27 - nixos-unstable: last one guaranteed to build a correct system
28
29 The LDAP directory
30 ==================
31
32 The LDAP directory has been hand-tuned and may not have the same layout
33 as "regular" LDAP directories.
34
35 Sections
36 --------
37 The directory is divided in several sections:
38 - The "service" section (ou=services,dc=immae,dc=eu) contains all the
39 services. Each service usually has a list of dn (can be real user or
40 hosts or anything) that can access it. Conversely, a service usually
41 cannot list users he doesn’t have access to.
42 - The "hosts" section (ou=hosts,dc=immae,dc=eu) contains the identities
43 of hosts. They tend to be less used now. But sometimes they need to
44 have an identity (mostly to be able to send e-mails) The subsection
45 "roles" (ou=roles,ou=hosts,dc=immae,dc=eu) was from a Puppet age and
46 is deprecated. Only one host remains handled by Puppet (caldance) and
47 should be replaced with an internal VM.
48 - The "groups" section (ou=groups,dc=immae,dc=eu) contains the generic
49 groups of users not associated to a service.
50 - The "group_users" and "users" sections contain the users (usually with
51 password) of the infrastructure. The "users" section usally contains a
52 single "physical" person, while the "group_user" represents a shared
53 identity (plus one legacy "buildbot" system identity).
54
55 How does nixpkgs resolve
56 ========================
57
58 To build nixops machines
59 ------------------------
60
61 The `NIX_PATH` environment variable is built in nixops/Makefile and
62 contains three paths: nixpkgs, nixpkgsNext, nixpkgsPrevious. Only the
63 first one is actually used most of the time. Derivations that need
64 pinned nixpkgs should declare it in `nix/sources.json` (it’s the case
65 for some buildbot scripts for instance).
66
67 The config and overlays in standard directories will be read
68 (~/.config/nixpkgs) and parsed, but should not have any consequence on
69 the result.
70
71 In shells
72 ---------
73
74 When calling nix tools from shells, the environment variable `NIX_PATH`
75 will determine the `<nixpkgs>` version that will be used. In addition,
76 `~/.config/nixpkgs/overlays.nix` will add an overlay to this nixpkgs. In
77 its current state, it resolves to
78 builtins.attrValues (import "${thisRepository}/overlays")
79
80 To build Home-manager
81 ---------------------
82
83 Home-manager will use the same mechanisms as the shell and use the
84 configuration from ~/.config/nixpkgs/home.nix . In the current state,
85 pkgs will refer to the `<nixpkgs>` content, that is nixos-unstable
86 channel. The exception is `home.packages` (which will populate
87 ~/.nix-profile), which is built partly by importing the
88 `myEnvironments.immae-eu.packages` key of this repository’s
89 `default.nix`. This file uses the nix/sources.json "nixpkgs" key by
90 default, which may differ from `<nixpkgs>`.
91 Consistency needs to be ensured either by using nix/sources.json
92 everywhere or by using the unstable channel.
93 default.nix was changed to accept pkgs, making in effect use of the
94 unstable channel everywhere.
95
96 To build NUR
97 ------------
98
99 The NUR bot will evaluate default.nix with `pkgs` argument set with its
100 own nixpkgs version. It will not permit fetching urls related to nixpkgs
101 during the evaluation.
102
103 HTTP
104 ====
105
106 [Reference](https://infosec.mozilla.org/guidelines/web_security)
107 Protections that can be implemented:
108 - `Header always set Strict-Transport-Security "max-age=31536000"`
109 Force https on this domain for the age duration
110 - Public-Key-Pins -> dangerous if acme private key changes
111 - Always load script/style by https (don’t use `//`)
112 - Content-Security-Policy
113 Either as `<meta http-equiv="Content-Security-Policy" content="...">` (must be first one!)
114 Or as Content-Security-Policy header
115 - Restrictive referrer status
116
117 Packaging issues
118 ================
119
120 Yarn
121 ----
122
123 - Sometimes yarn will silently ignore package.json: the build will succeed but output will be empty. In that case it means that some mandatory fields are missing (mainly: `name`, `version`)
124 - If yarn complains about `TypeError: Cannot read property 'lang' of undefined`:
125 make sure that all package names in yarn-packages.nix finish in .tar.gz where due
126 - If yarn complains about `error Couldn't find the binary git`:
127 It’s related to the previous error:
128 - the initial yarn.lock and package.json contain some reference to github repositories.
129 - They need to be changed to a fixed version, and the resolved url needs to point to a tar.gz file.
130 - Example (the diff should be saved as a patch and applied to the yarn derivation build):
131 --- a/package.json
132 +++ b/package.json
133 @@ -2,9 +2,9 @@
134 "name": "foo",
135 "private": true,
136 "dependencies": {
137 - "@danielfarrell/bootstrap-combobox": "https://github.com/berrnd/bootstrap-combobox.git#master",
138 + "@danielfarrell/bootstrap-combobox": "^1.1.8",
139 "@fortawesome/fontawesome-free": "^5.12.1",
140 "animate.css": "^3.7.2",
141 "bootbox": "^5.3.2",
142 "bootstrap": "^4.3.1",
143 --- a/yarn.lock
144 +++ b/yarn.lock
145 @@ -2,18 +2,18 @@
146 # yarn lockfile v1
147 -"@danielfarrell/bootstrap-combobox@https://github.com/berrnd/bootstrap-combobox.git#master":
148 +"@danielfarrell/bootstrap-combobox@^1.1.8":
149 version "1.1.8"
150 - resolved "https://github.com/berrnd/bootstrap-combobox.git#fcf0110146f4daab94888234c57d198b4ca5f129"
151 + resolved "https://github.com/berrnd/bootstrap-combobox/archive/fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz"
152 - Also the yarn-packages.nix needs to be changed accordingly: the name of the package in that file needs to match the one computed by yarn2nix (special chars replaced with underscores), and the url needs to point to the correct url too
153 - Example:
154 --- a/yarn-packages.nix
155 +++ b/yarn-packages.nix
156 @@ -3,11 +3,11 @@
157 {
158 - name = "https___github.com_berrnd_bootstrap_combobox.git";
159 + name = "https___github.com_berrnd_bootstrap_combobox_archive_fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
160 path = fetchurl {
161 - name = "https___github.com_berrnd_bootstrap_combobox.git";
162 - url = "https://github.com/berrnd/bootstrap-combobox.git";
163 - sha1 = "fcf0110146f4daab94888234c57d198b4ca5f129";
164 + name = "https___github.com_berrnd_bootstrap_combobox_archive_fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
165 + url = "https://github.com/berrnd/bootstrap-combobox/archive/fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
166 + sha1 = "0nvnkr1w9nnn7r2h63zmkjpphawwmfca";
167 };
168 }
169
170 NUR
171 ===
172
173 [Link to the CI latest job](https://travis-ci.com/github/nix-community/NUR)
174
175 Upgrades
176 ========
177
178 Things to look at during upgrades:
179
180 Upgrade to latest unstable
181 -------------------
182
183 - Nothing in particular yet
184
185 Etherpad-lite
186 -------------
187
188 When upgrading etherpad-lite modules, make sure that possible hacks are
189 carried along (usually as preBuild hook). Run the following command in
190 the module directory:
191
192 node2nix -i node-packages.json
193
194 Nodejs
195 ------
196
197 - The nodeHeaders will change at each bump, making previous hash
198 incorrect (current at unstable: 1df3yhlwlvai0m9kvjyknjg11hnw0kj0rnhyzbwvsfjnmr6z8r76)
199 - At runtime, peertube may complain about mismatching NODE_MODULE_VERSION for bcrypt. Check the url to make sure that it contains the same module version:
200 [](https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.2/bcrypt_lib-v3.0.2-node-v64-linux-x64-glibc.tar.gz)
201 change the v64 to whichever value corresponds in
202 [](https://nodejs.org/en/download/releases/)
203
204 PHP/Mysql/PAM
205 -------------
206
207 adminer installation requires pam module (it’s the only one, since pam
208 is only used in php environment by "humans" account). It currently
209 doesn’t work, I couldn’t find a way to properly make php find the
210 mariadb include files. It seems like php74 might solve the issue
211 https://stackoverflow.com/questions/50026939/php-mysqli-connect-authentication-method-unknown-to-the-client-caching-sha2-pa
212
213 Postgresql
214 ----------
215
216 Postgresql is linked to glibc version. Any change in this version may
217 corrupt the indexes:
218 [](https://wiki.postgresql.org/wiki/Locale_data_changes#What_to_do)
219
220 For postgresql major upgrade:
221 - Change the postgresql overlay to define `postgresql_next`
222 - import `modules/private/databases/postgresql_upgrade.nix` and adjust the header to mark the cluster to upgrade.
223 - Deploy it
224 - run `upgrade-pg-cluster` as root
225 - Add an overlay for the server:
226 ```
227 nixpkgs.overlays = [ (self: super: {
228 postgresql = self.postgresql_next;
229 }) ];
230 ```
231 - Deploy (make sure that the new datadir is the one that gets used)
232 - If everyone has upgraded, the per-server overlay may be removed and
233 the global one modified
234
235 Nextcloud
236 ---------
237
238 - Do not skip major versions!
239 - Put nextcloud in maintenance mode :
240 ```
241 nextcloud-occ maintenance:mode --on
242 ```
243 - Do a backup :
244 ```
245 sudo -u postgres pg_dump -n owncloud webapps > nextcloud.sql
246 ```
247 - Upgrade
248 - Run the upgrade task :
249 ```
250 nextcloud-occ upgrade
251 ```
252 - Stop maintenance mode :
253 ```
254 nextcloud-occ maintenance:mode --off
255 ```
256 - Issues :
257 https://docs.nextcloud.com/server/16/admin_manual/maintenance/manual_upgrade.html