Upgrade syden peertube to flake
[perso/Immae/Config/Nix.git] / DOCUMENTATION.md
CommitLineData
a0d2610c
IB
1Get Started
2===========
3
4- You may run ./nixops/scripts/setup to setup the directory and variables
5 The script won’t do anything without asking first, you may stop at any
6 step.
7
8Nix snippets
9============
10
11- Evaluate nixos build:
12 `nix eval -f '<nixpkgs/nixos>' options.virtualisation.anbox.image.value.outPath`
13
14Helpers / External documentation
15================================
16
17- How to write good options in a module (not too much to ensure it’s maintainable, ...):
18 https://github.com/NixOS/rfcs/blob/e719102ace17c2b4a8b98efe0f08e344a7713dec/rfcs/0042-config-option.md
19- SSL configuration: https://ssl-config.mozilla.org/
20 And check: https://www.ssllabs.com/
21
22Channels
23========
24
25- [channel](https://nixos.org/channels/nixos-20.03)
26- nixpkgs-unstable: last version of nixpkgs
27- nixos-unstable: last one guaranteed to build a correct system
28
29The LDAP directory
30==================
31
32The LDAP directory has been hand-tuned and may not have the same layout
33as "regular" LDAP directories.
34
35Sections
36--------
37The directory is divided in several sections:
38- The "service" section (ou=services,dc=immae,dc=eu) contains all the
39 services. Each service usually has a list of dn (can be real user or
40 hosts or anything) that can access it. Conversely, a service usually
41 cannot list users he doesn’t have access to.
42- The "hosts" section (ou=hosts,dc=immae,dc=eu) contains the identities
43 of hosts. They tend to be less used now. But sometimes they need to
44 have an identity (mostly to be able to send e-mails) The subsection
45 "roles" (ou=roles,ou=hosts,dc=immae,dc=eu) was from a Puppet age and
46 is deprecated. Only one host remains handled by Puppet (caldance) and
47 should be replaced with an internal VM.
48- The "groups" section (ou=groups,dc=immae,dc=eu) contains the generic
49 groups of users not associated to a service.
50- The "group_users" and "users" sections contain the users (usually with
51 password) of the infrastructure. The "users" section usally contains a
52 single "physical" person, while the "group_user" represents a shared
53 identity (plus one legacy "buildbot" system identity).
54
55How does nixpkgs resolve
56========================
57
58To build nixops machines
59------------------------
60
61The `NIX_PATH` environment variable is built in nixops/Makefile and
62contains three paths: nixpkgs, nixpkgsNext, nixpkgsPrevious. Only the
63first one is actually used most of the time. Derivations that need
64pinned nixpkgs should declare it in `nix/sources.json` (it’s the case
65for some buildbot scripts for instance).
66
67The config and overlays in standard directories will be read
68(~/.config/nixpkgs) and parsed, but should not have any consequence on
69the result.
70
71In shells
72---------
73
74When calling nix tools from shells, the environment variable `NIX_PATH`
75will determine the `<nixpkgs>` version that will be used. In addition,
76`~/.config/nixpkgs/overlays.nix` will add an overlay to this nixpkgs. In
77its current state, it resolves to
78 builtins.attrValues (import "${thisRepository}/overlays")
79
80To build Home-manager
81---------------------
82
83Home-manager will use the same mechanisms as the shell and use the
84configuration from ~/.config/nixpkgs/home.nix . In the current state,
85pkgs will refer to the `<nixpkgs>` content, that is nixos-unstable
86channel. The exception is `home.packages` (which will populate
87~/.nix-profile), which is built partly by importing the
88`myEnvironments.immae-eu.packages` key of this repository’s
89`default.nix`. This file uses the nix/sources.json "nixpkgs" key by
90default, which may differ from `<nixpkgs>`.
91Consistency needs to be ensured either by using nix/sources.json
92everywhere or by using the unstable channel.
93default.nix was changed to accept pkgs, making in effect use of the
94unstable channel everywhere.
95
96To build NUR
97------------
98
99The NUR bot will evaluate default.nix with `pkgs` argument set with its
100own nixpkgs version. It will not permit fetching urls related to nixpkgs
101during the evaluation.
102
103HTTP
104====
105
106[Reference](https://infosec.mozilla.org/guidelines/web_security)
107Protections that can be implemented:
108- `Header always set Strict-Transport-Security "max-age=31536000"`
109 Force https on this domain for the age duration
110- Public-Key-Pins -> dangerous if acme private key changes
111- Always load script/style by https (don’t use `//`)
112- Content-Security-Policy
113 Either as `<meta http-equiv="Content-Security-Policy" content="...">` (must be first one!)
114 Or as Content-Security-Policy header
115- Restrictive referrer status
116
117Packaging issues
118================
119
120Yarn
121----
122
123- Sometimes yarn will silently ignore package.json: the build will succeed but output will be empty. In that case it means that some mandatory fields are missing (mainly: `name`, `version`)
124- If yarn complains about `TypeError: Cannot read property 'lang' of undefined`:
125 make sure that all package names in yarn-packages.nix finish in .tar.gz where due
126- If yarn complains about `error Couldn't find the binary git`:
127 It’s related to the previous error:
128 - the initial yarn.lock and package.json contain some reference to github repositories.
129 - They need to be changed to a fixed version, and the resolved url needs to point to a tar.gz file.
130 - Example (the diff should be saved as a patch and applied to the yarn derivation build):
131 --- a/package.json
132 +++ b/package.json
133 @@ -2,9 +2,9 @@
134 "name": "foo",
135 "private": true,
136 "dependencies": {
137 - "@danielfarrell/bootstrap-combobox": "https://github.com/berrnd/bootstrap-combobox.git#master",
138 + "@danielfarrell/bootstrap-combobox": "^1.1.8",
139 "@fortawesome/fontawesome-free": "^5.12.1",
140 "animate.css": "^3.7.2",
141 "bootbox": "^5.3.2",
142 "bootstrap": "^4.3.1",
143 --- a/yarn.lock
144 +++ b/yarn.lock
145 @@ -2,18 +2,18 @@
146 # yarn lockfile v1
147 -"@danielfarrell/bootstrap-combobox@https://github.com/berrnd/bootstrap-combobox.git#master":
148 +"@danielfarrell/bootstrap-combobox@^1.1.8":
149 version "1.1.8"
150 - resolved "https://github.com/berrnd/bootstrap-combobox.git#fcf0110146f4daab94888234c57d198b4ca5f129"
151 + resolved "https://github.com/berrnd/bootstrap-combobox/archive/fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz"
152 - Also the yarn-packages.nix needs to be changed accordingly: the name of the package in that file needs to match the one computed by yarn2nix (special chars replaced with underscores), and the url needs to point to the correct url too
153 - Example:
154 --- a/yarn-packages.nix
155 +++ b/yarn-packages.nix
156 @@ -3,11 +3,11 @@
157 {
158 - name = "https___github.com_berrnd_bootstrap_combobox.git";
159 + name = "https___github.com_berrnd_bootstrap_combobox_archive_fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
160 path = fetchurl {
161 - name = "https___github.com_berrnd_bootstrap_combobox.git";
162 - url = "https://github.com/berrnd/bootstrap-combobox.git";
163 - sha1 = "fcf0110146f4daab94888234c57d198b4ca5f129";
164 + name = "https___github.com_berrnd_bootstrap_combobox_archive_fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
165 + url = "https://github.com/berrnd/bootstrap-combobox/archive/fcf0110146f4daab94888234c57d198b4ca5f129.tar.gz";
166 + sha1 = "0nvnkr1w9nnn7r2h63zmkjpphawwmfca";
167 };
168 }
169
170NUR
171===
172
173[Link to the CI latest job](https://travis-ci.com/github/nix-community/NUR)
174
175Upgrades
176========
177
178Things to look at during upgrades:
179
180Upgrade to latest unstable
181-------------------
182
183- Nothing in particular yet
184
185Etherpad-lite
186-------------
187
188When upgrading etherpad-lite modules, make sure that possible hacks are
189carried along (usually as preBuild hook). Run the following command in
190the module directory:
191
192 node2nix -i node-packages.json
193
194Nodejs
195------
196
197- The nodeHeaders will change at each bump, making previous hash
198 incorrect (current at unstable: 1df3yhlwlvai0m9kvjyknjg11hnw0kj0rnhyzbwvsfjnmr6z8r76)
199- At runtime, peertube may complain about mismatching NODE_MODULE_VERSION for bcrypt. Check the url to make sure that it contains the same module version:
200 [](https://github.com/kelektiv/node.bcrypt.js/releases/download/v3.0.2/bcrypt_lib-v3.0.2-node-v64-linux-x64-glibc.tar.gz)
201 change the v64 to whichever value corresponds in
202 [](https://nodejs.org/en/download/releases/)
203
204PHP/Mysql/PAM
205-------------
206
207adminer installation requires pam module (it’s the only one, since pam
208is only used in php environment by "humans" account). It currently
209doesn’t work, I couldn’t find a way to properly make php find the
210mariadb include files. It seems like php74 might solve the issue
211https://stackoverflow.com/questions/50026939/php-mysqli-connect-authentication-method-unknown-to-the-client-caching-sha2-pa
212
213Postgresql
214----------
215
216Postgresql is linked to glibc version. Any change in this version may
217corrupt the indexes:
218[](https://wiki.postgresql.org/wiki/Locale_data_changes#What_to_do)
219
220For postgresql major upgrade:
221- Change the postgresql overlay to define `postgresql_next`
222- import `modules/private/databases/postgresql_upgrade.nix` and adjust the header to mark the cluster to upgrade.
223- Deploy it
224- run `upgrade-pg-cluster` as root
225- Add an overlay for the server:
226 ```
227 nixpkgs.overlays = [ (self: super: {
228 postgresql = self.postgresql_next;
229 }) ];
230 ```
231- Deploy (make sure that the new datadir is the one that gets used)
232- If everyone has upgraded, the per-server overlay may be removed and
233 the global one modified
234
235Nextcloud
236---------
237
238- Do not skip major versions!
239- Put nextcloud in maintenance mode :
240 ```
241 nextcloud-occ maintenance:mode --on
242 ```
243- Do a backup :
244 ```
245 sudo -u postgres pg_dump -n owncloud webapps > nextcloud.sql
246 ```
247- Upgrade
248- Run the upgrade task :
249 ```
250 nextcloud-occ upgrade
251 ```
252 - Stop maintenance mode :
253 ```
254 nextcloud-occ maintenance:mode --off
255 ```
256- Issues :
257 https://docs.nextcloud.com/server/16/admin_manual/maintenance/manual_upgrade.html