diff options
author | Chocobozzz <me@florianbigard.com> | 2023-07-31 14:34:36 +0200 |
---|---|---|
committer | Chocobozzz <me@florianbigard.com> | 2023-08-11 15:02:33 +0200 |
commit | 3a4992633ee62d5edfbb484d9c6bcb3cf158489d (patch) | |
tree | e4510b39bdac9c318fdb4b47018d08f15368b8f0 /support/doc/development/server.md | |
parent | 04d1da5621d25d59bd5fa1543b725c497bf5d9a8 (diff) | |
download | PeerTube-3a4992633ee62d5edfbb484d9c6bcb3cf158489d.tar.gz PeerTube-3a4992633ee62d5edfbb484d9c6bcb3cf158489d.tar.zst PeerTube-3a4992633ee62d5edfbb484d9c6bcb3cf158489d.zip |
Migrate server to ESM
Sorry for the very big commit that may lead to git log issues and merge
conflicts, but it's a major step forward:
* Server can be faster at startup because imports() are async and we can
easily lazy import big modules
* Angular doesn't seem to support ES import (with .js extension), so we
had to correctly organize peertube into a monorepo:
* Use yarn workspace feature
* Use typescript reference projects for dependencies
* Shared projects have been moved into "packages", each one is now a
node module (with a dedicated package.json/tsconfig.json)
* server/tools have been moved into apps/ and is now a dedicated app
bundled and published on NPM so users don't have to build peertube
cli tools manually
* server/tests have been moved into packages/ so we don't compile
them every time we want to run the server
* Use isolatedModule option:
* Had to move from const enum to const
(https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums)
* Had to explictely specify "type" imports when used in decorators
* Prefer tsx (that uses esbuild under the hood) instead of ts-node to
load typescript files (tests with mocha or scripts):
* To reduce test complexity as esbuild doesn't support decorator
metadata, we only test server files that do not import server
models
* We still build tests files into js files for a faster CI
* Remove unmaintained peertube CLI import script
* Removed some barrels to speed up execution (less imports)
Diffstat (limited to 'support/doc/development/server.md')
-rw-r--r-- | support/doc/development/server.md | 66 |
1 files changed, 33 insertions, 33 deletions
diff --git a/support/doc/development/server.md b/support/doc/development/server.md index 7a9fa571f..5c83af704 100644 --- a/support/doc/development/server.md +++ b/support/doc/development/server.md | |||
@@ -1,11 +1,11 @@ | |||
1 | # Server code | 1 | # Server code |
2 | 2 | ||
3 | ## Database model typing | 3 | ## Database model typing |
4 | 4 | ||
5 | Sequelize models contain optional fields corresponding to table joins. | 5 | Sequelize models contain optional fields corresponding to table joins. |
6 | For example, `VideoModel` has a `VideoChannel?: VideoChannelModel` field. It can be filled if the SQL query joined with the `videoChannel` table or empty if not. | 6 | For example, `VideoModel` has a `VideoChannel?: VideoChannelModel` field. It can be filled if the SQL query joined with the `videoChannel` table or empty if not. |
7 | It can be difficult in TypeScript to understand if a function argument expects associations to be filled or not. | 7 | It can be difficult in TypeScript to understand if a function argument expects associations to be filled or not. |
8 | To improve clarity and reduce bugs, PeerTube defines multiple versions of a database model depending on its associations in `server/types/models/`. | 8 | To improve clarity and reduce bugs, PeerTube defines multiple versions of a database model depending on its associations in `server/server/types/models/`. |
9 | These models start with `M` and by default do not include any association. `MVideo` for example corresponds to `VideoModel` without any association, where `VideoChannel` attribute doesn't exist. On the other hand, `MVideoWithChannel` is a `MVideo` that has a `VideoChannel` field. This way, a function that accepts `video: MVideoWithChannel` argument expects a video with channel populated. Main PeerTube code should never use `...Model` (`VideoModel`) database type, but always `M...` instead (`MVideo`, `MVideoChannel` etc). | 9 | These models start with `M` and by default do not include any association. `MVideo` for example corresponds to `VideoModel` without any association, where `VideoChannel` attribute doesn't exist. On the other hand, `MVideoWithChannel` is a `MVideo` that has a `VideoChannel` field. This way, a function that accepts `video: MVideoWithChannel` argument expects a video with channel populated. Main PeerTube code should never use `...Model` (`VideoModel`) database type, but always `M...` instead (`MVideo`, `MVideoChannel` etc). |
10 | 10 | ||
11 | ## Add a new feature walkthrough | 11 | ## Add a new feature walkthrough |
@@ -16,67 +16,67 @@ Some of these may be optional (for example your new endpoint may not need to sen | |||
16 | * Configuration: | 16 | * Configuration: |
17 | - Add you new configuration key in `config/default.yaml` and `config/production.yaml` | 17 | - Add you new configuration key in `config/default.yaml` and `config/production.yaml` |
18 | - If you configuration needs to be different in dev or tests environments, also update `config/dev.yaml` and `config/test.yaml` | 18 | - If you configuration needs to be different in dev or tests environments, also update `config/dev.yaml` and `config/test.yaml` |
19 | - Load your configuration in `server/initializers/config.ts` | 19 | - Load your configuration in `server/server/initializers/config.ts` |
20 | - Check new configuration keys are set in `server/initializers/checker-before-init.ts` | 20 | - Check new configuration keys are set in `server/server/initializers/checker-before-init.ts` |
21 | - You can also ensure configuration consistency in `server/initializers/checker-after-init.ts` | 21 | - You can also ensure configuration consistency in `server/server/initializers/checker-after-init.ts` |
22 | - If you want your configuration to be available in the client: | 22 | - If you want your configuration to be available in the client: |
23 | + Add your field in `shared/models/server/server-config.model.ts` | 23 | + Add your field in `packages/models/server/server/server-config.model.ts` |
24 | + Update `server/lib/server-config-manager.ts` to include your new configuration | 24 | + Update `server/server/lib/server-config-manager.ts` to include your new configuration |
25 | - If you want your configuration to be updatable by the web admin in the client: | 25 | - If you want your configuration to be updatable by the web admin in the client: |
26 | + Add your field in `shared/models/server/custom-config.model.ts` | 26 | + Add your field in `packages/models/server/server/custom-config.model.ts` |
27 | + Add the configuration to the config object in the `server/controllers/api/config.ts` controller | 27 | + Add the configuration to the config object in the `server/server/controllers/api/config.ts` controller |
28 | * Controllers: | 28 | * Controllers: |
29 | - Create the controller file and fill it with your REST API routes | 29 | - Create the controller file and fill it with your REST API routes |
30 | - Import and use your controller in the parent controller | 30 | - Import and use your controller in the parent controller |
31 | * Middlewares: | 31 | * Middlewares: |
32 | - Create your validator middleware in `server/middlewares/validators` that will be used by your controllers | 32 | - Create your validator middleware in `server/server/middlewares/validators` that will be used by your controllers |
33 | - Add your new middleware file `server/middlewares/validators/index.ts` so it's easier to import | 33 | - Add your new middleware file `server/server/middlewares/validators/index.ts` so it's easier to import |
34 | - Create the entry in `server/types/express.d.ts` to attach the database model loaded by your middleware to the express response | 34 | - Create the entry in `server/server/types/express.d.ts` to attach the database model loaded by your middleware to the express response |
35 | * Validators: | 35 | * Validators: |
36 | - Create your validators that will be used by your middlewares in `server/helpers/custom-validators` | 36 | - Create your validators that will be used by your middlewares in `server/server/helpers/custom-validators` |
37 | * Typescript models: | 37 | * Typescript models: |
38 | - Create the API models (request parameters or response) in `shared/models` | 38 | - Create the API models (request parameters or response) in `packages/models` |
39 | - Add your models in `index.ts` of current directory to facilitate the imports | 39 | - Add your models in `index.ts` of current directory to facilitate the imports |
40 | * Sequelize model (BDD): | 40 | * Sequelize model (BDD): |
41 | - If you need to create a new table: | 41 | - If you need to create a new table: |
42 | + Create the Sequelize model in `server/models/`: | 42 | + Create the Sequelize model in `server/server/models/`: |
43 | * Create the `@Column` | 43 | * Create the `@Column` |
44 | * Add some indexes if you need | 44 | * Add some indexes if you need |
45 | * Create static methods to load a specific from the database `loadBy...` | 45 | * Create static methods to load a specific from the database `loadBy...` |
46 | * Create static methods to load a list of models from the database `listBy...` | 46 | * Create static methods to load a list of models from the database `listBy...` |
47 | * Create the instance method `toFormattedJSON` that creates the JSON to send to the REST API from the model | 47 | * Create the instance method `toFormattedJSON` that creates the JSON to send to the REST API from the model |
48 | + Add your new Sequelize model to `server/initializers/database.ts` | 48 | + Add your new Sequelize model to `server/server/initializers/database.ts` |
49 | + Create a new file in `server/types` to define multiple versions of your Sequelize model depending on database associations | 49 | + Create a new file in `server/server/types` to define multiple versions of your Sequelize model depending on database associations |
50 | + Add this new file to `server/types/*/index.ts` to facilitate the imports | 50 | + Add this new file to `server/server/types/*/index.ts` to facilitate the imports |
51 | + Create database migrations: | 51 | + Create database migrations: |
52 | * Create the migration file in `server/initializers/migrations` using raw SQL (copy the same SQL query as at PeerTube startup) | 52 | * Create the migration file in `server/server/initializers/migrations` using raw SQL (copy the same SQL query as at PeerTube startup) |
53 | * Update `LAST_MIGRATION_VERSION` in `server/initializers/constants.ts` | 53 | * Update `LAST_MIGRATION_VERSION` in `server/server/initializers/constants.ts` |
54 | - If updating database schema (adding/removing/renaming a column): | 54 | - If updating database schema (adding/removing/renaming a column): |
55 | + Update the sequelize models in `server/models/` | 55 | + Update the sequelize models in `server/server/models/` |
56 | + Add migrations: | 56 | + Add migrations: |
57 | * Create the migration file in `initializers/migrations` using Sequelize Query Interface (`.addColumn`, `.dropTable`, `.changeColumn`) | 57 | * Create the migration file in `initializers/migrations` using Sequelize Query Interface (`.addColumn`, `.dropTable`, `.changeColumn`) |
58 | * Update `LAST_MIGRATION_VERSION` in `server/initializers/constants.ts` | 58 | * Update `LAST_MIGRATION_VERSION` in `server/server/initializers/constants.ts` |
59 | * Notifications: | 59 | * Notifications: |
60 | - Create the new notification model in `shared/models/users/user-notification.model.ts` | 60 | - Create the new notification model in `packages/models/users/user-notification.model.ts` |
61 | - Create the notification logic in `server/lib/notifier/shared`: | 61 | - Create the notification logic in `server/server/lib/notifier/shared`: |
62 | + Email subject has a common prefix (defined by the admin in PeerTube configuration) | 62 | + Email subject has a common prefix (defined by the admin in PeerTube configuration) |
63 | - Add your notification to `server/lib/notifier/notifier.ts` | 63 | - Add your notification to `server/server/lib/notifier/notifier.ts` |
64 | - Create the email template in `server/lib/emails`: | 64 | - Create the email template in `server/server/lib/emails`: |
65 | + A text version is automatically generated from the HTML | 65 | + A text version is automatically generated from the HTML |
66 | + The template usually extends `../common/grettings` that already says "Hi" and "Cheers". You just have to write the title and the content blocks that will be inserted in the appropriate places in the HTML template | 66 | + The template usually extends `../common/grettings` that already says "Hi" and "Cheers". You just have to write the title and the content blocks that will be inserted in the appropriate places in the HTML template |
67 | - If you need to associate a new table with `userNotification`: | 67 | - If you need to associate a new table with `userNotification`: |
68 | + Associate the new table in `UserNotificationModel` (don't forget the index) | 68 | + Associate the new table in `UserNotificationModel` (don't forget the index) |
69 | + Add the object property in the API model definition (`shared/models/users/user-notification.model.ts`) | 69 | + Add the object property in the API model definition (`packages/models/users/user-notification.model.ts`) |
70 | + Add the object in `UserNotificationModel.toFormattedJSON` | 70 | + Add the object in `UserNotificationModel.toFormattedJSON` |
71 | + Handle this new notification type in client (`UserNotificationsComponent`) | 71 | + Handle this new notification type in client (`UserNotificationsComponent`) |
72 | + Handle the new object property in client model (`UserNotification`) | 72 | + Handle the new object property in client model (`UserNotification`) |
73 | * Tests: | 73 | * Tests: |
74 | - Create your command class in `shared/server-commands/` that will wrap HTTP requests to your new endpoint | 74 | - Create your command class in `packages/server-commands/` that will wrap HTTP requests to your new endpoint |
75 | - Add your command file in `index.ts` of current directory | 75 | - Add your command file in `index.ts` of current directory |
76 | - Instantiate your command class in `shared/server-commands/server/server.ts` | 76 | - Instantiate your command class in `packages/server-commands/server/server/server.ts` |
77 | - Create your test file in `server/tests/api/check-params` to test middleware validators/authentification/user rights (offensive tests) | 77 | - Create your test file in `server/server/tests/api/check-params` to test middleware validators/authentification/user rights (offensive tests) |
78 | - Add it to `server/tests/api/check-params/index.ts` | 78 | - Add it to `server/server/tests/api/check-params/index.ts` |
79 | - Create your test file in `server/tests/api` to test your new endpoints | 79 | - Create your test file in `server/server/tests/api` to test your new endpoints |
80 | - Add it to `index.ts` of current directory | 80 | - Add it to `index.ts` of current directory |
81 | - Add your notification test in `server/tests/api/notifications` | 81 | - Add your notification test in `server/server/tests/api/notifications` |
82 | * Update REST API documentation in `support/doc/api/openapi.yaml` | 82 | * Update REST API documentation in `support/doc/api/openapi.yaml` |