Migrate server to ESM

Sorry for the very big commit that may lead to git log issues and merge
conflicts, but it's a major step forward:

 * Server can be faster at startup because imports() are async and we can
   easily lazy import big modules
 * Angular doesn't seem to support ES import (with .js extension), so we
   had to correctly organize peertube into a monorepo:
    * Use yarn workspace feature
    * Use typescript reference projects for dependencies
    * Shared projects have been moved into "packages", each one is now a
      node module (with a dedicated package.json/tsconfig.json)
    * server/tools have been moved into apps/ and is now a dedicated app
      bundled and published on NPM so users don't have to build peertube
      cli tools manually
    * server/tests have been moved into packages/ so we don't compile
      them every time we want to run the server
 * Use isolatedModule option:
   * Had to move from const enum to const
     (https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums)
   * Had to explictely specify "type" imports when used in decorators
 * Prefer tsx (that uses esbuild under the hood) instead of ts-node to
   load typescript files (tests with mocha or scripts):
     * To reduce test complexity as esbuild doesn't support decorator
       metadata, we only test server files that do not import server
       models
     * We still build tests files into js files for a faster CI
 * Remove unmaintained peertube CLI import script
 * Removed some barrels to speed up execution (less imports)

1 files changed, 111 insertions, 175 deletions

diff --git a/support/doc/tools.md b/support/doc/tools.md
index 2b3ebf159..40d9ec66a 100644
--- a/support/doc/tools.md
+++ b/support/doc/tools.md
@@ -4,59 +4,48 @@
 
 ## Remote PeerTube CLI
 
-You need at least 512MB RAM to run the script.
-Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
-You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).
-
-### Dependencies
-
-Install the [PeerTube dependencies](/support/doc/dependencies.md) except PostgreSQL and Redis.
+`peertube-cli` is a tool that communicates with a PeerTube instance using its [REST API](https://docs.joinpeertube.org/api-rest-reference.html).
+It can be launched from a remote server/computer to easily upload videos, manage plugins, redundancies etc.
 
 ### Installation
 
-Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):
+Ensure you have `node` installed on your system:
 
 ```bash
-git clone https://github.com/Chocobozzz/PeerTube.git
-CLONE="$(pwd)/PeerTube"
-cd ${CLONE}
+node --version # Should be >= 16.x
 ```
 
-Install dependencies and build CLI tools:
+Then install the CLI:
 
 ```bash
-NOCLIENT=1 yarn install --pure-lockfile
-npm run setup:cli
+sudo npm install -g @peertube/peertube-cli
 ```
 
 ### CLI wrapper
 
-The wrapper provides a convenient interface to the following scripts.
-You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):
+The wrapper provides a convenient interface to the following sub-commands.
 
 ```
-  Usage: peertube [command] [options]
-
-  Options:
+Usage: peertube-cli [command] [options]
 
-    -v, --version         output the version number
-    -h, --help            output usage information
+Options:
+  -v, --version                     output the version number
+  -h, --help                        display help for command
 
-  Commands:
-
-    auth [action]         register your accounts on remote instances to use them with other commands
-    upload|up             upload a video
-    import-videos|import  import a video from a streaming platform
-    plugins|p [action]    manage instance plugins
-    redundancy|r [action] manage video redundancies
-    help [cmd]            display help for [cmd]
+Commands:
+  auth                              Register your accounts on remote instances to use them with other commands
+  upload|up [options]               Upload a video on a PeerTube instance
+  redundancy|r                      Manage instance redundancies
+  plugins|p                         Manage instance plugins/themes
+  get-access-token|token [options]  Get a peertube access token
+  help [command]                    display help for command
 ```
 
 The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.
 
 ```bash
-peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
-peertube auth list
+peertube-cli auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
+peertube-cli auth list
 ┌──────────────────────────────┬──────────────────────────────┐
 │ instance                     │ login                        │
 ├──────────────────────────────┼──────────────────────────────┤
@@ -64,124 +53,142 @@ peertube auth list
 └──────────────────────────────┴──────────────────────────────┘
 ```
 
-You can now use that account to upload videos without feeding the same parameters again.
+You can now use that account to execute sub-commands without feeding the `--url`, `--username` and `--password` parameters:
 
 ```bash
-peertube up <videoFile>
+peertube-cli upload <videoFile>
+peertube-cli plugins list
+...
 ```
 
-To list, install, uninstall dynamically plugins/themes of an instance:
+#### peertube-cli upload
+
+You can use this script to upload videos directly from the CLI.
+
+Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 
 ```bash
-peertube plugins list
-peertube plugins install --path /local/plugin/path
-peertube plugins install --npm-name peertube-plugin-myplugin
-peertube plugins uninstall --npm-name peertube-plugin-myplugin
+cd ${CLONE}
+peertube-cli upload --help
 ```
 
-#### peertube-import-videos.js
+#### peertube-cli plugins
+
+Install/update/uninstall or list local or NPM PeerTube plugins:
 
-You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
-Be sure you own the videos or have the author's authorization to do so.
+```bash
+cd ${CLONE}
+peertube-cli plugins --help
+peertube-cli plugins list --help
+peertube-cli plugins install --help
+peertube-cli plugins update --help
+peertube-cli plugins uninstall --help
 
-```sh
-node dist/server/tools/peertube-import-videos.js \
-    -u 'PEERTUBE_URL' \
-    -U 'PEERTUBE_USER' \
-    --password 'PEERTUBE_PASSWORD' \
-    --target-url 'TARGET_URL'
+peertube-cli plugins install --path /my/plugin/path
+peertube-cli plugins install --npm-name peertube-theme-example
 ```
 
-* `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
-* `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
-* `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
-* `TARGET_URL` : the target url you want to import. Examples:
-  * YouTube:
-    * Channel: https://www.youtube.com/channel/ChannelId
-    * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
-    * Video https://www.youtube.com/watch?v=blabla
-  * Vimeo: https://vimeo.com/xxxxxx
-  * Dailymotion: https://www.dailymotion.com/xxxxx
+#### peertube-cli redundancy
 
-The script will get all public videos from Youtube, download them and upload to PeerTube.
-Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...
+Manage (list/add/remove) video redundancies:
 
-Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
+To list your videos that are duplicated by remote instances:
+
+```bash
+peertube-cli redundancy list-remote-redundancies
+```
 
-**NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
-you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
-these rules (ensure to customize them to your needs):
+To list remote videos that your instance duplicated:
 
+```bash
+peertube-cli redundancy list-my-redundancies
+```
+
+To duplicate a specific video in your redundancy system:
+
+```bash
+peertube-cli redundancy add --video 823
 ```
-# Update youtube-dl every day at midnight
-0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/
 
-# Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
-0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +\%Y-\%m-\%d)
+To remove a video redundancy:
+
+```bash
+peertube-cli redundancy remove --video 823
 ```
 
-Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.
 
-#### peertube-upload.js
+## PeerTube runner
 
-You can use this script to import videos directly from the CLI.
+PeerTube >= 5.2 supports VOD or Live transcoding by a remote PeerTube runner.
 
-Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
+### Installation
+
+Ensure you have `node`, `ffmpeg` and `ffprobe` installed on your system:
 
 ```bash
-cd ${CLONE}
-node dist/server/tools/peertube-upload.js --help
+node --version # Should be >= 16.x
+ffprobe -version # Should be >= 4.3
+ffmpeg -version # Should be >= 4.3
 ```
 
-#### peertube-plugins.js
-
-Install/update/uninstall or list local or NPM PeerTube plugins:
+Then install the CLI:
 
 ```bash
-cd ${CLONE}
-node dist/server/tools/peertube-plugins.js --help
-node dist/server/tools/peertube-plugins.js list --help
-node dist/server/tools/peertube-plugins.js install --help
-node dist/server/tools/peertube-plugins.js update --help
-node dist/server/tools/peertube-plugins.js uninstall --help
+sudo npm install -g @peertube/peertube-runner
+```
+
+### Configuration
+
+The runner uses env paths like `~/.config`, `~/.cache` and `~/.local/share` directories to store runner configuration or temporary files.
+
+Multiple PeerTube runners can run on the same OS by using the `--id` CLI option (each runner uses its own config/tmp directories):
 
-node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
-node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
+```bash
+peertube-runner [commands] --id instance-1
+peertube-runner [commands] --id instance-2
+peertube-runner [commands] --id instance-3
 ```
 
-#### peertube-redundancy.js
+You can change the runner configuration (jobs concurrency, ffmpeg threads/nice etc) by editing `~/.config/peertube-runner-nodejs/[id]/config.toml`.
 
-Manage (list/add/remove) video redundancies:
+### Run the server
 
-To list your videos that are duplicated by remote instances:
+You need to run the runner in server mode first so it can run transcoding jobs of registered PeerTube instances:
 
 ```bash
-node dist/server/tools/peertube.js redundancy list-remote-redundancies
+peertube-runner server
 ```
 
-To list remote videos that your instance duplicated:
+### Register
+
+Then, you can register the runner to process transcoding job of a remote PeerTube instance:
 
 ```bash
-node dist/server/tools/peertube.js redundancy list-my-redundancies
+peertube-runner register --url http://peertube.example.com --registration-token ptrrt-... --runner-name my-runner-name
 ```
 
-To duplicate a specific video in your redundancy system:
+The runner will then use a websocket connection with the PeerTube instance to be notified about new available transcoding jobs.
+
+### Unregister
+
+To unregister a PeerTube instance:
 
 ```bash
-node dist/server/tools/peertube.js redundancy add --video 823
+peertube-runner unregister --url http://peertube.example.com --runner-name my-runner-name
 ```
 
-To remove a video redundancy:
+### List registered instances
 
 ```bash
-node dist/server/tools/peertube.js redundancy remove --video 823
+peertube-runner list-registered
 ```
 
 ## Server tools
 
-These scripts should be run on the server, in `peertube-latest` directory.
+Server tools are scripts that interect directly with the code of your PeerTube instance.
+They must be run on the server, in `peertube-latest` directory.
 
-### parse-log
+### Parse logs
 
 To parse PeerTube last log file:
 
@@ -209,9 +216,7 @@ cd /var/www/peertube-docker
 docker-compose exec -u peertube peertube npm run parse-log -- --level debug --not-tags http sql
 ```
 
-### regenerate-thumbnails.js
-
-**PeerTube >= 3.2**
+### Regenerate video thumbnails
 
 Regenerating local video thumbnails could be useful because new PeerTube releases may increase thumbnail sizes:
 
@@ -225,7 +230,7 @@ cd /var/www/peertube-docker
 docker-compose exec -u peertube peertube npm run regenerate-thumbnails
 ```
 
-### create-import-video-file-job.js
+### Add or replace specific video file
 
 You can use this script to import a video file to replace an already uploaded file or to add a new web compatible resolution to a video. PeerTube needs to be running.
 You can then create a transcoding job using the web interface if you need to optimize your file or create an HLS version of it.
@@ -240,9 +245,7 @@ cd /var/www/peertube-docker
 docker-compose exec -u peertube peertube npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 ```
 
-### create-move-video-storage-job.js
-
-**PeerTube >= 4.0**
+### Move video files to object storage
 
 Use this script to move all video files or a specific video file to object storage.
 
@@ -269,7 +272,7 @@ docker-compose exec -u peertube peertube npm run create-move-video-storage-job -
 ```
 
 <!-- TODO: uncomment when PeerTube 6 is released
-### create-generate-storyboard-job
+### Generate storyboard
 
 **PeerTube >= 6.0**
 
@@ -298,7 +301,7 @@ docker-compose exec -u peertube peertube npm run create-generate-storyboard-job
 ```
 -->
 
-### prune-storage.js
+### Prune filesystem storage
 
 Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
 Stop PeerTube and delete these files (a confirmation will be demanded first):
@@ -308,8 +311,7 @@ cd /var/www/peertube/peertube-latest
 sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
 ```
 
-
-### update-host.js
+### Update PeerTube instance domain name
 
 **Changing the hostname is unsupported and may be a risky operation, especially if you have already federated.**
 If you started PeerTube with a domain, and then changed it you will have
@@ -326,7 +328,7 @@ cd /var/www/peertube-docker
 docker-compose exec -u peertube peertube npm run update-host
 ```
 
-### reset-password.js
+### Reset user password
 
 To reset a user password from CLI, run:
 
@@ -341,7 +343,7 @@ docker-compose exec -u peertube peertube npm run reset-password -- -u target_use
 ```
 
 
-### plugin install/uninstall
+### Install or uninstall plugins
 
 The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
 If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).
@@ -378,69 +380,3 @@ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production np
 cd /var/www/peertube-docker
 docker-compose exec -u peertube peertube npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 ```
-
-## PeerTube runner
-
-PeerTube >= 5.2 supports VOD or Live transcoding by a remote PeerTube runner.
-
-
-### Installation
-
-Ensure you have `ffmpeg` and `ffprobe` installed on your system:
-
-```bash
-ffprobe -version # Should be >= 4.3
-ffmpeg -version # Should be >= 4.3
-```
-
-Then install the CLI:
-
-```bash
-sudo npm install -g @peertube/peertube-runner
-```
-
-### Configuration
-
-The runner uses env paths like `~/.config`, `~/.cache` and `~/.local/share` directories to store runner configuration or temporary files.
-
-Multiple PeerTube runners can run on the same OS by using the `--id` CLI option (each runner uses its own config/tmp directories):
-
-```bash
-peertube-runner [commands] --id instance-1
-peertube-runner [commands] --id instance-2
-peertube-runner [commands] --id instance-3
-```
-
-You can change the runner configuration (jobs concurrency, ffmpeg threads/nice etc) by editing `~/.config/peertube-runner-nodejs/[id]/config.toml`.
-
-### Run the server
-
-You need to run the runner in server mode first so it can run transcoding jobs of registered PeerTube instances:
-
-```bash
-peertube-runner server
-```
-
-### Register
-
-Then, you can register the runner to process transcoding job of a remote PeerTube instance:
-
-```bash
-peertube-runner register --url http://peertube.example.com --registration-token ptrrt-... --runner-name my-runner-name
-```
-
-The runner will then use a websocket connection with the PeerTube instance to be notified about new available transcoding jobs.
-
-### Unregister
-
-To unregister a PeerTube instance:
-
-```bash
-peertube-runner unregister --url http://peertube.example.com --runner-name my-runner-name
-```
-
-### List registered instances
-
-```bash
-peertube-runner list-registered
-```