support/doc/tools.md

   1 # CLI tools guide
   2
   3 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
   4 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
   5 **Table of Contents**
   6
   7 - [Remote Tools](#remote-tools)
   8   - [Dependencies](#dependencies)
   9   - [Installation](#installation)
  10   - [CLI wrapper](#cli-wrapper)
  11     - [peertube-import-videos.js](#peertube-import-videosjs)
  12     - [peertube-upload.js](#peertube-uploadjs)
  13     - [peertube-watch.js](#peertube-watchjs)
  14     - [peertube-plugins.js](#peertube-pluginsjs)
  15     - [peertube-redundancy.js](#peertube-redundancyjs)
  16 - [Server tools](#server-tools)
  17   - [parse-log](#parse-log)
  18   - [regenerate-thumbnails.js](#regenerate-thumbnailsjs)
  19   - [create-transcoding-job.js](#create-transcoding-jobjs)
  20   - [create-import-video-file-job.js](#create-import-video-file-jobjs)
  21   - [prune-storage.js](#prune-storagejs)
  22   - [update-host.js](#update-hostjs)
  23   - [reset-password.js](#reset-passwordjs)
  24   - [plugin install/uninstall](#plugin-installuninstall)
  25   - [REPL (Read Eval Print Loop)](#repl-read-eval-print-loop)
  26     - [.help](#help)
  27     - [Lodash example](#lodash-example)
  28     - [YoutubeDL example](#youtubedl-example)
  29     - [Models examples](#models-examples)
  30
  31 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
  32
  33 ## Remote Tools
  34
  35 You need at least 512MB RAM to run the script.
  36 Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
  37 You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).
  38
  39 ### Dependencies
  40
  41 Install the [PeerTube dependencies](dependencies.md) except PostgreSQL and Redis.
  42
  43 ### Installation
  44
  45 Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):
  46
  47 ```bash
  48 $ git clone https://github.com/Chocobozzz/PeerTube.git
  49 $ CLONE="$(pwd)/PeerTube"
  50 $ cd ${CLONE}
  51 ```
  52
  53 Install dependencies and build CLI tools:
  54
  55 ```bash
  56 $ NOCLIENT=1 yarn install --pure-lockfile
  57 $ npm run setup:cli
  58 ```
  59
  60 ### CLI wrapper
  61
  62 The wrapper provides a convenient interface to the following scripts.
  63 You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):
  64
  65 ```
  66   Usage: peertube [command] [options]
  67
  68   Options:
  69
  70     -v, --version         output the version number
  71     -h, --help            output usage information
  72
  73   Commands:
  74
  75     auth [action]         register your accounts on remote instances to use them with other commands
  76     upload|up             upload a video
  77     import-videos|import  import a video from a streaming platform
  78     watch|w               watch a video in the terminal ✩°｡⋆
  79     repl                  initiate a REPL to access internals
  80     plugins|p [action]    manage instance plugins
  81     redundancy|r [action] manage video redundancies
  82     help [cmd]            display help for [cmd]
  83 ```
  84
  85 The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.
  86
  87 ```bash
  88 $ peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
  89 $ peertube auth list
  90 ┌──────────────────────────────┬──────────────────────────────┐
  91 │ instance                     │ login                        │
  92 ├──────────────────────────────┼──────────────────────────────┤
  93 │ 'PEERTUBE_URL'               │ 'PEERTUBE_USER'              │
  94 └──────────────────────────────┴──────────────────────────────┘
  95 ```
  96
  97 You can now use that account to upload videos without feeding the same parameters again.
  98
  99 ```bash
 100 $ peertube up <videoFile>
 101 ```
 102
 103 And now that your video is online, you can watch it from the confort of your terminal (use `peertube watch --help` to see the supported players):
 104
 105 ```bash
 106 $ peertube watch https://peertube.cpy.re/videos/watch/e8a1af4e-414a-4d58-bfe6-2146eed06d10
 107 ```
 108
 109 To list, install, uninstall dynamically plugins/themes of an instance:
 110
 111 ```bash
 112 $ peertube plugins list
 113 $ peertube plugins install --path /local/plugin/path
 114 $ peertube plugins install --npm-name peertube-plugin-myplugin
 115 $ peertube plugins uninstall --npm-name peertube-plugin-myplugin
 116 ```
 117
 118 #### peertube-import-videos.js
 119
 120 You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
 121 Be sure you own the videos or have the author's authorization to do so.
 122
 123 ```sh
 124 $ node dist/server/tools/peertube-import-videos.js \
 125     -u 'PEERTUBE_URL' \
 126     -U 'PEERTUBE_USER' \
 127     --password 'PEERTUBE_PASSWORD' \
 128     --target-url 'TARGET_URL'
 129 ```
 130
 131 * `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
 132 * `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
 133 * `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
 134 * `TARGET_URL` : the target url you want to import. Examples:
 135   * YouTube:
 136     * Channel: https://www.youtube.com/channel/ChannelId
 137     * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
 138     * Video https://www.youtube.com/watch?v=blabla
 139   * Vimeo: https://vimeo.com/xxxxxx
 140   * Dailymotion: https://www.dailymotion.com/xxxxx
 141
 142 The script will get all public videos from Youtube, download them and upload to PeerTube.
 143 Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...
 144
 145 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 146
 147 **NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
 148 you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
 149 these rules (ensure to customize them to your needs):
 150
 151 ```
 152 # Update youtube-dl every day at midnight
 153 0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/
 154
 155 # Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
 156 0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +%Y-%m-%d)
 157 ```
 158
 159 Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.
 160
 161 #### peertube-upload.js
 162
 163 You can use this script to import videos directly from the CLI.
 164
 165 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 166
 167 ```bash
 168 $ cd ${CLONE}
 169 $ node dist/server/tools/peertube-upload.js --help
 170 ```
 171
 172 #### peertube-watch.js
 173
 174 You can use this script to play videos directly from the CLI.
 175
 176 It provides support for different players:
 177
 178 - ascii (default ; plays in ascii art in your terminal!)
 179 - mpv
 180 - mplayer
 181 - vlc
 182 - stdout
 183 - xbmc
 184 - airplay
 185 - chromecast
 186
 187
 188 #### peertube-plugins.js
 189
 190 Install/update/uninstall or list local or NPM PeerTube plugins:
 191
 192 ```bash
 193 $ cd ${CLONE}
 194 $ node dist/server/tools/peertube-plugins.js --help
 195 $ node dist/server/tools/peertube-plugins.js list --help
 196 $ node dist/server/tools/peertube-plugins.js install --help
 197 $ node dist/server/tools/peertube-plugins.js update --help
 198 $ node dist/server/tools/peertube-plugins.js uninstall --help
 199
 200 $ node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
 201 $ node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
 202 ```
 203
 204 #### peertube-redundancy.js
 205
 206 Manage (list/add/remove) video redundancies:
 207
 208 To list your videos that are duplicated by remote instances:
 209
 210 ```bash
 211 $ node dist/server/tools/peertube.js redundancy list-remote-redundancies
 212 ```
 213
 214 To list remote videos that your instance duplicated:
 215
 216 ```bash
 217 $ node dist/server/tools/peertube.js redundancy list-my-redundancies
 218 ```
 219
 220 To duplicate a specific video in your redundancy system:
 221
 222 ```bash
 223 $ node dist/server/tools/peertube.js redundancy add --video 823
 224 ```
 225
 226 To remove a video redundancy:
 227
 228 ```bash
 229 $ node dist/server/tools/peertube.js redundancy remove --video 823
 230 ```
 231
 232 ## Server tools
 233
 234 These scripts should be run on the server, in `peertube-latest` directory.
 235
 236 ### parse-log
 237
 238 To parse PeerTube last log file:
 239
 240 ```bash
 241 $ # Basic installation
 242 $ cd /var/www/peertube/peertube-latest
 243 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level info
 244
 245 $ # Docker installation
 246 $ cd /var/www/peertube-docker
 247 $ docker-compose exec -u peertube peertube npm run parse-log -- --level info
 248 ```
 249
 250 `--level` is optional and could be `info`/`warn`/`error`
 251
 252 You can also remove SQL or HTTP logs using `--not-tags` (PeerTube >= 3.2):
 253
 254 ```bash
 255 $ # Basic installation
 256 $ cd /var/www/peertube/peertube-latest
 257 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level debug --not-tags http sql
 258
 259 $ # Docker installation
 260 $ cd /var/www/peertube-docker
 261 $ docker-compose exec -u peertube peertube npm run parse-log -- --level debug --not-tags http sql
 262 ```
 263
 264 ### regenerate-thumbnails.js
 265
 266 **PeerTube >= 3.2**
 267
 268 Regenerating local video thumbnails could be useful because new PeerTube releases may increase thumbnail sizes:
 269
 270 ```bash
 271 $ # Basic installation
 272 $ cd /var/www/peertube/peertube-latest
 273 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run regenerate-thumbnails
 274
 275 $ # Docker installation
 276 $ cd /var/www/peertube-docker
 277 $ docker-compose exec -u peertube peertube npm run regenerate-thumbnails
 278 ```
 279
 280 ### create-transcoding-job.js
 281
 282 You can use this script to force transcoding of an existing video. PeerTube needs to be running.
 283
 284 To generate transcoding jobs depending on the instance configuration:
 285
 286 ```bash
 287 $ # Basic installation
 288 $ cd /var/www/peertube/peertube-latest
 289 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID]
 290
 291 $ # Docker installation
 292 $ cd /var/www/peertube-docker
 293 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID]
 294 ```
 295
 296 Or to transcode to a specific resolution:
 297
 298 ```bash
 299 $ # Basic installation
 300 $ cd /var/www/peertube/peertube-latest
 301 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 302
 303 $ # Docker installation
 304 $ cd /var/www/peertube-docker
 305 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 306 ```
 307
 308 The resolution should be an integer (`1080`, `720`, `480`, etc.)
 309
 310 To generate an HLS playlist for a video:
 311
 312 ```bash
 313 $ # Basic installation
 314 $ cd /var/www/peertube/peertube-latest
 315 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- --generate-hls -v [videoUUID]
 316
 317 $ # Docker installation
 318 $ cd /var/www/peertube-docker
 319 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- --generate-hls -v [videoUUID]
 320 ```
 321
 322 ### create-import-video-file-job.js
 323
 324 You can use this script to import a video file to replace an already uploaded file or to add a new resolution to a video. PeerTube needs to be running.
 325
 326 ```bash
 327 $ # Basic installation
 328 $ cd /var/www/peertube/peertube-latest
 329 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 330
 331 $ # Docker installation
 332 $ cd /var/www/peertube-docker
 333 $ docker-compose exec -u peertube peertube npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 334 ```
 335
 336 ### prune-storage.js
 337
 338 Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
 339 Stop PeerTube and delete these files (a confirmation will be demanded first):
 340
 341 ```bash
 342 $ cd /var/www/peertube/peertube-latest
 343 $ sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
 344 ```
 345
 346
 347 ### update-host.js
 348
 349 **Changing the hostname is unsupported and may be a risky operation, especially if you have already federated.**
 350 If you started PeerTube with a domain, and then changed it you will have
 351 invalid torrent files and invalid URLs in your database. To fix this, you have
 352 to run the command below (keep in mind your follower instances will NOT update their URLs).
 353
 354 ```bash
 355 $ # Basic installation
 356 $ cd /var/www/peertube/peertube-latest
 357 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run update-host
 358
 359 $ # Docker installation
 360 $ cd /var/www/peertube-docker
 361 $ docker-compose exec -u peertube peertube npm run update-host
 362 ```
 363
 364 ### reset-password.js
 365
 366 To reset a user password from CLI, run:
 367
 368 ```bash
 369 $ # Basic installation
 370 $ cd /var/www/peertube/peertube-latest
 371 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run reset-password -- -u target_username
 372
 373 $ # Docker installation
 374 $ cd /var/www/peertube-docker
 375 $ docker-compose exec -u peertube peertube npm run reset-password -- -u target_username
 376 ```
 377
 378
 379 ### plugin install/uninstall
 380
 381 The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
 382 If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).
 383
 384 To install/update a plugin or a theme from the disk:
 385
 386 ```bash
 387 $ cd /var/www/peertube/peertube-latest
 388 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --plugin-path /local/plugin/path
 389
 390 $ # Docker installation
 391 $ cd /var/www/peertube-docker
 392 $ docker-compose exec -u peertube peertube npm run plugin:install -- --plugin-path /local/plugin/path
 393 ```
 394
 395 From NPM:
 396
 397 ```bash
 398 $ cd /var/www/peertube/peertube-latest
 399 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --npm-name peertube-plugin-myplugin
 400
 401 $ # Docker installation
 402 $ cd /var/www/peertube-docker
 403 $ docker-compose exec -u peertube peertube npm run plugin:install -- --npm-name peertube-plugin-myplugin
 404 ```
 405
 406 To uninstall a plugin or a theme:
 407
 408 ```bash
 409 $ cd /var/www/peertube/peertube-latest
 410 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 411
 412 $ # Docker installation
 413 $ cd /var/www/peertube-docker
 414 $ docker-compose exec -u peertube peertube npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 415 ```
 416
 417 ### REPL (Read Eval Print Loop)
 418
 419 If you want to interact with the application libraries and objects even when PeerTube is not running, there is a REPL for that.
 420
 421 usage: `node ./dist/server/tools/peertube-repl.js`
 422
 423 "The default evaluator will, by default, assign the result of the most recently evaluated expression to the special variable `_` (underscore). Explicitly setting `_` to a value will disable this behavior."
 424
 425 - type `.help` to list commands available in the repl, notice it starts with a dot
 426 - type `.exit` to exit, note that you still have to press CTRL-C to actually exit, or press CTRL-C (3 times) without typing `.exit` to exit
 427 - type `context` to list all available objects and libraries in the context, note: `Promise` is also available but it's not listed in the context, in case you need promises for something
 428 - type `env` to see the loaded environment variables
 429 - type `path` to access path library
 430 - type `lodash` to access lodash library
 431 - type `uuidv1` to access uuid/v1 library
 432 - type `uuidv3` to access uuid/v3 library
 433 - type `uuidv4` to access uuid/v4 library
 434 - type `uuidv5` to access uuid/v5 library
 435 - type `YoutubeDL` to access youtube-dl library
 436 - type `cli` to access the cli helpers object
 437 - type `logger` to access the logger; if you log to it, it will write to stdout and to the peertube.log file
 438 - type `constants` to access the constants loaded by the server
 439 - type `coreUtils` to access the core-utils helpers object
 440 - type `ffmpegUtils` to access the ffmpeg-utils helpers object
 441 - type `peertubeCryptoUtils` to access the peertube-crypto helpers object
 442 - type `signupUtils` to access the signup helpers object
 443 - type `utils` to access the utils helpers object
 444 - type `YoutubeDLUtils` to access the youtube-dl helpers object
 445 - type `sequelizeTypescript` to access sequelizeTypescript
 446 - type `modelsUtils` to access the models/utils
 447 - type `models` to access the shortcut to sequelizeTypescript.models
 448 - type `transaction` to access the shortcut to sequelizeTypescript.transaction
 449 - type `query` to access the shortcut to sequelizeTypescript.query
 450 - type `queryInterface` to access the shortcut to sequelizeTypescript.queryInterface
 451
 452 #### .help
 453
 454 ```
 455 PeerTube [1.0.0] (b10eb595)> .help
 456 .break    Sometimes you get stuck, this gets you out
 457 .clear    Break, and also clear the local context
 458 .editor   Enter editor mode
 459 .exit     Exit the repl
 460 .help     Print this help message
 461 .load     Load JS from a file into the REPL session
 462 .r        Reset REPL
 463 .reset    Reset REPL
 464 .save     Save all evaluated commands in this REPL session to a file
 465 PeerTube [1.0.0] (b10eb595)>
 466 ```
 467
 468 #### Lodash example
 469
 470 ```
 471 PeerTube [1.0.0] (b10eb595)> lodash.keys(context)
 472 [ 'global',
 473   'console',
 474   'DTRACE_NET_SERVER_CONNECTION',
 475   'DTRACE_NET_STREAM_END',
 476   'DTRACE_HTTP_SERVER_REQUEST',
 477   'DTRACE_HTTP_SERVER_RESPONSE',
 478   'DTRACE_HTTP_CLIENT_REQUEST',
 479   'DTRACE_HTTP_CLIENT_RESPONSE',
 480   'process',
 481   'Buffer',
 482   'clearImmediate',
 483   'clearInterval',
 484   'clearTimeout',
 485   'setImmediate',
 486   'setInterval',
 487   'setTimeout',
 488   'XMLHttpRequest',
 489   'compact2string',
 490   'module',
 491   'require',
 492   'path',
 493   'repl',
 494   'context',
 495   'env',
 496   'lodash',
 497   'uuidv1',
 498   'uuidv3',
 499   'uuidv4',
 500   'uuidv5',
 501   'cli',
 502   'logger',
 503   'constants',
 504   'Sequelize',
 505   'sequelizeTypescript',
 506   'modelsUtils',
 507   'models',
 508   'transaction',
 509   'query',
 510   'queryInterface',
 511   'YoutubeDL',
 512   'coreUtils',
 513   'ffmpegUtils',
 514   'peertubeCryptoUtils',
 515   'signupUtils',
 516   'utils',
 517   'YoutubeDLUtils' ]
 518 PeerTube [1.0.0] (b10eb595)>
 519 ```
 520
 521 #### YoutubeDL example
 522 ```
 523 YoutubeDL.getInfo('https://www.youtube.com/watch?v=I5ZN289jjDo', function(err, data) {console.log(err, data)})
 524 ```
 525
 526 #### Models examples
 527 ```
 528 PeerTube [1.0.0] (b10eb595)> new models.ActorModel({id: 3}).getVideoChannel().then(function(data){console.log(data.dataValues.name)})
 529 Promise {
 530   _bitField: 0,
 531   _fulfillmentHandler0: undefined,
 532   _rejectionHandler0: undefined,
 533   _promise0: undefined,
 534   _receiver0: undefined }
 535 PeerTube [1.0.0] (b10eb595)> Main root channel
 536 PeerTube [1.0.0] (b10eb595)> let out; new models.UserModel({id: 1}).getAccount().then(function (data) {out = data.dataValues.id})
 537 Promise {
 538   _bitField: 0,
 539   _fulfillmentHandler0: undefined,
 540   _rejectionHandler0: undefined,
 541   _promise0: undefined,
 542   _receiver0: undefined }
 543 PeerTube [1.0.0] (b10eb595)> out
 544 2
 545 PeerTube [1.0.0] (b10eb595)>
 546 ```