support/doc/tools.md

   1 # CLI tools guide
   2
   3 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
   4 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
   5 **Table of Contents**
   6
   7 - [Remote Tools](#remote-tools)
   8   - [Dependencies](#dependencies)
   9   - [Installation](#installation)
  10   - [CLI wrapper](#cli-wrapper)
  11     - [peertube-import-videos.js](#peertube-import-videosjs)
  12     - [peertube-upload.js](#peertube-uploadjs)
  13     - [peertube-watch.js](#peertube-watchjs)
  14     - [peertube-plugins.js](#peertube-pluginsjs)
  15     - [peertube-redundancy.js](#peertube-redundancyjs)
  16 - [Server tools](#server-tools)
  17   - [parse-log](#parse-log)
  18   - [regenerate-thumbnails.js](#regenerate-thumbnailsjs)
  19   - [create-transcoding-job.js](#create-transcoding-jobjs)
  20   - [create-import-video-file-job.js](#create-import-video-file-jobjs)
  21   - [prune-storage.js](#prune-storagejs)
  22   - [optimize-old-videos.js](#optimize-old-videosjs)
  23   - [update-host.js](#update-hostjs)
  24   - [reset-password.js](#reset-passwordjs)
  25   - [plugin install/uninstall](#plugin-installuninstall)
  26   - [REPL (Read Eval Print Loop)](#repl-read-eval-print-loop)
  27     - [.help](#help)
  28     - [Lodash example](#lodash-example)
  29     - [YoutubeDL example](#youtubedl-example)
  30     - [Models examples](#models-examples)
  31
  32 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
  33
  34 ## Remote Tools
  35
  36 You need at least 512MB RAM to run the script.
  37 Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
  38 You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).
  39
  40 ### Dependencies
  41
  42 Install the [PeerTube dependencies](dependencies.md) except PostgreSQL and Redis.
  43
  44 ### Installation
  45
  46 Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):
  47
  48 ```bash
  49 $ git clone https://github.com/Chocobozzz/PeerTube.git
  50 $ CLONE="$(pwd)/PeerTube"
  51 $ cd ${CLONE}
  52 ```
  53
  54 Install dependencies and build CLI tools:
  55
  56 ```bash
  57 $ NOCLIENT=1 yarn install --pure-lockfile
  58 $ npm run setup:cli
  59 ```
  60
  61 ### CLI wrapper
  62
  63 The wrapper provides a convenient interface to the following scripts.
  64 You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):
  65
  66 ```
  67   Usage: peertube [command] [options]
  68
  69   Options:
  70
  71     -v, --version         output the version number
  72     -h, --help            output usage information
  73
  74   Commands:
  75
  76     auth [action]         register your accounts on remote instances to use them with other commands
  77     upload|up             upload a video
  78     import-videos|import  import a video from a streaming platform
  79     watch|w               watch a video in the terminal ✩°｡⋆
  80     repl                  initiate a REPL to access internals
  81     plugins|p [action]    manage instance plugins
  82     redundancy|r [action] manage video redundancies
  83     help [cmd]            display help for [cmd]
  84 ```
  85
  86 The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.
  87
  88 ```bash
  89 $ peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
  90 $ peertube auth list
  91 ┌──────────────────────────────┬──────────────────────────────┐
  92 │ instance                     │ login                        │
  93 ├──────────────────────────────┼──────────────────────────────┤
  94 │ 'PEERTUBE_URL'               │ 'PEERTUBE_USER'              │
  95 └──────────────────────────────┴──────────────────────────────┘
  96 ```
  97
  98 You can now use that account to upload videos without feeding the same parameters again.
  99
 100 ```bash
 101 $ peertube up <videoFile>
 102 ```
 103
 104 And now that your video is online, you can watch it from the confort of your terminal (use `peertube watch --help` to see the supported players):
 105
 106 ```bash
 107 $ peertube watch https://peertube.cpy.re/videos/watch/e8a1af4e-414a-4d58-bfe6-2146eed06d10
 108 ```
 109
 110 To list, install, uninstall dynamically plugins/themes of an instance:
 111
 112 ```bash
 113 $ peertube plugins list
 114 $ peertube plugins install --path /local/plugin/path
 115 $ peertube plugins install --npm-name peertube-plugin-myplugin
 116 $ peertube plugins uninstall --npm-name peertube-plugin-myplugin
 117 ```
 118
 119 #### peertube-import-videos.js
 120
 121 You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
 122 Be sure you own the videos or have the author's authorization to do so.
 123
 124 ```sh
 125 $ node dist/server/tools/peertube-import-videos.js \
 126     -u 'PEERTUBE_URL' \
 127     -U 'PEERTUBE_USER' \
 128     --password 'PEERTUBE_PASSWORD' \
 129     --target-url 'TARGET_URL'
 130 ```
 131
 132 * `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
 133 * `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
 134 * `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
 135 * `TARGET_URL` : the target url you want to import. Examples:
 136   * YouTube:
 137     * Channel: https://www.youtube.com/channel/ChannelId
 138     * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
 139     * Video https://www.youtube.com/watch?v=blabla
 140   * Vimeo: https://vimeo.com/xxxxxx
 141   * Dailymotion: https://www.dailymotion.com/xxxxx
 142
 143 The script will get all public videos from Youtube, download them and upload to PeerTube.
 144 Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...
 145
 146 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 147
 148 **NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
 149 you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
 150 these rules (ensure to customize them to your needs):
 151
 152 ```
 153 # Update youtube-dl every day at midnight
 154 0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/
 155
 156 # Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
 157 0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +%Y-%m-%d)
 158 ```
 159
 160 Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.
 161
 162 #### peertube-upload.js
 163
 164 You can use this script to import videos directly from the CLI.
 165
 166 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 167
 168 ```bash
 169 $ cd ${CLONE}
 170 $ node dist/server/tools/peertube-upload.js --help
 171 ```
 172
 173 #### peertube-watch.js
 174
 175 You can use this script to play videos directly from the CLI.
 176
 177 It provides support for different players:
 178
 179 - ascii (default ; plays in ascii art in your terminal!)
 180 - mpv
 181 - mplayer
 182 - vlc
 183 - stdout
 184 - xbmc
 185 - airplay
 186 - chromecast
 187
 188
 189 #### peertube-plugins.js
 190
 191 Install/update/uninstall or list local or NPM PeerTube plugins:
 192
 193 ```bash
 194 $ cd ${CLONE}
 195 $ node dist/server/tools/peertube-plugins.js --help
 196 $ node dist/server/tools/peertube-plugins.js list --help
 197 $ node dist/server/tools/peertube-plugins.js install --help
 198 $ node dist/server/tools/peertube-plugins.js update --help
 199 $ node dist/server/tools/peertube-plugins.js uninstall --help
 200
 201 $ node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
 202 $ node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
 203 ```
 204
 205 #### peertube-redundancy.js
 206
 207 Manage (list/add/remove) video redundancies:
 208
 209 To list your videos that are duplicated by remote instances:
 210
 211 ```bash
 212 $ node dist/server/tools/peertube.js redundancy list-remote-redundancies
 213 ```
 214
 215 To list remote videos that your instance duplicated:
 216
 217 ```bash
 218 $ node dist/server/tools/peertube.js redundancy list-my-redundancies
 219 ```
 220
 221 To duplicate a specific video in your redundancy system:
 222
 223 ```bash
 224 $ node dist/server/tools/peertube.js redundancy add --video 823
 225 ```
 226
 227 To remove a video redundancy:
 228
 229 ```bash
 230 $ node dist/server/tools/peertube.js redundancy remove --video 823
 231 ```
 232
 233 ## Server tools
 234
 235 These scripts should be run on the server, in `peertube-latest` directory.
 236
 237 ### parse-log
 238
 239 To parse PeerTube last log file:
 240
 241 ```bash
 242 $ # Basic installation
 243 $ cd /var/www/peertube/peertube-latest
 244 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level info
 245
 246 $ # Docker installation
 247 $ cd /var/www/peertube-docker
 248 $ docker-compose exec -u peertube peertube npm run parse-log -- --level info
 249 ```
 250
 251 `--level` is optional and could be `info`/`warn`/`error`
 252
 253 You can also remove SQL or HTTP logs using `--not-tags` (PeerTube >= 3.2):
 254
 255 ```bash
 256 $ # Basic installation
 257 $ cd /var/www/peertube/peertube-latest
 258 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level debug --not-tags http sql
 259
 260 $ # Docker installation
 261 $ cd /var/www/peertube-docker
 262 $ docker-compose exec -u peertube peertube npm run parse-log -- --level debug --not-tags http sql
 263 ```
 264
 265 ### regenerate-thumbnails.js
 266
 267 **PeerTube >= 3.2**
 268
 269 Regenerating local video thumbnails could be useful because new PeerTube releases may increase thumbnail sizes:
 270
 271 ```bash
 272 $ # Basic installation
 273 $ cd /var/www/peertube/peertube-latest
 274 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run regenerate-thumbnails
 275
 276 $ # Docker installation
 277 $ cd /var/www/peertube-docker
 278 $ docker-compose exec -u peertube peertube npm run regenerate-thumbnails
 279 ```
 280
 281 ### create-transcoding-job.js
 282
 283 You can use this script to force transcoding of an existing video. PeerTube needs to be running.
 284
 285 To generate transcoding jobs depending on the instance configuration:
 286
 287 ```bash
 288 $ # Basic installation
 289 $ cd /var/www/peertube/peertube-latest
 290 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID]
 291
 292 $ # Docker installation
 293 $ cd /var/www/peertube-docker
 294 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID]
 295 ```
 296
 297 Or to transcode to a specific resolution:
 298
 299 ```bash
 300 $ # Basic installation
 301 $ cd /var/www/peertube/peertube-latest
 302 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 303
 304 $ # Docker installation
 305 $ cd /var/www/peertube-docker
 306 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 307 ```
 308
 309 The resolution should be an integer (`1080`, `720`, `480`, etc.)
 310
 311 To generate an HLS playlist for a video:
 312
 313 ```bash
 314 $ # Basic installation
 315 $ cd /var/www/peertube/peertube-latest
 316 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- --generate-hls -v [videoUUID]
 317
 318 $ # Docker installation
 319 $ cd /var/www/peertube-docker
 320 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- --generate-hls -v [videoUUID]
 321 ```
 322
 323 ### create-import-video-file-job.js
 324
 325 You can use this script to import a video file to replace an already uploaded file or to add a new resolution to a video. PeerTube needs to be running.
 326
 327 ```bash
 328 $ # Basic installation
 329 $ cd /var/www/peertube/peertube-latest
 330 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 331
 332 $ # Docker installation
 333 $ cd /var/www/peertube-docker
 334 $ docker-compose exec -u peertube peertube npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 335 ```
 336
 337 ### prune-storage.js
 338
 339 Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
 340 Stop PeerTube and delete these files (a confirmation will be demanded first):
 341
 342 ```bash
 343 $ cd /var/www/peertube/peertube-latest
 344 $ sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
 345 ```
 346
 347 ### optimize-old-videos.js
 348
 349 Before version v1.0.0-beta.16, Peertube did not specify a bitrate for the
 350 transcoding of uploaded videos. This means that videos might be encoded into
 351 very large files that are too large for streaming. This script re-transcodes
 352 these videos so that they can be watched properly, even on slow connections.
 353
 354 ```bash
 355 $ # Basic installation
 356 $ cd /var/www/peertube/peertube-latest
 357 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run optimize-old-videos
 358
 359 $ # Docker installation
 360 $ cd /var/www/peertube-docker
 361 $ docker-compose exec -u peertube peertube npm run optimize-old-videos
 362 ```
 363
 364
 365 ### update-host.js
 366
 367 **Changing the hostname is unsupported and may be a risky operation, especially if you have already federated.**
 368 If you started PeerTube with a domain, and then changed it you will have
 369 invalid torrent files and invalid URLs in your database. To fix this, you have
 370 to run the command below (keep in mind your follower instances will NOT update their URLs).
 371
 372 ```bash
 373 $ # Basic installation
 374 $ cd /var/www/peertube/peertube-latest
 375 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run update-host
 376
 377 $ # Docker installation
 378 $ cd /var/www/peertube-docker
 379 $ docker-compose exec -u peertube peertube npm run update-host
 380 ```
 381
 382 ### reset-password.js
 383
 384 To reset a user password from CLI, run:
 385
 386 ```bash
 387 $ # Basic installation
 388 $ cd /var/www/peertube/peertube-latest
 389 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run reset-password -- -u target_username
 390
 391 $ # Docker installation
 392 $ cd /var/www/peertube-docker
 393 $ docker-compose exec -u peertube peertube npm run reset-password -- -u target_username
 394 ```
 395
 396
 397 ### plugin install/uninstall
 398
 399 The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
 400 If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).
 401
 402 To install/update a plugin or a theme from the disk:
 403
 404 ```bash
 405 $ cd /var/www/peertube/peertube-latest
 406 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --plugin-path /local/plugin/path
 407
 408 $ # Docker installation
 409 $ cd /var/www/peertube-docker
 410 $ docker-compose exec -u peertube peertube npm run plugin:install -- --plugin-path /local/plugin/path
 411 ```
 412
 413 From NPM:
 414
 415 ```bash
 416 $ cd /var/www/peertube/peertube-latest
 417 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --npm-name peertube-plugin-myplugin
 418
 419 $ # Docker installation
 420 $ cd /var/www/peertube-docker
 421 $ docker-compose exec -u peertube peertube npm run plugin:install -- --npm-name peertube-plugin-myplugin
 422 ```
 423
 424 To uninstall a plugin or a theme:
 425
 426 ```bash
 427 $ cd /var/www/peertube/peertube-latest
 428 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 429
 430 $ # Docker installation
 431 $ cd /var/www/peertube-docker
 432 $ docker-compose exec -u peertube peertube npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 433 ```
 434
 435 ### REPL (Read Eval Print Loop)
 436
 437 If you want to interact with the application libraries and objects even when PeerTube is not running, there is a REPL for that.
 438
 439 usage: `node ./dist/server/tools/peertube-repl.js`
 440
 441 "The default evaluator will, by default, assign the result of the most recently evaluated expression to the special variable `_` (underscore). Explicitly setting `_` to a value will disable this behavior."
 442
 443 - type `.help` to list commands available in the repl, notice it starts with a dot
 444 - type `.exit` to exit, note that you still have to press CTRL-C to actually exit, or press CTRL-C (3 times) without typing `.exit` to exit
 445 - type `context` to list all available objects and libraries in the context, note: `Promise` is also available but it's not listed in the context, in case you need promises for something
 446 - type `env` to see the loaded environment variables
 447 - type `path` to access path library
 448 - type `lodash` to access lodash library
 449 - type `uuidv1` to access uuid/v1 library
 450 - type `uuidv3` to access uuid/v3 library
 451 - type `uuidv4` to access uuid/v4 library
 452 - type `uuidv5` to access uuid/v5 library
 453 - type `YoutubeDL` to access youtube-dl library
 454 - type `cli` to access the cli helpers object
 455 - type `logger` to access the logger; if you log to it, it will write to stdout and to the peertube.log file
 456 - type `constants` to access the constants loaded by the server
 457 - type `coreUtils` to access the core-utils helpers object
 458 - type `ffmpegUtils` to access the ffmpeg-utils helpers object
 459 - type `peertubeCryptoUtils` to access the peertube-crypto helpers object
 460 - type `signupUtils` to access the signup helpers object
 461 - type `utils` to access the utils helpers object
 462 - type `YoutubeDLUtils` to access the youtube-dl helpers object
 463 - type `sequelizeTypescript` to access sequelizeTypescript
 464 - type `modelsUtils` to access the models/utils
 465 - type `models` to access the shortcut to sequelizeTypescript.models
 466 - type `transaction` to access the shortcut to sequelizeTypescript.transaction
 467 - type `query` to access the shortcut to sequelizeTypescript.query
 468 - type `queryInterface` to access the shortcut to sequelizeTypescript.queryInterface
 469
 470 #### .help
 471
 472 ```
 473 PeerTube [1.0.0] (b10eb595)> .help
 474 .break    Sometimes you get stuck, this gets you out
 475 .clear    Break, and also clear the local context
 476 .editor   Enter editor mode
 477 .exit     Exit the repl
 478 .help     Print this help message
 479 .load     Load JS from a file into the REPL session
 480 .r        Reset REPL
 481 .reset    Reset REPL
 482 .save     Save all evaluated commands in this REPL session to a file
 483 PeerTube [1.0.0] (b10eb595)>
 484 ```
 485
 486 #### Lodash example
 487
 488 ```
 489 PeerTube [1.0.0] (b10eb595)> lodash.keys(context)
 490 [ 'global',
 491   'console',
 492   'DTRACE_NET_SERVER_CONNECTION',
 493   'DTRACE_NET_STREAM_END',
 494   'DTRACE_HTTP_SERVER_REQUEST',
 495   'DTRACE_HTTP_SERVER_RESPONSE',
 496   'DTRACE_HTTP_CLIENT_REQUEST',
 497   'DTRACE_HTTP_CLIENT_RESPONSE',
 498   'process',
 499   'Buffer',
 500   'clearImmediate',
 501   'clearInterval',
 502   'clearTimeout',
 503   'setImmediate',
 504   'setInterval',
 505   'setTimeout',
 506   'XMLHttpRequest',
 507   'compact2string',
 508   'module',
 509   'require',
 510   'path',
 511   'repl',
 512   'context',
 513   'env',
 514   'lodash',
 515   'uuidv1',
 516   'uuidv3',
 517   'uuidv4',
 518   'uuidv5',
 519   'cli',
 520   'logger',
 521   'constants',
 522   'Sequelize',
 523   'sequelizeTypescript',
 524   'modelsUtils',
 525   'models',
 526   'transaction',
 527   'query',
 528   'queryInterface',
 529   'YoutubeDL',
 530   'coreUtils',
 531   'ffmpegUtils',
 532   'peertubeCryptoUtils',
 533   'signupUtils',
 534   'utils',
 535   'YoutubeDLUtils' ]
 536 PeerTube [1.0.0] (b10eb595)>
 537 ```
 538
 539 #### YoutubeDL example
 540 ```
 541 YoutubeDL.getInfo('https://www.youtube.com/watch?v=I5ZN289jjDo', function(err, data) {console.log(err, data)})
 542 ```
 543
 544 #### Models examples
 545 ```
 546 PeerTube [1.0.0] (b10eb595)> new models.ActorModel({id: 3}).getVideoChannel().then(function(data){console.log(data.dataValues.name)})
 547 Promise {
 548   _bitField: 0,
 549   _fulfillmentHandler0: undefined,
 550   _rejectionHandler0: undefined,
 551   _promise0: undefined,
 552   _receiver0: undefined }
 553 PeerTube [1.0.0] (b10eb595)> Main root channel
 554 PeerTube [1.0.0] (b10eb595)> let out; new models.UserModel({id: 1}).getAccount().then(function (data) {out = data.dataValues.id})
 555 Promise {
 556   _bitField: 0,
 557   _fulfillmentHandler0: undefined,
 558   _rejectionHandler0: undefined,
 559   _promise0: undefined,
 560   _receiver0: undefined }
 561 PeerTube [1.0.0] (b10eb595)> out
 562 2
 563 PeerTube [1.0.0] (b10eb595)>
 564 ```