support/doc/tools.md

   1 # CLI tools guide
   2
   3 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
   4 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
   5 **Table of Contents**
   6
   7 - [Remote Tools](#remote-tools)
   8   - [Dependencies](#dependencies)
   9   - [Installation](#installation)
  10   - [CLI wrapper](#cli-wrapper)
  11     - [peertube-import-videos.js](#peertube-import-videosjs)
  12     - [peertube-upload.js](#peertube-uploadjs)
  13     - [peertube-watch.js](#peertube-watchjs)
  14     - [peertube-plugins.js](#peertube-pluginsjs)
  15 - [Server tools](#server-tools)
  16   - [parse-log](#parse-log)
  17   - [create-transcoding-job.js](#create-transcoding-jobjs)
  18   - [create-import-video-file-job.js](#create-import-video-file-jobjs)
  19   - [prune-storage.js](#prune-storagejs)
  20   - [optimize-old-videos.js](#optimize-old-videosjs)
  21   - [update-host.js](#update-hostjs)
  22   - [reset-password.js](#reset-passwordjs)
  23   - [plugin install/uninstall](#plugin-installuninstall)
  24   - [REPL (Read Eval Print Loop)](#repl-read-eval-print-loop)
  25     - [.help](#help)
  26     - [Lodash example](#lodash-example)
  27     - [YoutubeDL example](#youtubedl-example)
  28     - [Models examples](#models-examples)
  29
  30 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
  31
  32 ## Remote Tools
  33
  34 You need at least 512MB RAM to run the script.
  35 Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
  36 You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).
  37
  38 ### Dependencies
  39
  40 Install the [PeerTube dependencies](dependencies.md) except PostgreSQL and Redis.
  41 PeerTube only supports NodeJS 10.x.
  42
  43 ### Installation
  44
  45 Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):
  46
  47 ```
  48 $ git clone https://github.com/Chocobozzz/PeerTube.git
  49 $ CLONE="$(pwd)/PeerTube"
  50 $ cd ${CLONE}
  51 ```
  52
  53 Install dependencies and build CLI tools:
  54
  55 ```
  56 $ NOCLIENT=1 yarn install --pure-lockfile
  57 $ npm run setup:cli
  58 ```
  59
  60 ### CLI wrapper
  61
  62 The wrapper provides a convenient interface to the following scripts.
  63 You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):
  64
  65 ```
  66   Usage: peertube [command] [options]
  67
  68   Options:
  69
  70     -v, --version         output the version number
  71     -h, --help            output usage information
  72
  73   Commands:
  74
  75     auth [action]         register your accounts on remote instances to use them with other commands
  76     upload|up             upload a video
  77     import-videos|import  import a video from a streaming platform
  78     watch|w               watch a video in the terminal ✩°｡⋆
  79     repl                  initiate a REPL to access internals
  80     plugins|p [action]    manag instance plugins
  81     help [cmd]            display help for [cmd]
  82 ```
  83
  84 The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.
  85
  86 ```bash
  87 $ peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
  88 $ peertube auth list
  89 ┌──────────────────────────────┬──────────────────────────────┐
  90 │ instance                     │ login                        │
  91 ├──────────────────────────────┼──────────────────────────────┤
  92 │ 'PEERTUBE_URL'               │ 'PEERTUBE_USER'              │
  93 └──────────────────────────────┴──────────────────────────────┘
  94 ```
  95
  96 You can now use that account to upload videos without feeding the same parameters again.
  97
  98 ```bash
  99 $ peertube up <videoFile>
 100 ```
 101
 102 And now that your video is online, you can watch it from the confort of your terminal (use `peertube watch --help` to see the supported players):
 103
 104 ```bash
 105 $ peertube watch https://peertube.cpy.re/videos/watch/e8a1af4e-414a-4d58-bfe6-2146eed06d10
 106 ```
 107
 108 To list, install, uninstall dynamically plugins/themes of an instance:
 109
 110 ```bash
 111 $ peertube plugins list
 112 $ peertube plugins install --path /local/plugin/path
 113 $ peertube plugins install --npm-name peertube-plugin-myplugin
 114 $ peertube plugins uninstall --npm-name peertube-plugin-myplugin
 115 ```
 116
 117 #### peertube-import-videos.js
 118
 119 You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
 120 Be sure you own the videos or have the author's authorization to do so.
 121
 122 ```sh
 123 $ node dist/server/tools/peertube-import-videos.js \
 124     -u 'PEERTUBE_URL' \
 125     -U 'PEERTUBE_USER' \
 126     --password 'PEERTUBE_PASSWORD' \
 127     -t 'TARGET_URL'
 128 ```
 129
 130 * `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
 131 * `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
 132 * `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
 133 * `TARGET_URL` : the target url you want to import. Examples:
 134   * YouTube:
 135     * Channel: https://www.youtube.com/channel/ChannelId
 136     * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
 137     * Video https://www.youtube.com/watch?v=blabla
 138   * Vimeo: https://vimeo.com/xxxxxx
 139   * Dailymotion: https://www.dailymotion.com/xxxxx
 140
 141 The script will get all public videos from Youtube, download them and upload to PeerTube.
 142 Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...
 143
 144 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 145
 146 **NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
 147 you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
 148 these rules (ensure to customize them to your needs):
 149
 150 ```
 151 # Update youtube-dl every day at midnight
 152 0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/
 153
 154 # Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
 155 0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +%Y-%m-%d)
 156 ```
 157
 158 Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.
 159
 160 #### peertube-upload.js
 161
 162 You can use this script to import videos directly from the CLI.
 163
 164 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 165
 166 ```
 167 $ cd ${CLONE}
 168 $ node dist/server/tools/peertube-upload.js --help
 169 ```
 170
 171 #### peertube-watch.js
 172
 173 You can use this script to play videos directly from the CLI.
 174
 175 It provides support for different players:
 176
 177 - ascii (default ; plays in ascii art in your terminal!)
 178 - mpv
 179 - mplayer
 180 - vlc
 181 - stdout
 182 - xbmc
 183 - airplay
 184 - chromecast
 185
 186
 187 #### peertube-plugins.js
 188
 189 Install/update/uninstall or list local or NPM PeerTube plugins:
 190
 191 ```
 192 $ cd ${CLONE}
 193 $ node dist/server/tools/peertube-plugins.js --help
 194 $ node dist/server/tools/peertube-plugins.js list --help
 195 $ node dist/server/tools/peertube-plugins.js install --help
 196 $ node dist/server/tools/peertube-plugins.js update --help
 197 $ node dist/server/tools/peertube-plugins.js uninstall --help
 198
 199 $ node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
 200 $ node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
 201 ```
 202
 203 ## Server tools
 204
 205 These scripts should be run on the server, in `peertube-latest` directory.
 206
 207 ### parse-log
 208
 209 To parse PeerTube last log file:
 210
 211 ```
 212 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level info
 213 ```
 214
 215 `--level` is optional and could be `info`/`warn`/`error`
 216
 217 ### create-transcoding-job.js
 218
 219 You can use this script to force transcoding of an existing video. PeerTube needs to be running.
 220
 221 ```
 222 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID]
 223 ```
 224
 225 Or to transcode to a specific resolution:
 226 ```
 227 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 228 ```
 229
 230 ### create-import-video-file-job.js
 231
 232 You can use this script to import a video file to replace an already uploaded file or to add a new resolution to a video. PeerTube needs to be running.
 233
 234 ```
 235 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 236 ```
 237
 238 ### prune-storage.js
 239
 240 Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
 241 Stop PeerTube and delete these files (a confirmation will be demanded first):
 242
 243 ```
 244 $ sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
 245 ```
 246
 247 ### optimize-old-videos.js
 248
 249 Before version v1.0.0-beta.16, Peertube did not specify a bitrate for the
 250 transcoding of uploaded videos. This means that videos might be encoded into
 251 very large files that are too large for streaming. This script re-transcodes
 252 these videos so that they can be watched properly, even on slow connections.
 253
 254 ```
 255 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run optimize-old-videos
 256 ```
 257
 258
 259 ### update-host.js
 260
 261 If you started PeerTube with a domain, and then changed it you will have
 262 invalid torrent files and invalid URLs in your database. To fix this, you have
 263 to run:
 264
 265 ```
 266 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run update-host
 267 ```
 268
 269 ### reset-password.js
 270
 271 To reset a user password from CLI, run:
 272
 273 ```
 274 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run reset-password -- -u target_username
 275 ```
 276
 277
 278 ### plugin install/uninstall
 279
 280 The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
 281 If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).
 282
 283 To install/update a plugin or a theme from the disk:
 284
 285 ```
 286 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --plugin-path /local/plugin/path
 287 ```
 288
 289 From NPM:
 290
 291 ```
 292 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --npm-name peertube-plugin-myplugin
 293 ```
 294
 295 To uninstall a plugin or a theme:
 296
 297 ```
 298 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 299 ```
 300
 301 ### REPL ([Read Eval Print Loop](https://nodejs.org/docs/latest-v10.x/api/repl.html))
 302
 303 If you want to interact with the application libraries and objects even when PeerTube is not running, there is a REPL for that.
 304
 305 usage: `node ./dist/server/tools/peertube-repl.js`
 306
 307 "The default evaluator will, by default, assign the result of the most recently evaluated expression to the special variable `_` (underscore). Explicitly setting `_` to a value will disable this behavior."
 308
 309 - type `.help` to list commands available in the repl, notice it starts with a dot
 310 - type `.exit` to exit, note that you still have to press CTRL-C to actually exit, or press CTRL-C (3 times) without typing `.exit` to exit
 311 - type `context` to list all available objects and libraries in the context, note: `Promise` is also available but it's not listed in the context, in case you need promises for something
 312 - type `env` to see the loaded environment variables
 313 - type `path` to access path library
 314 - type `lodash` to access lodash library
 315 - type `uuidv1` to access uuid/v1 library
 316 - type `uuidv3` to access uuid/v3 library
 317 - type `uuidv4` to access uuid/v4 library
 318 - type `uuidv5` to access uuid/v5 library
 319 - type `YoutubeDL` to access youtube-dl library
 320 - type `cli` to access the cli helpers object
 321 - type `logger` to access the logger; if you log to it, it will write to stdout and to the peertube.log file
 322 - type `constants` to access the constants loaded by the server
 323 - type `coreUtils` to access the core-utils helpers object
 324 - type `ffmpegUtils` to access the ffmpeg-utils helpers object
 325 - type `peertubeCryptoUtils` to access the peertube-crypto helpers object
 326 - type `signupUtils` to access the signup helpers object
 327 - type `utils` to access the utils helpers object
 328 - type `YoutubeDLUtils` to access the youtube-dl helpers object
 329 - type `sequelizeTypescript` to access sequelizeTypescript
 330 - type `modelsUtils` to access the models/utils
 331 - type `models` to access the shortcut to sequelizeTypescript.models
 332 - type `transaction` to access the shortcut to sequelizeTypescript.transaction
 333 - type `query` to access the shortcut to sequelizeTypescript.query
 334 - type `queryInterface` to access the shortcut to sequelizeTypescript.queryInterface
 335
 336 #### .help
 337
 338 ```
 339 PeerTube [1.0.0] (b10eb595)> .help
 340 .break    Sometimes you get stuck, this gets you out
 341 .clear    Break, and also clear the local context
 342 .editor   Enter editor mode
 343 .exit     Exit the repl
 344 .help     Print this help message
 345 .load     Load JS from a file into the REPL session
 346 .r        Reset REPL
 347 .reset    Reset REPL
 348 .save     Save all evaluated commands in this REPL session to a file
 349 PeerTube [1.0.0] (b10eb595)>
 350 ```
 351
 352 #### Lodash example
 353
 354 ```
 355 PeerTube [1.0.0] (b10eb595)> lodash.keys(context)
 356 [ 'global',
 357   'console',
 358   'DTRACE_NET_SERVER_CONNECTION',
 359   'DTRACE_NET_STREAM_END',
 360   'DTRACE_HTTP_SERVER_REQUEST',
 361   'DTRACE_HTTP_SERVER_RESPONSE',
 362   'DTRACE_HTTP_CLIENT_REQUEST',
 363   'DTRACE_HTTP_CLIENT_RESPONSE',
 364   'process',
 365   'Buffer',
 366   'clearImmediate',
 367   'clearInterval',
 368   'clearTimeout',
 369   'setImmediate',
 370   'setInterval',
 371   'setTimeout',
 372   'XMLHttpRequest',
 373   'compact2string',
 374   'module',
 375   'require',
 376   'path',
 377   'repl',
 378   'context',
 379   'env',
 380   'lodash',
 381   'uuidv1',
 382   'uuidv3',
 383   'uuidv4',
 384   'uuidv5',
 385   'cli',
 386   'logger',
 387   'constants',
 388   'Sequelize',
 389   'sequelizeTypescript',
 390   'modelsUtils',
 391   'models',
 392   'transaction',
 393   'query',
 394   'queryInterface',
 395   'YoutubeDL',
 396   'coreUtils',
 397   'ffmpegUtils',
 398   'peertubeCryptoUtils',
 399   'signupUtils',
 400   'utils',
 401   'YoutubeDLUtils' ]
 402 PeerTube [1.0.0] (b10eb595)>
 403 ```
 404
 405 #### YoutubeDL example
 406 ```
 407 YoutubeDL.getInfo('https://www.youtube.com/watch?v=I5ZN289jjDo', function(err, data) {console.log(err, data)})
 408 ```
 409
 410 #### Models examples
 411 ```
 412 PeerTube [1.0.0] (b10eb595)> new models.ActorModel({id: 3}).getVideoChannel().then(function(data){console.log(data.dataValues.name)})
 413 Promise {
 414   _bitField: 0,
 415   _fulfillmentHandler0: undefined,
 416   _rejectionHandler0: undefined,
 417   _promise0: undefined,
 418   _receiver0: undefined }
 419 PeerTube [1.0.0] (b10eb595)> Main root channel
 420 PeerTube [1.0.0] (b10eb595)> let out; new models.UserModel({id: 1}).getAccount().then(function (data) {out = data.dataValues.id})
 421 Promise {
 422   _bitField: 0,
 423   _fulfillmentHandler0: undefined,
 424   _rejectionHandler0: undefined,
 425   _promise0: undefined,
 426   _receiver0: undefined }
 427 PeerTube [1.0.0] (b10eb595)> out
 428 2
 429 PeerTube [1.0.0] (b10eb595)>
 430 ```