support/doc/tools.md

   1 # CLI tools guide
   2
   3 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
   4 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
   5 **Table of Contents**
   6
   7 - [Remote Tools](#remote-tools)
   8   - [Dependencies](#dependencies)
   9   - [Installation](#installation)
  10   - [CLI wrapper](#cli-wrapper)
  11     - [peertube-import-videos.js](#peertube-import-videosjs)
  12     - [peertube-upload.js](#peertube-uploadjs)
  13     - [peertube-plugins.js](#peertube-pluginsjs)
  14     - [peertube-redundancy.js](#peertube-redundancyjs)
  15 - [Server tools](#server-tools)
  16   - [parse-log](#parse-log)
  17   - [regenerate-thumbnails.js](#regenerate-thumbnailsjs)
  18   - [create-transcoding-job.js](#create-transcoding-jobjs)
  19   - [create-import-video-file-job.js](#create-import-video-file-jobjs)
  20   - [create-move-video-storage-job.js](#create-move-video-storage-jobjs)
  21   - [prune-storage.js](#prune-storagejs)
  22   - [update-host.js](#update-hostjs)
  23   - [reset-password.js](#reset-passwordjs)
  24   - [plugin install/uninstall](#plugin-installuninstall)
  25
  26 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
  27
  28 ## Remote Tools
  29
  30 You need at least 512MB RAM to run the script.
  31 Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
  32 You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).
  33
  34 ### Dependencies
  35
  36 Install the [PeerTube dependencies](/support/doc/dependencies.md) except PostgreSQL and Redis.
  37
  38 ### Installation
  39
  40 Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):
  41
  42 ```bash
  43 $ git clone https://github.com/Chocobozzz/PeerTube.git
  44 $ CLONE="$(pwd)/PeerTube"
  45 $ cd ${CLONE}
  46 ```
  47
  48 Install dependencies and build CLI tools:
  49
  50 ```bash
  51 $ NOCLIENT=1 yarn install --pure-lockfile
  52 $ npm run setup:cli
  53 ```
  54
  55 ### CLI wrapper
  56
  57 The wrapper provides a convenient interface to the following scripts.
  58 You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):
  59
  60 ```
  61   Usage: peertube [command] [options]
  62
  63   Options:
  64
  65     -v, --version         output the version number
  66     -h, --help            output usage information
  67
  68   Commands:
  69
  70     auth [action]         register your accounts on remote instances to use them with other commands
  71     upload|up             upload a video
  72     import-videos|import  import a video from a streaming platform
  73     plugins|p [action]    manage instance plugins
  74     redundancy|r [action] manage video redundancies
  75     help [cmd]            display help for [cmd]
  76 ```
  77
  78 The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.
  79
  80 ```bash
  81 $ peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
  82 $ peertube auth list
  83 ┌──────────────────────────────┬──────────────────────────────┐
  84 │ instance                     │ login                        │
  85 ├──────────────────────────────┼──────────────────────────────┤
  86 │ 'PEERTUBE_URL'               │ 'PEERTUBE_USER'              │
  87 └──────────────────────────────┴──────────────────────────────┘
  88 ```
  89
  90 You can now use that account to upload videos without feeding the same parameters again.
  91
  92 ```bash
  93 $ peertube up <videoFile>
  94 ```
  95
  96 To list, install, uninstall dynamically plugins/themes of an instance:
  97
  98 ```bash
  99 $ peertube plugins list
 100 $ peertube plugins install --path /local/plugin/path
 101 $ peertube plugins install --npm-name peertube-plugin-myplugin
 102 $ peertube plugins uninstall --npm-name peertube-plugin-myplugin
 103 ```
 104
 105 #### peertube-import-videos.js
 106
 107 You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
 108 Be sure you own the videos or have the author's authorization to do so.
 109
 110 ```sh
 111 $ node dist/server/tools/peertube-import-videos.js \
 112     -u 'PEERTUBE_URL' \
 113     -U 'PEERTUBE_USER' \
 114     --password 'PEERTUBE_PASSWORD' \
 115     --target-url 'TARGET_URL'
 116 ```
 117
 118 * `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
 119 * `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
 120 * `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
 121 * `TARGET_URL` : the target url you want to import. Examples:
 122   * YouTube:
 123     * Channel: https://www.youtube.com/channel/ChannelId
 124     * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
 125     * Video https://www.youtube.com/watch?v=blabla
 126   * Vimeo: https://vimeo.com/xxxxxx
 127   * Dailymotion: https://www.dailymotion.com/xxxxx
 128
 129 The script will get all public videos from Youtube, download them and upload to PeerTube.
 130 Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...
 131
 132 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 133
 134 **NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
 135 you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
 136 these rules (ensure to customize them to your needs):
 137
 138 ```
 139 # Update youtube-dl every day at midnight
 140 0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/
 141
 142 # Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
 143 0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +\%Y-\%m-\%d)
 144 ```
 145
 146 Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.
 147
 148 #### peertube-upload.js
 149
 150 You can use this script to import videos directly from the CLI.
 151
 152 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 153
 154 ```bash
 155 $ cd ${CLONE}
 156 $ node dist/server/tools/peertube-upload.js --help
 157 ```
 158
 159 #### peertube-plugins.js
 160
 161 Install/update/uninstall or list local or NPM PeerTube plugins:
 162
 163 ```bash
 164 $ cd ${CLONE}
 165 $ node dist/server/tools/peertube-plugins.js --help
 166 $ node dist/server/tools/peertube-plugins.js list --help
 167 $ node dist/server/tools/peertube-plugins.js install --help
 168 $ node dist/server/tools/peertube-plugins.js update --help
 169 $ node dist/server/tools/peertube-plugins.js uninstall --help
 170
 171 $ node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
 172 $ node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
 173 ```
 174
 175 #### peertube-redundancy.js
 176
 177 Manage (list/add/remove) video redundancies:
 178
 179 To list your videos that are duplicated by remote instances:
 180
 181 ```bash
 182 $ node dist/server/tools/peertube.js redundancy list-remote-redundancies
 183 ```
 184
 185 To list remote videos that your instance duplicated:
 186
 187 ```bash
 188 $ node dist/server/tools/peertube.js redundancy list-my-redundancies
 189 ```
 190
 191 To duplicate a specific video in your redundancy system:
 192
 193 ```bash
 194 $ node dist/server/tools/peertube.js redundancy add --video 823
 195 ```
 196
 197 To remove a video redundancy:
 198
 199 ```bash
 200 $ node dist/server/tools/peertube.js redundancy remove --video 823
 201 ```
 202
 203 ## Server tools
 204
 205 These scripts should be run on the server, in `peertube-latest` directory.
 206
 207 ### parse-log
 208
 209 To parse PeerTube last log file:
 210
 211 ```bash
 212 $ # Basic installation
 213 $ cd /var/www/peertube/peertube-latest
 214 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level info
 215
 216 $ # Docker installation
 217 $ cd /var/www/peertube-docker
 218 $ docker-compose exec -u peertube peertube npm run parse-log -- --level info
 219 ```
 220
 221 `--level` is optional and could be `info`/`warn`/`error`
 222
 223 You can also remove SQL or HTTP logs using `--not-tags` (PeerTube >= 3.2):
 224
 225 ```bash
 226 $ # Basic installation
 227 $ cd /var/www/peertube/peertube-latest
 228 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level debug --not-tags http sql
 229
 230 $ # Docker installation
 231 $ cd /var/www/peertube-docker
 232 $ docker-compose exec -u peertube peertube npm run parse-log -- --level debug --not-tags http sql
 233 ```
 234
 235 ### regenerate-thumbnails.js
 236
 237 **PeerTube >= 3.2**
 238
 239 Regenerating local video thumbnails could be useful because new PeerTube releases may increase thumbnail sizes:
 240
 241 ```bash
 242 $ # Basic installation
 243 $ cd /var/www/peertube/peertube-latest
 244 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run regenerate-thumbnails
 245
 246 $ # Docker installation
 247 $ cd /var/www/peertube-docker
 248 $ docker-compose exec -u peertube peertube npm run regenerate-thumbnails
 249 ```
 250
 251 ### create-transcoding-job.js
 252
 253 You can use this script to force transcoding of an existing video. PeerTube needs to be running.
 254
 255 To generate transcoding jobs depending on the instance configuration:
 256
 257 ```bash
 258 $ # Basic installation
 259 $ cd /var/www/peertube/peertube-latest
 260 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID]
 261
 262 $ # Docker installation
 263 $ cd /var/www/peertube-docker
 264 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID]
 265 ```
 266
 267 Or to transcode to a specific resolution:
 268
 269 ```bash
 270 $ # Basic installation
 271 $ cd /var/www/peertube/peertube-latest
 272 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 273
 274 $ # Docker installation
 275 $ cd /var/www/peertube-docker
 276 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
 277 ```
 278
 279 The resolution should be an integer (`1080`, `720`, `480`, etc.)
 280
 281 To generate an HLS playlist for a video:
 282
 283 ```bash
 284 $ # Basic installation
 285 $ cd /var/www/peertube/peertube-latest
 286 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- --generate-hls -v [videoUUID]
 287
 288 $ # Docker installation
 289 $ cd /var/www/peertube-docker
 290 $ docker-compose exec -u peertube peertube npm run create-transcoding-job -- --generate-hls -v [videoUUID]
 291 ```
 292
 293 ### create-import-video-file-job.js
 294
 295 You can use this script to import a video file to replace an already uploaded file or to add a new webtorrent resolution to a video. PeerTube needs to be running.
 296 You can then create a transcoding job using `npm run create-transcoding-job` if you need to optimize your file or create an HLS version of it.
 297
 298 ```bash
 299 $ # Basic installation
 300 $ cd /var/www/peertube/peertube-latest
 301 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 302
 303 $ # Docker installation
 304 $ cd /var/www/peertube-docker
 305 $ docker-compose exec -u peertube peertube npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 306 ```
 307
 308 ### create-move-video-storage-job.js
 309
 310 **PeerTube >= 4.0**
 311
 312 Use this script to move all video files or a specific video file to object storage.
 313
 314 ```bash
 315 $ # Basic installation
 316 $ cd /var/www/peertube/peertube-latest
 317 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-move-video-storage-job -- --to-object-storage -v [videoUUID]
 318
 319 $ # Docker installation
 320 $ cd /var/www/peertube-docker
 321 $ docker-compose exec -u peertube peertube npm run create-move-video-storage-job -- --to-object-storage -v [videoUUID]
 322 ```
 323
 324 The script can also move all video files that are not already in object storage:
 325
 326 ```bash
 327 $ # Basic installation
 328 $ cd /var/www/peertube/peertube-latest
 329 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-move-video-storage-job -- --to-object-storage --all-videos
 330
 331 $ # Docker installation
 332 $ cd /var/www/peertube-docker
 333 $ docker-compose exec -u peertube peertube npm run create-move-video-storage-job -- --to-object-storage --all-videos
 334 ```
 335
 336
 337 ### prune-storage.js
 338
 339 Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
 340 Stop PeerTube and delete these files (a confirmation will be demanded first):
 341
 342 ```bash
 343 $ cd /var/www/peertube/peertube-latest
 344 $ sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
 345 ```
 346
 347
 348 ### update-host.js
 349
 350 **Changing the hostname is unsupported and may be a risky operation, especially if you have already federated.**
 351 If you started PeerTube with a domain, and then changed it you will have
 352 invalid torrent files and invalid URLs in your database. To fix this, you have
 353 to run the command below (keep in mind your follower instances will NOT update their URLs).
 354
 355 ```bash
 356 $ # Basic installation
 357 $ cd /var/www/peertube/peertube-latest
 358 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run update-host
 359
 360 $ # Docker installation
 361 $ cd /var/www/peertube-docker
 362 $ docker-compose exec -u peertube peertube npm run update-host
 363 ```
 364
 365 ### reset-password.js
 366
 367 To reset a user password from CLI, run:
 368
 369 ```bash
 370 $ # Basic installation
 371 $ cd /var/www/peertube/peertube-latest
 372 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run reset-password -- -u target_username
 373
 374 $ # Docker installation
 375 $ cd /var/www/peertube-docker
 376 $ docker-compose exec -u peertube peertube npm run reset-password -- -u target_username
 377 ```
 378
 379
 380 ### plugin install/uninstall
 381
 382 The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
 383 If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).
 384
 385 To install/update a plugin or a theme from the disk:
 386
 387 ```bash
 388 $ cd /var/www/peertube/peertube-latest
 389 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --plugin-path /local/plugin/path
 390
 391 $ # Docker installation
 392 $ cd /var/www/peertube-docker
 393 $ docker-compose exec -u peertube peertube npm run plugin:install -- --plugin-path /local/plugin/path
 394 ```
 395
 396 From NPM:
 397
 398 ```bash
 399 $ cd /var/www/peertube/peertube-latest
 400 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --npm-name peertube-plugin-myplugin
 401
 402 $ # Docker installation
 403 $ cd /var/www/peertube-docker
 404 $ docker-compose exec -u peertube peertube npm run plugin:install -- --npm-name peertube-plugin-myplugin
 405 ```
 406
 407 To uninstall a plugin or a theme:
 408
 409 ```bash
 410 $ cd /var/www/peertube/peertube-latest
 411 $ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 412
 413 $ # Docker installation
 414 $ cd /var/www/peertube-docker
 415 $ docker-compose exec -u peertube peertube npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 416 ```