support/doc/tools.md

   1 # CLI tools guide
   2
   3 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
   4 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
   5 **Table of Contents**
   6
   7 - [Remote Tools](#remote-tools)
   8   - [Dependencies](#dependencies)
   9   - [Installation](#installation)
  10   - [CLI wrapper](#cli-wrapper)
  11     - [peertube-import-videos.js](#peertube-import-videosjs)
  12     - [peertube-upload.js](#peertube-uploadjs)
  13     - [peertube-plugins.js](#peertube-pluginsjs)
  14     - [peertube-redundancy.js](#peertube-redundancyjs)
  15 - [Server tools](#server-tools)
  16   - [parse-log](#parse-log)
  17   - [regenerate-thumbnails.js](#regenerate-thumbnailsjs)
  18   - [create-import-video-file-job.js](#create-import-video-file-jobjs)
  19   - [create-move-video-storage-job.js](#create-move-video-storage-jobjs)
  20   - [prune-storage.js](#prune-storagejs)
  21   - [update-host.js](#update-hostjs)
  22   - [reset-password.js](#reset-passwordjs)
  23   - [plugin install/uninstall](#plugin-installuninstall)
  24
  25 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
  26
  27 ## Remote Tools
  28
  29 You need at least 512MB RAM to run the script.
  30 Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
  31 You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).
  32
  33 ### Dependencies
  34
  35 Install the [PeerTube dependencies](/support/doc/dependencies.md) except PostgreSQL and Redis.
  36
  37 ### Installation
  38
  39 Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):
  40
  41 ```bash
  42 git clone https://github.com/Chocobozzz/PeerTube.git
  43 CLONE="$(pwd)/PeerTube"
  44 cd ${CLONE}
  45 ```
  46
  47 Install dependencies and build CLI tools:
  48
  49 ```bash
  50 NOCLIENT=1 yarn install --pure-lockfile
  51 npm run setup:cli
  52 ```
  53
  54 ### CLI wrapper
  55
  56 The wrapper provides a convenient interface to the following scripts.
  57 You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):
  58
  59 ```
  60   Usage: peertube [command] [options]
  61
  62   Options:
  63
  64     -v, --version         output the version number
  65     -h, --help            output usage information
  66
  67   Commands:
  68
  69     auth [action]         register your accounts on remote instances to use them with other commands
  70     upload|up             upload a video
  71     import-videos|import  import a video from a streaming platform
  72     plugins|p [action]    manage instance plugins
  73     redundancy|r [action] manage video redundancies
  74     help [cmd]            display help for [cmd]
  75 ```
  76
  77 The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.
  78
  79 ```bash
  80 peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
  81 peertube auth list
  82 ┌──────────────────────────────┬──────────────────────────────┐
  83 │ instance                     │ login                        │
  84 ├──────────────────────────────┼──────────────────────────────┤
  85 │ 'PEERTUBE_URL'               │ 'PEERTUBE_USER'              │
  86 └──────────────────────────────┴──────────────────────────────┘
  87 ```
  88
  89 You can now use that account to upload videos without feeding the same parameters again.
  90
  91 ```bash
  92 peertube up <videoFile>
  93 ```
  94
  95 To list, install, uninstall dynamically plugins/themes of an instance:
  96
  97 ```bash
  98 peertube plugins list
  99 peertube plugins install --path /local/plugin/path
 100 peertube plugins install --npm-name peertube-plugin-myplugin
 101 peertube plugins uninstall --npm-name peertube-plugin-myplugin
 102 ```
 103
 104 #### peertube-import-videos.js
 105
 106 You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
 107 Be sure you own the videos or have the author's authorization to do so.
 108
 109 ```sh
 110 node dist/server/tools/peertube-import-videos.js \
 111     -u 'PEERTUBE_URL' \
 112     -U 'PEERTUBE_USER' \
 113     --password 'PEERTUBE_PASSWORD' \
 114     --target-url 'TARGET_URL'
 115 ```
 116
 117 * `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
 118 * `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
 119 * `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
 120 * `TARGET_URL` : the target url you want to import. Examples:
 121   * YouTube:
 122     * Channel: https://www.youtube.com/channel/ChannelId
 123     * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
 124     * Video https://www.youtube.com/watch?v=blabla
 125   * Vimeo: https://vimeo.com/xxxxxx
 126   * Dailymotion: https://www.dailymotion.com/xxxxx
 127
 128 The script will get all public videos from Youtube, download them and upload to PeerTube.
 129 Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...
 130
 131 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 132
 133 **NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
 134 you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
 135 these rules (ensure to customize them to your needs):
 136
 137 ```
 138 # Update youtube-dl every day at midnight
 139 0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/
 140
 141 # Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
 142 0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +\%Y-\%m-\%d)
 143 ```
 144
 145 Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.
 146
 147 #### peertube-upload.js
 148
 149 You can use this script to import videos directly from the CLI.
 150
 151 Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).
 152
 153 ```bash
 154 cd ${CLONE}
 155 node dist/server/tools/peertube-upload.js --help
 156 ```
 157
 158 #### peertube-plugins.js
 159
 160 Install/update/uninstall or list local or NPM PeerTube plugins:
 161
 162 ```bash
 163 cd ${CLONE}
 164 node dist/server/tools/peertube-plugins.js --help
 165 node dist/server/tools/peertube-plugins.js list --help
 166 node dist/server/tools/peertube-plugins.js install --help
 167 node dist/server/tools/peertube-plugins.js update --help
 168 node dist/server/tools/peertube-plugins.js uninstall --help
 169
 170 node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
 171 node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
 172 ```
 173
 174 #### peertube-redundancy.js
 175
 176 Manage (list/add/remove) video redundancies:
 177
 178 To list your videos that are duplicated by remote instances:
 179
 180 ```bash
 181 node dist/server/tools/peertube.js redundancy list-remote-redundancies
 182 ```
 183
 184 To list remote videos that your instance duplicated:
 185
 186 ```bash
 187 node dist/server/tools/peertube.js redundancy list-my-redundancies
 188 ```
 189
 190 To duplicate a specific video in your redundancy system:
 191
 192 ```bash
 193 node dist/server/tools/peertube.js redundancy add --video 823
 194 ```
 195
 196 To remove a video redundancy:
 197
 198 ```bash
 199 node dist/server/tools/peertube.js redundancy remove --video 823
 200 ```
 201
 202 ## Server tools
 203
 204 These scripts should be run on the server, in `peertube-latest` directory.
 205
 206 ### parse-log
 207
 208 To parse PeerTube last log file:
 209
 210 ```bash
 211 # Basic installation
 212 cd /var/www/peertube/peertube-latest
 213 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level info
 214
 215 # Docker installation
 216 cd /var/www/peertube-docker
 217 docker-compose exec -u peertube peertube npm run parse-log -- --level info
 218 ```
 219
 220 `--level` is optional and could be `info`/`warn`/`error`
 221
 222 You can also remove SQL or HTTP logs using `--not-tags` (PeerTube >= 3.2):
 223
 224 ```bash
 225 # Basic installation
 226 cd /var/www/peertube/peertube-latest
 227 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level debug --not-tags http sql
 228
 229 # Docker installation
 230 cd /var/www/peertube-docker
 231 docker-compose exec -u peertube peertube npm run parse-log -- --level debug --not-tags http sql
 232 ```
 233
 234 ### regenerate-thumbnails.js
 235
 236 **PeerTube >= 3.2**
 237
 238 Regenerating local video thumbnails could be useful because new PeerTube releases may increase thumbnail sizes:
 239
 240 ```bash
 241 # Basic installation
 242 cd /var/www/peertube/peertube-latest
 243 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run regenerate-thumbnails
 244
 245 # Docker installation
 246 cd /var/www/peertube-docker
 247 docker-compose exec -u peertube peertube npm run regenerate-thumbnails
 248 ```
 249
 250 ### create-import-video-file-job.js
 251
 252 You can use this script to import a video file to replace an already uploaded file or to add a new webtorrent resolution to a video. PeerTube needs to be running.
 253 You can then create a transcoding job using the web interface if you need to optimize your file or create an HLS version of it.
 254
 255 ```bash
 256 # Basic installation
 257 cd /var/www/peertube/peertube-latest
 258 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 259
 260 # Docker installation
 261 cd /var/www/peertube-docker
 262 docker-compose exec -u peertube peertube npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
 263 ```
 264
 265 ### create-move-video-storage-job.js
 266
 267 **PeerTube >= 4.0**
 268
 269 Use this script to move all video files or a specific video file to object storage.
 270
 271 ```bash
 272 # Basic installation
 273 cd /var/www/peertube/peertube-latest
 274 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-move-video-storage-job -- --to-object-storage -v [videoUUID]
 275
 276 # Docker installation
 277 cd /var/www/peertube-docker
 278 docker-compose exec -u peertube peertube npm run create-move-video-storage-job -- --to-object-storage -v [videoUUID]
 279 ```
 280
 281 The script can also move all video files that are not already in object storage:
 282
 283 ```bash
 284 # Basic installation
 285 cd /var/www/peertube/peertube-latest
 286 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-move-video-storage-job -- --to-object-storage --all-videos
 287
 288 # Docker installation
 289 cd /var/www/peertube-docker
 290 docker-compose exec -u peertube peertube npm run create-move-video-storage-job -- --to-object-storage --all-videos
 291 ```
 292
 293
 294 ### prune-storage.js
 295
 296 Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
 297 Stop PeerTube and delete these files (a confirmation will be demanded first):
 298
 299 ```bash
 300 cd /var/www/peertube/peertube-latest
 301 sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
 302 ```
 303
 304
 305 ### update-host.js
 306
 307 **Changing the hostname is unsupported and may be a risky operation, especially if you have already federated.**
 308 If you started PeerTube with a domain, and then changed it you will have
 309 invalid torrent files and invalid URLs in your database. To fix this, you have
 310 to run the command below (keep in mind your follower instances will NOT update their URLs).
 311
 312 ```bash
 313 # Basic installation
 314 cd /var/www/peertube/peertube-latest
 315 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run update-host
 316
 317 # Docker installation
 318 cd /var/www/peertube-docker
 319 docker-compose exec -u peertube peertube npm run update-host
 320 ```
 321
 322 ### reset-password.js
 323
 324 To reset a user password from CLI, run:
 325
 326 ```bash
 327 # Basic installation
 328 cd /var/www/peertube/peertube-latest
 329 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run reset-password -- -u target_username
 330
 331 # Docker installation
 332 cd /var/www/peertube-docker
 333 docker-compose exec -u peertube peertube npm run reset-password -- -u target_username
 334 ```
 335
 336
 337 ### plugin install/uninstall
 338
 339 The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
 340 If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).
 341
 342 To install/update a plugin or a theme from the disk:
 343
 344 ```bash
 345 cd /var/www/peertube/peertube-latest
 346 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --plugin-path /local/plugin/path
 347
 348 # Docker installation
 349 cd /var/www/peertube-docker
 350 docker-compose exec -u peertube peertube npm run plugin:install -- --plugin-path /local/plugin/path
 351 ```
 352
 353 From NPM:
 354
 355 ```bash
 356 cd /var/www/peertube/peertube-latest
 357 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --npm-name peertube-plugin-myplugin
 358
 359 # Docker installation
 360 cd /var/www/peertube-docker
 361 docker-compose exec -u peertube peertube npm run plugin:install -- --npm-name peertube-plugin-myplugin
 362 ```
 363
 364 To uninstall a plugin or a theme:
 365
 366 ```bash
 367 cd /var/www/peertube/peertube-latest
 368 sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 369
 370 # Docker installation
 371 cd /var/www/peertube-docker
 372 docker-compose exec -u peertube peertube npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
 373 ```