aboutsummaryrefslogtreecommitdiffhomepage
path: root/support/doc/tools.md
blob: 1956dc572b0c6f38663b7200a05d7de5363b6158 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
# CLI tools guide

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents**

- [Remote Tools](#remote-tools)
  - [Dependencies](#dependencies)
  - [Installation](#installation)
  - [CLI wrapper](#cli-wrapper)
    - [peertube-import-videos.js](#peertube-import-videosjs)
    - [peertube-upload.js](#peertube-uploadjs)
    - [peertube-watch.js](#peertube-watchjs)
    - [peertube-plugins.js](#peertube-pluginsjs)
    - [peertube-redundancy.js](#peertube-redundancyjs)
- [Server tools](#server-tools)
  - [parse-log](#parse-log)
  - [regenerate-thumbnails.js](#regenerate-thumbnailsjs)
  - [create-transcoding-job.js](#create-transcoding-jobjs)
  - [create-import-video-file-job.js](#create-import-video-file-jobjs)
  - [prune-storage.js](#prune-storagejs)
  - [optimize-old-videos.js](#optimize-old-videosjs)
  - [update-host.js](#update-hostjs)
  - [reset-password.js](#reset-passwordjs)
  - [plugin install/uninstall](#plugin-installuninstall)
  - [REPL (Read Eval Print Loop)](#repl-read-eval-print-loop)
    - [.help](#help)
    - [Lodash example](#lodash-example)
    - [YoutubeDL example](#youtubedl-example)
    - [Models examples](#models-examples)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

## Remote Tools

You need at least 512MB RAM to run the script.
Scripts can be launched directly from a PeerTube server, or from a separate server, even a desktop PC.
You need to follow all the following steps even if you are on a PeerTube server (including cloning the git repository in a different directory than your production installation because the scripts utilize non-production dependencies).

### Dependencies

Install the [PeerTube dependencies](dependencies.md) except PostgreSQL and Redis.

### Installation

Clone the PeerTube repo to get the latest version (even if you are on your PeerTube server):

```bash
$ git clone https://github.com/Chocobozzz/PeerTube.git
$ CLONE="$(pwd)/PeerTube"
$ cd ${CLONE}
```

Install dependencies and build CLI tools:

```bash
$ NOCLIENT=1 yarn install --pure-lockfile
$ npm run setup:cli
```

### CLI wrapper

The wrapper provides a convenient interface to the following scripts.
You can access it as `peertube` via an alias in your `.bashrc` like `alias peertube="cd /your/peertube/directory/ && node ./dist/server/tools/peertube.js"` (you have to keep the `cd` command):

```
  Usage: peertube [command] [options]

  Options:

    -v, --version         output the version number
    -h, --help            output usage information

  Commands:

    auth [action]         register your accounts on remote instances to use them with other commands
    upload|up             upload a video
    import-videos|import  import a video from a streaming platform
    watch|w               watch a video in the terminal ✩°。⋆
    repl                  initiate a REPL to access internals
    plugins|p [action]    manage instance plugins
    redundancy|r [action] manage video redundancies
    help [cmd]            display help for [cmd]
```

The wrapper can keep track of instances you have an account on. We limit to one account per instance for now.

```bash
$ peertube auth add -u 'PEERTUBE_URL' -U 'PEERTUBE_USER' --password 'PEERTUBE_PASSWORD'
$ peertube auth list
┌──────────────────────────────┬──────────────────────────────┐
│ instance                      login                        │
├──────────────────────────────┼──────────────────────────────┤
│ 'PEERTUBE_URL'                'PEERTUBE_USER'              │
└──────────────────────────────┴──────────────────────────────┘
```

You can now use that account to upload videos without feeding the same parameters again.

```bash
$ peertube up <videoFile>
```

And now that your video is online, you can watch it from the confort of your terminal (use `peertube watch --help` to see the supported players):

```bash
$ peertube watch https://peertube.cpy.re/videos/watch/e8a1af4e-414a-4d58-bfe6-2146eed06d10
```

To list, install, uninstall dynamically plugins/themes of an instance:

```bash
$ peertube plugins list
$ peertube plugins install --path /local/plugin/path
$ peertube plugins install --npm-name peertube-plugin-myplugin
$ peertube plugins uninstall --npm-name peertube-plugin-myplugin
```

#### peertube-import-videos.js

You can use this script to import videos from all [supported sites of youtube-dl](https://rg3.github.io/youtube-dl/supportedsites.html) into PeerTube.
Be sure you own the videos or have the author's authorization to do so.

```sh
$ node dist/server/tools/peertube-import-videos.js \
    -u 'PEERTUBE_URL' \
    -U 'PEERTUBE_USER' \
    --password 'PEERTUBE_PASSWORD' \
    --target-url 'TARGET_URL'
```

* `PEERTUBE_URL` : the full URL of your PeerTube server where you want to import, eg: https://peertube.cpy.re
* `PEERTUBE_USER` : your PeerTube account where videos will be uploaded
* `PEERTUBE_PASSWORD` : password of your PeerTube account (if `--password PEERTUBE_PASSWORD` is omitted, you will be prompted for it)
* `TARGET_URL` : the target url you want to import. Examples:
  * YouTube:
    * Channel: https://www.youtube.com/channel/ChannelId
    * User https://www.youtube.com/c/UserName or https://www.youtube.com/user/UserName
    * Video https://www.youtube.com/watch?v=blabla
  * Vimeo: https://vimeo.com/xxxxxx
  * Dailymotion: https://www.dailymotion.com/xxxxx

The script will get all public videos from Youtube, download them and upload to PeerTube.
Already downloaded videos will not be uploaded twice, so you can run and re-run the script in case of crash, disconnection...

Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).

**NB**: If you want to synchronize a Youtube channel to your PeerTube instance (ensure you have the agreement from the author),
you can add a [crontab rule](https://help.ubuntu.com/community/CronHowto) (or an equivalent of your OS) and insert
these rules (ensure to customize them to your needs):

```
# Update youtube-dl every day at midnight
0 0 * * * /usr/bin/npm rebuild youtube-dl --prefix /PATH/TO/PEERTUBE/

# Synchronize the YT channel every sunday at 22:00 all the videos published since last monday included
0 22 * * 0 /usr/bin/node /PATH/TO/PEERTUBE/dist/server/tools/peertube-import-videos.js -u '__PEERTUBE_URL__' -U '__USER__' --password '__PASSWORD__' --target-url 'https://www.youtube.com/channel/___CHANNEL__' --since $(date --date="-6 days" +%Y-%m-%d)
```

Also you may want to subscribe to the PeerTube channel in order to manually check the synchronization is successful.

#### peertube-upload.js

You can use this script to import videos directly from the CLI.

Videos will be publicly available after transcoding (you can see them before that in your account on the web interface).

```bash
$ cd ${CLONE}
$ node dist/server/tools/peertube-upload.js --help
```

#### peertube-watch.js

You can use this script to play videos directly from the CLI.

It provides support for different players:

- ascii (default ; plays in ascii art in your terminal!)
- mpv
- mplayer
- vlc
- stdout
- xbmc
- airplay
- chromecast


#### peertube-plugins.js

Install/update/uninstall or list local or NPM PeerTube plugins:

```bash
$ cd ${CLONE}
$ node dist/server/tools/peertube-plugins.js --help
$ node dist/server/tools/peertube-plugins.js list --help
$ node dist/server/tools/peertube-plugins.js install --help
$ node dist/server/tools/peertube-plugins.js update --help
$ node dist/server/tools/peertube-plugins.js uninstall --help

$ node dist/server/tools/peertube-plugins.js install --path /my/plugin/path
$ node dist/server/tools/peertube-plugins.js install --npm-name peertube-theme-example
```

#### peertube-redundancy.js

Manage (list/add/remove) video redundancies:

To list your videos that are duplicated by remote instances:

```bash
$ node dist/server/tools/peertube.js redundancy list-remote-redundancies
```

To list remote videos that your instance duplicated:

```bash
$ node dist/server/tools/peertube.js redundancy list-my-redundancies
```

To duplicate a specific video in your redundancy system:

```bash
$ node dist/server/tools/peertube.js redundancy add --video 823
```

To remove a video redundancy:

```bash
$ node dist/server/tools/peertube.js redundancy remove --video 823
```

## Server tools

These scripts should be run on the server, in `peertube-latest` directory.

### parse-log

To parse PeerTube last log file:

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level info

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run parse-log -- --level info
```

`--level` is optional and could be `info`/`warn`/`error`

You can also remove SQL or HTTP logs using `--not-tags` (PeerTube >= 3.2):

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run parse-log -- --level debug --not-tags http sql

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run parse-log -- --level debug --not-tags http sql
```

### regenerate-thumbnails.js

**PeerTube >= 3.2**

Regenerating local video thumbnails could be useful because new PeerTube releases may increase thumbnail sizes:

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run regenerate-thumbnails

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run regenerate-thumbnails
```

### create-transcoding-job.js

You can use this script to force transcoding of an existing video. PeerTube needs to be running.

To generate transcoding jobs depending on the instance configuration:

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID]

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID]
```

Or to transcode to a specific resolution:

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- -v [videoUUID] -r [resolution]

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run create-transcoding-job -- -v [videoUUID] -r [resolution]
```

The resolution should be an integer (`1080`, `720`, `480`, etc.)

To generate an HLS playlist for a video:

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-transcoding-job -- --generate-hls -v [videoUUID]

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run create-transcoding-job -- --generate-hls -v [videoUUID]
```

### create-import-video-file-job.js

You can use this script to import a video file to replace an already uploaded file or to add a new resolution to a video. PeerTube needs to be running.

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run create-import-video-file-job -- -v [videoUUID] -i [videoFile]
```

### prune-storage.js

Some transcoded videos or shutdown at a bad time can leave some unused files on your storage.
Stop PeerTube and delete these files (a confirmation will be demanded first):

```bash
$ cd /var/www/peertube/peertube-latest
$ sudo systemctl stop peertube && sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run prune-storage
```

### optimize-old-videos.js

Before version v1.0.0-beta.16, Peertube did not specify a bitrate for the
transcoding of uploaded videos. This means that videos might be encoded into
very large files that are too large for streaming. This script re-transcodes
these videos so that they can be watched properly, even on slow connections.

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run optimize-old-videos

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run optimize-old-videos
```


### update-host.js

**Changing the hostname is unsupported and may be a risky operation, especially if you have already federated.**
If you started PeerTube with a domain, and then changed it you will have
invalid torrent files and invalid URLs in your database. To fix this, you have
to run the command below (keep in mind your follower instances will NOT update their URLs).

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run update-host

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run update-host
```

### reset-password.js

To reset a user password from CLI, run:

```bash
$ # Basic installation
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run reset-password -- -u target_username

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run reset-password -- -u target_username
```


### plugin install/uninstall

The difference with `peertube plugins` CLI is that these scripts can be used even if PeerTube is not running.
If PeerTube is running, you need to restart it for the changes to take effect (whereas with `peertube plugins` CLI, plugins/themes are dynamically loaded on the server).

To install/update a plugin or a theme from the disk:

```bash
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --plugin-path /local/plugin/path

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run plugin:install -- --plugin-path /local/plugin/path
```

From NPM:

```bash
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:install -- --npm-name peertube-plugin-myplugin

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run plugin:install -- --npm-name peertube-plugin-myplugin
```

To uninstall a plugin or a theme:

```bash
$ cd /var/www/peertube/peertube-latest
$ sudo -u peertube NODE_CONFIG_DIR=/var/www/peertube/config NODE_ENV=production npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin

$ # Docker installation
$ cd /var/www/peertube-docker
$ docker-compose exec -u peertube peertube npm run plugin:uninstall -- --npm-name peertube-plugin-myplugin
```

### REPL (Read Eval Print Loop)

If you want to interact with the application libraries and objects even when PeerTube is not running, there is a REPL for that.

usage: `node ./dist/server/tools/peertube-repl.js`

"The default evaluator will, by default, assign the result of the most recently evaluated expression to the special variable `_` (underscore). Explicitly setting `_` to a value will disable this behavior."

- type `.help` to list commands available in the repl, notice it starts with a dot
- type `.exit` to exit, note that you still have to press CTRL-C to actually exit, or press CTRL-C (3 times) without typing `.exit` to exit
- type `context` to list all available objects and libraries in the context, note: `Promise` is also available but it's not listed in the context, in case you need promises for something
- type `env` to see the loaded environment variables
- type `path` to access path library
- type `lodash` to access lodash library
- type `uuidv1` to access uuid/v1 library
- type `uuidv3` to access uuid/v3 library
- type `uuidv4` to access uuid/v4 library
- type `uuidv5` to access uuid/v5 library
- type `YoutubeDL` to access youtube-dl library
- type `cli` to access the cli helpers object
- type `logger` to access the logger; if you log to it, it will write to stdout and to the peertube.log file
- type `constants` to access the constants loaded by the server
- type `coreUtils` to access the core-utils helpers object
- type `ffmpegUtils` to access the ffmpeg-utils helpers object
- type `peertubeCryptoUtils` to access the peertube-crypto helpers object
- type `signupUtils` to access the signup helpers object
- type `utils` to access the utils helpers object
- type `YoutubeDLUtils` to access the youtube-dl helpers object
- type `sequelizeTypescript` to access sequelizeTypescript
- type `modelsUtils` to access the models/utils
- type `models` to access the shortcut to sequelizeTypescript.models
- type `transaction` to access the shortcut to sequelizeTypescript.transaction
- type `query` to access the shortcut to sequelizeTypescript.query
- type `queryInterface` to access the shortcut to sequelizeTypescript.queryInterface

#### .help

```
PeerTube [1.0.0] (b10eb595)> .help
.break    Sometimes you get stuck, this gets you out
.clear    Break, and also clear the local context
.editor   Enter editor mode
.exit     Exit the repl
.help     Print this help message
.load     Load JS from a file into the REPL session
.r        Reset REPL
.reset    Reset REPL
.save     Save all evaluated commands in this REPL session to a file
PeerTube [1.0.0] (b10eb595)>
```

#### Lodash example

```
PeerTube [1.0.0] (b10eb595)> lodash.keys(context)
[ 'global',
  'console',
  'DTRACE_NET_SERVER_CONNECTION',
  'DTRACE_NET_STREAM_END',
  'DTRACE_HTTP_SERVER_REQUEST',
  'DTRACE_HTTP_SERVER_RESPONSE',
  'DTRACE_HTTP_CLIENT_REQUEST',
  'DTRACE_HTTP_CLIENT_RESPONSE',
  'process',
  'Buffer',
  'clearImmediate',
  'clearInterval',
  'clearTimeout',
  'setImmediate',
  'setInterval',
  'setTimeout',
  'XMLHttpRequest',
  'compact2string',
  'module',
  'require',
  'path',
  'repl',
  'context',
  'env',
  'lodash',
  'uuidv1',
  'uuidv3',
  'uuidv4',
  'uuidv5',
  'cli',
  'logger',
  'constants',
  'Sequelize',
  'sequelizeTypescript',
  'modelsUtils',
  'models',
  'transaction',
  'query',
  'queryInterface',
  'YoutubeDL',
  'coreUtils',
  'ffmpegUtils',
  'peertubeCryptoUtils',
  'signupUtils',
  'utils',
  'YoutubeDLUtils' ]
PeerTube [1.0.0] (b10eb595)>
```

#### YoutubeDL example
```
YoutubeDL.getInfo('https://www.youtube.com/watch?v=I5ZN289jjDo', function(err, data) {console.log(err, data)})
```

#### Models examples
```
PeerTube [1.0.0] (b10eb595)> new models.ActorModel({id: 3}).getVideoChannel().then(function(data){console.log(data.dataValues.name)})
Promise {
  _bitField: 0,
  _fulfillmentHandler0: undefined,
  _rejectionHandler0: undefined,
  _promise0: undefined,
  _receiver0: undefined }
PeerTube [1.0.0] (b10eb595)> Main root channel
PeerTube [1.0.0] (b10eb595)> let out; new models.UserModel({id: 1}).getAccount().then(function (data) {out = data.dataValues.id})
Promise {
  _bitField: 0,
  _fulfillmentHandler0: undefined,
  _rejectionHandler0: undefined,
  _promise0: undefined,
  _receiver0: undefined }
PeerTube [1.0.0] (b10eb595)> out
2
PeerTube [1.0.0] (b10eb595)>
```