aboutsummaryrefslogtreecommitdiffhomepage
path: root/support/doc
diff options
context:
space:
mode:
authorkontrollanten <6680299+kontrollanten@users.noreply.github.com>2021-05-10 11:13:41 +0200
committerGitHub <noreply@github.com>2021-05-10 11:13:41 +0200
commitf6d6e7f861189a4446f406efb775a29688764b48 (patch)
treec3dda9958c3f189d4c39e8743c738d8c1fef4c2d /support/doc
parentd29ced1a8582d99b776f664475a157adcf555d98 (diff)
downloadPeerTube-f6d6e7f861189a4446f406efb775a29688764b48.tar.gz
PeerTube-f6d6e7f861189a4446f406efb775a29688764b48.tar.zst
PeerTube-f6d6e7f861189a4446f406efb775a29688764b48.zip
Resumable video uploads (#3933)
* WIP: resumable video uploads relates to #324 * fix review comments * video upload: error handling * fix audio upload * fixes after self review * Update server/controllers/api/videos/index.ts Co-authored-by: Rigel Kent <par@rigelk.eu> * Update server/middlewares/validators/videos/videos.ts Co-authored-by: Rigel Kent <par@rigelk.eu> * Update server/controllers/api/videos/index.ts Co-authored-by: Rigel Kent <par@rigelk.eu> * update after code review * refactor upload route - restore multipart upload route - move resumable to dedicated upload-resumable route - move checks to middleware - do not leak internal fs structure in response * fix yarn.lock upon rebase * factorize addVideo for reuse in both endpoints * add resumable upload API to openapi spec * add initial test and test helper for resumable upload * typings for videoAddResumable middleware * avoid including aws and google packages via node-uploadx, by only including uploadx/core * rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio * add video-upload-tmp-folder-cleaner job * stronger typing of video upload middleware * reduce dependency to @uploadx/core * add audio upload test * refactor resumable uploads cleanup from job to scheduler * refactor resumable uploads scheduler to compare to last execution time * make resumable upload validator to always cleanup on failure * move legacy upload request building outside of uploadVideo test helper * filter upload-resumable middlewares down to POST, PUT, DELETE also begin to type metadata * merge add duration functions * stronger typings and documentation for uploadx behaviour, move init validator up * refactor(client/video-edit): options > uploadxOptions * refactor(client/video-edit): remove obsolete else * scheduler/remove-dangling-resum: rename tag * refactor(server/video): add UploadVideoFiles type * refactor(mw/validators): restructure eslint disable * refactor(mw/validators/videos): rename import * refactor(client/vid-upload): rename html elem id * refactor(sched/remove-dangl): move fn to method * refactor(mw/async): add method typing * refactor(mw/vali/video): double quote > single * refactor(server/upload-resum): express use > all * proper http methud enum server/middlewares/async.ts * properly type http methods * factorize common video upload validation steps * add check for maximum partially uploaded file size * fix audioBg use * fix extname(filename) in addVideo * document parameters for uploadx's resumable protocol * clear META files in scheduler * last audio refactor before cramming preview in the initial POST form data * refactor as mulitpart/form-data initial post request this allows preview/thumbnail uploads alongside the initial request, and cleans up the upload form * Add more tests for resumable uploads * Refactor remove dangling resumable uploads * Prepare changelog * Add more resumable upload tests * Remove user quota check for resumable uploads * Fix upload error handler * Update nginx template for upload-resumable * Cleanup comment * Remove unused express methods * Prefer to use got instead of raw http * Don't retry on error 500 Co-authored-by: Rigel Kent <par@rigelk.eu> Co-authored-by: Rigel Kent <sendmemail@rigelk.eu> Co-authored-by: Chocobozzz <me@florianbigard.com>
Diffstat (limited to 'support/doc')
-rw-r--r--support/doc/api/openapi.yaml414
1 files changed, 315 insertions, 99 deletions
diff --git a/support/doc/api/openapi.yaml b/support/doc/api/openapi.yaml
index 90e30545f..050ab82f8 100644
--- a/support/doc/api/openapi.yaml
+++ b/support/doc/api/openapi.yaml
@@ -125,7 +125,7 @@ tags:
125 Redundancy is part of the inter-server solidarity that PeerTube fosters. 125 Redundancy is part of the inter-server solidarity that PeerTube fosters.
126 Manage the list of instances you wish to help by seeding their videos according 126 Manage the list of instances you wish to help by seeding their videos according
127 to the policy of video selection of your choice. Note that you have a similar functionality 127 to the policy of video selection of your choice. Note that you have a similar functionality
128 to mirror individual videos, see `Video Mirroring`. 128 to mirror individual videos, see [video mirroring](#tag/Video-Mirroring).
129 externalDocs: 129 externalDocs:
130 url: https://docs.joinpeertube.org/admin-following-instances?id=instances-redundancy 130 url: https://docs.joinpeertube.org/admin-following-instances?id=instances-redundancy
131 - name: Plugins 131 - name: Plugins
@@ -139,6 +139,50 @@ tags:
139 - name: Video 139 - name: Video
140 description: | 140 description: |
141 Operations dealing with listing, uploading, fetching or modifying videos. 141 Operations dealing with listing, uploading, fetching or modifying videos.
142 - name: Video Upload
143 description: |
144 Operations dealing with adding video or audio. PeerTube supports two upload modes, and three import modes.
145
146 ### Upload
147
148 - [_legacy_](#tag/Video-Upload/paths/~1videos~1upload/post), where the video file is sent in a single request
149 - [_resumable_](#tag/Video-Upload/paths/~1videos~1upload-resumable/post), where the video file is sent in chunks
150
151 You can upload videos more reliably by using the resumable variant. Its protocol lets
152 you resume an upload operation after a network interruption or other transmission failure,
153 saving time and bandwidth in the event of network failures.
154
155 Favor using resumable uploads in any of the following cases:
156 - You are transferring large files
157 - The likelihood of a network interruption is high
158 - Uploads are originating from a device with a low-bandwidth or unstable Internet connection,
159 such as a mobile device
160
161 ### Import
162
163 - _URL_-based: where the URL points to any service supported by [youtube-dl](https://ytdl-org.github.io/youtube-dl/)
164 - _magnet_-based: where the URI resolves to a BitTorrent ressource containing a single supported video file
165 - _torrent_-based: where the metainfo file resolves to a BitTorrent ressource containing a single supported video file
166
167 The import function is practical when the desired video/audio is available online. It makes PeerTube
168 download it for you, saving you as much bandwidth and avoiding any instability or limitation your network might have.
169 - name: Video Captions
170 description: Operations dealing with listing, adding and removing closed captions of a video.
171 - name: Video Channels
172 description: Operations dealing with the creation, modification and listing of videos within a channel.
173 - name: Video Comments
174 description: >
175 Operations dealing with comments to a video. Comments are organized in threads: adding a
176 comment in response to the video starts a thread, adding a reply to a comment adds it to
177 its root comment thread.
178 - name: Video Blocks
179 description: Operations dealing with blocking videos (removing them from view and preventing interactions).
180 - name: Video Rates
181 description: Like/dislike a video.
182 - name: Video Playlists
183 description: Operations dealing with playlists of videos. Playlists are bound to users and/or channels.
184 - name: Feeds
185 description: Server syndication feeds
142 - name: Search 186 - name: Search
143 description: | 187 description: |
144 The search helps to find _videos_ or _channels_ from within the instance and beyond. 188 The search helps to find _videos_ or _channels_ from within the instance and beyond.
@@ -148,27 +192,11 @@ tags:
148 192
149 Administrators can also enable the use of a remote search system, indexing 193 Administrators can also enable the use of a remote search system, indexing
150 videos and channels not could be not federated by the instance. 194 videos and channels not could be not federated by the instance.
151 - name: Video Comments 195 - name: Video Mirroring
152 description: > 196 description: |
153 Operations dealing with comments to a video. Comments are organized in 197 PeerTube instances can mirror videos from one another, and help distribute some videos.
154 threads. 198
155 - name: Video Playlists 199 For importing videos as your own, refer to [video imports](#tag/Video-Upload/paths/~1videos~1imports/post).
156 description: >
157 Operations dealing with playlists of videos. Playlists are bound to users
158 and/or channels.
159 - name: Video Channels
160 description: >
161 Operations dealing with the creation, modification and listing of videos within a channel.
162 - name: Video Blocks
163 description: >
164 Operations dealing with blocking videos (removing them from view and
165 preventing interactions).
166 - name: Video Rates
167 description: >
168 Like/dislike a video.
169 - name: Feeds
170 description: >
171 Server syndication feeds
172x-tagGroups: 200x-tagGroups:
173 - name: Accounts 201 - name: Accounts
174 tags: 202 tags:
@@ -181,6 +209,7 @@ x-tagGroups:
181 - name: Videos 209 - name: Videos
182 tags: 210 tags:
183 - Video 211 - Video
212 - Video Upload
184 - Video Captions 213 - Video Captions
185 - Video Channels 214 - Video Channels
186 - Video Comments 215 - Video Comments
@@ -1347,10 +1376,12 @@ paths:
1347 /videos/upload: 1376 /videos/upload:
1348 post: 1377 post:
1349 summary: Upload a video 1378 summary: Upload a video
1379 description: Uses a single request to upload a video.
1350 security: 1380 security:
1351 - OAuth2: [] 1381 - OAuth2: []
1352 tags: 1382 tags:
1353 - Video 1383 - Video
1384 - Video Upload
1354 responses: 1385 responses:
1355 '200': 1386 '200':
1356 description: successful operation 1387 description: successful operation
@@ -1380,80 +1411,7 @@ paths:
1380 content: 1411 content:
1381 multipart/form-data: 1412 multipart/form-data:
1382 schema: 1413 schema:
1383 type: object 1414 $ref: '#/components/schemas/VideoUploadRequestLegacy'
1384 properties:
1385 videofile:
1386 description: Video file
1387 type: string
1388 format: binary
1389 channelId:
1390 description: Channel id that will contain this video
1391 type: integer
1392 thumbnailfile:
1393 description: Video thumbnail file
1394 type: string
1395 format: binary
1396 previewfile:
1397 description: Video preview file
1398 type: string
1399 format: binary
1400 privacy:
1401 $ref: '#/components/schemas/VideoPrivacySet'
1402 category:
1403 description: Video category
1404 type: integer
1405 example: 4
1406 licence:
1407 description: Video licence
1408 type: integer
1409 example: 2
1410 language:
1411 description: Video language
1412 type: string
1413 description:
1414 description: Video description
1415 type: string
1416 waitTranscoding:
1417 description: Whether or not we wait transcoding before publish the video
1418 type: boolean
1419 support:
1420 description: A text tell the audience how to support the video creator
1421 example: Please support my work on <insert crowdfunding plateform>! <3
1422 type: string
1423 nsfw:
1424 description: Whether or not this video contains sensitive content
1425 type: boolean
1426 name:
1427 description: Video name
1428 type: string
1429 minLength: 3
1430 maxLength: 120
1431 tags:
1432 description: Video tags (maximum 5 tags each between 2 and 30 characters)
1433 type: array
1434 minItems: 1
1435 maxItems: 5
1436 uniqueItems: true
1437 items:
1438 type: string
1439 minLength: 2
1440 maxLength: 30
1441 commentsEnabled:
1442 description: Enable or disable comments for this video
1443 type: boolean
1444 downloadEnabled:
1445 description: Enable or disable downloading for this video
1446 type: boolean
1447 originallyPublishedAt:
1448 description: Date when the content was originally published
1449 type: string
1450 format: date-time
1451 scheduleUpdate:
1452 $ref: '#/components/schemas/VideoScheduledUpdate'
1453 required:
1454 - videofile
1455 - channelId
1456 - name
1457 encoding: 1415 encoding:
1458 videofile: 1416 videofile:
1459 contentType: video/mp4, video/webm, video/ogg, video/avi, video/quicktime, video/x-msvideo, video/x-flv, video/x-matroska, application/octet-stream 1417 contentType: video/mp4, video/webm, video/ogg, video/avi, video/quicktime, video/x-msvideo, video/x-flv, video/x-matroska, application/octet-stream
@@ -1490,6 +1448,164 @@ paths:
1490 --form videofile=@"$FILE_PATH" \ 1448 --form videofile=@"$FILE_PATH" \
1491 --form channelId=$CHANNEL_ID \ 1449 --form channelId=$CHANNEL_ID \
1492 --form name="$NAME" 1450 --form name="$NAME"
1451 /videos/upload-resumable:
1452 post:
1453 summary: Initialize the resumable upload of a video
1454 description: Uses [a resumable protocol](https://github.com/kukhariev/node-uploadx/blob/master/proto.md) to initialize the upload of a video
1455 security:
1456 - OAuth2: []
1457 tags:
1458 - Video
1459 - Video Upload
1460 parameters:
1461 - name: X-Upload-Content-Length
1462 in: header
1463 schema:
1464 type: number
1465 example: 2469036
1466 required: true
1467 description: Number of bytes that will be uploaded in subsequent requests. Set this value to the size of the file you are uploading.
1468 - name: X-Upload-Content-Type
1469 in: header
1470 schema:
1471 type: string
1472 format: mimetype
1473 example: video/mp4
1474 required: true
1475 description: MIME type of the file that you are uploading. Depending on your instance settings, acceptable values might vary.
1476 requestBody:
1477 content:
1478 application/json:
1479 schema:
1480 $ref: '#/components/schemas/VideoUploadRequestResumable'
1481 responses:
1482 '200':
1483 description: file already exists, send a [`resume`](https://github.com/kukhariev/node-uploadx/blob/master/proto.md) request instead
1484 '201':
1485 description: created
1486 headers:
1487 Location:
1488 schema:
1489 type: string
1490 format: url
1491 example: /api/v1/videos/upload-resumable?upload_id=471e97554f21dec3b8bb5d4602939c51
1492 Content-Length:
1493 schema:
1494 type: number
1495 example: 0
1496 '400':
1497 description: invalid file field, schedule date or parameter
1498 '413':
1499 description: video file too large, due to quota, absolute max file size or concurrent partial upload limit
1500 '415':
1501 description: video type unsupported
1502 put:
1503 summary: Send chunk for the resumable upload of a video
1504 description: Uses [a resumable protocol](https://github.com/kukhariev/node-uploadx/blob/master/proto.md) to continue, pause or resume the upload of a video
1505 security:
1506 - OAuth2: []
1507 tags:
1508 - Video
1509 - Video Upload
1510 parameters:
1511 - name: upload_id
1512 in: path
1513 required: true
1514 description: |
1515 Created session id to proceed with. If you didn't send chunks in the last 12 hours, it is
1516 not valid anymore and you need to initialize a new upload.
1517 schema:
1518 type: string
1519 - name: Content-Range
1520 in: header
1521 schema:
1522 type: string
1523 example: bytes 0-262143/2469036
1524 required: true
1525 description: |
1526 Specifies the bytes in the file that the request is uploading.
1527
1528 For example, a value of `bytes 0-262143/1000000` shows that the request is sending the first
1529 262144 bytes (256 x 1024) in a 2,469,036 byte file.
1530 - name: Content-Length
1531 in: header
1532 schema:
1533 type: number
1534 example: 262144
1535 required: true
1536 description: |
1537 Size of the chunk that the request is sending.
1538
1539 The chunk size __must be a multiple of 256 KB__, and unlike [Google Resumable](https://developers.google.com/youtube/v3/guides/using_resumable_upload_protocol)
1540 doesn't mandate for chunks to have the same size throughout the upload sequence.
1541
1542 Remember that larger chunks are more efficient. PeerTube's web client uses chunks varying from
1543 1048576 bytes (~1MB) and increases or reduces size depending on connection health.
1544 requestBody:
1545 content:
1546 application/octet-stream:
1547 schema:
1548 type: string
1549 format: binary
1550 responses:
1551 '200':
1552 description: last chunk received
1553 headers:
1554 Content-Length:
1555 schema:
1556 type: number
1557 content:
1558 application/json:
1559 schema:
1560 $ref: '#/components/schemas/VideoUploadResponse'
1561 '308':
1562 description: resume incomplete
1563 headers:
1564 Range:
1565 schema:
1566 type: string
1567 example: bytes=0-262143
1568 Content-Length:
1569 schema:
1570 type: number
1571 example: 0
1572 '403':
1573 description: video didn't pass upload filter
1574 '413':
1575 description: video file too large, due to quota or max body size limit set by the reverse-proxy
1576 '422':
1577 description: video unreadable
1578 delete:
1579 summary: Cancel the resumable upload of a video, deleting any data uploaded so far
1580 description: Uses [a resumable protocol](https://github.com/kukhariev/node-uploadx/blob/master/proto.md) to cancel the upload of a video
1581 security:
1582 - OAuth2: []
1583 tags:
1584 - Video
1585 - Video Upload
1586 parameters:
1587 - name: upload_id
1588 in: path
1589 required: true
1590 description: |
1591 Created session id to proceed with. If you didn't send chunks in the last 12 hours, it is
1592 not valid anymore and the upload session has already been deleted with its data ;-)
1593 schema:
1594 type: string
1595 - name: Content-Length
1596 in: header
1597 required: true
1598 schema:
1599 type: number
1600 example: 0
1601 responses:
1602 '204':
1603 description: upload cancelled
1604 headers:
1605 Content-Length:
1606 schema:
1607 type: number
1608 example: 0
1493 /videos/imports: 1609 /videos/imports:
1494 post: 1610 post:
1495 summary: Import a video 1611 summary: Import a video
@@ -1498,6 +1614,7 @@ paths:
1498 - OAuth2: [] 1614 - OAuth2: []
1499 tags: 1615 tags:
1500 - Video 1616 - Video
1617 - Video Upload
1501 requestBody: 1618 requestBody:
1502 content: 1619 content:
1503 multipart/form-data: 1620 multipart/form-data:
@@ -1688,7 +1805,7 @@ paths:
1688 1805
1689 /videos/live/{id}: 1806 /videos/live/{id}:
1690 get: 1807 get:
1691 summary: Get a live information 1808 summary: Get information about a live
1692 security: 1809 security:
1693 - OAuth2: [] 1810 - OAuth2: []
1694 tags: 1811 tags:
@@ -1704,7 +1821,7 @@ paths:
1704 schema: 1821 schema:
1705 $ref: '#/components/schemas/LiveVideoResponse' 1822 $ref: '#/components/schemas/LiveVideoResponse'
1706 put: 1823 put:
1707 summary: Update a live information 1824 summary: Update information about a live
1708 security: 1825 security:
1709 - OAuth2: [] 1826 - OAuth2: []
1710 tags: 1827 tags:
@@ -3940,6 +4057,7 @@ components:
3940 oneOf: 4057 oneOf:
3941 - type: string 4058 - type: string
3942 - type: array 4059 - type: array
4060 maxItems: 5
3943 items: 4061 items:
3944 type: string 4062 type: string
3945 style: form 4063 style: form
@@ -4636,7 +4754,7 @@ components:
4636 message: 4754 message:
4637 type: string 4755 type: string
4638 minLength: 2 4756 minLength: 2
4639 maxLength: 3000 4757 maxLength: 3000
4640 byModerator: 4758 byModerator:
4641 type: boolean 4759 type: boolean
4642 createdAt: 4760 createdAt:
@@ -5229,6 +5347,7 @@ components:
5229 PredefinedAbuseReasons: 5347 PredefinedAbuseReasons:
5230 description: Reason categories that help triage reports 5348 description: Reason categories that help triage reports
5231 type: array 5349 type: array
5350 maxItems: 8
5232 items: 5351 items:
5233 type: string 5352 type: string
5234 enum: 5353 enum:
@@ -5298,6 +5417,103 @@ components:
5298 id: 5417 id:
5299 type: integer 5418 type: integer
5300 example: 37 5419 example: 37
5420 VideoUploadRequestCommon:
5421 properties:
5422 name:
5423 description: Video name
5424 type: string
5425 channelId:
5426 description: Channel id that will contain this video
5427 type: integer
5428 privacy:
5429 $ref: '#/components/schemas/VideoPrivacySet'
5430 category:
5431 description: Video category
5432 type: integer
5433 example: 4
5434 licence:
5435 description: Video licence
5436 type: integer
5437 example: 2
5438 language:
5439 description: Video language
5440 type: string
5441 description:
5442 description: Video description
5443 type: string
5444 waitTranscoding:
5445 description: Whether or not we wait transcoding before publish the video
5446 type: boolean
5447 support:
5448 description: A text tell the audience how to support the video creator
5449 example: Please support my work on <insert crowdfunding plateform>! <3
5450 type: string
5451 nsfw:
5452 description: Whether or not this video contains sensitive content
5453 type: boolean
5454 tags:
5455 description: Video tags (maximum 5 tags each between 2 and 30 characters)
5456 type: array
5457 minItems: 1
5458 maxItems: 5
5459 uniqueItems: true
5460 items:
5461 type: string
5462 minLength: 2
5463 maxLength: 30
5464 commentsEnabled:
5465 description: Enable or disable comments for this video
5466 type: boolean
5467 downloadEnabled:
5468 description: Enable or disable downloading for this video
5469 type: boolean
5470 originallyPublishedAt:
5471 description: Date when the content was originally published
5472 type: string
5473 format: date-time
5474 scheduleUpdate:
5475 $ref: '#/components/schemas/VideoScheduledUpdate'
5476 thumbnailfile:
5477 description: Video thumbnail file
5478 type: string
5479 format: binary
5480 previewfile:
5481 description: Video preview file
5482 type: string
5483 format: binary
5484 required:
5485 - channelId
5486 - name
5487 VideoUploadRequestLegacy:
5488 allOf:
5489 - $ref: '#/components/schemas/VideoUploadRequestCommon'
5490 - type: object
5491 required:
5492 - videofile
5493 properties:
5494 videofile:
5495 description: Video file
5496 type: string
5497 format: binary
5498 VideoUploadRequestResumable:
5499 allOf:
5500 - $ref: '#/components/schemas/VideoUploadRequestCommon'
5501 - type: object
5502 required:
5503 - filename
5504 properties:
5505 filename:
5506 description: Video filename including extension
5507 type: string
5508 format: filename
5509 thumbnailfile:
5510 description: Video thumbnail file
5511 type: string
5512 format: binary
5513 previewfile:
5514 description: Video preview file
5515 type: string
5516 format: binary
5301 VideoUploadResponse: 5517 VideoUploadResponse:
5302 properties: 5518 properties:
5303 video: 5519 video: