aboutsummaryrefslogtreecommitdiffhomepage
path: root/server/lib/schedulers/videos-redundancy-scheduler.ts
Commit message (Collapse)AuthorAgeFilesLines
* More robust webtorrent redundancy downloadChocobozzz2021-09-061-6/+2
| | | | | Avoid issues with inconsistencies between magnet infohash and torrent infohash, blocking webtorrent upload that will timeout
* More logs for webtorrent downloadChocobozzz2021-08-261-4/+4
|
* Better logs for redundancyChocobozzz2021-08-261-19/+35
|
* Add support for saving video files to object storage (#4290)Jelle Besseling2021-08-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add support for saving video files to object storage * Add support for custom url generation on s3 stored files Uses two config keys to support url generation that doesn't directly go to (compatible s3). Can be used to generate urls to any cache server or CDN. * Upload files to s3 concurrently and delete originals afterwards * Only publish after move to object storage is complete * Use base url instead of url template * Fix mistyped config field * Add rudenmentary way to download before transcode * Implement Chocobozzz suggestions https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478 The remarks in question: Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET) I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs) https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs * Import correct function * Support multipart upload * Remove import of node 15.0 module stream/promises * Extend maximum upload job length Using the same value as for redundancy downloading seems logical * Use dynamic part size for really large uploads Also adds very small part size for local testing * Fix decreasePendingMove query * Resolve various PR comments * Move to object storage after optimize * Make upload size configurable and increase default * Prune webtorrent files that are stored in object storage * Move files after transcoding jobs * Fix federation * Add video path manager * Support move to external storage job in client * Fix live object storage tests Co-authored-by: Chocobozzz <me@florianbigard.com>
* Correctly handle large HLS files for redundancyChocobozzz2021-08-061-8/+7
|
* Use random names for VOD HLS playlistsChocobozzz2021-07-261-3/+4
|
* Refactor getOrCreateAPVideoChocobozzz2021-06-021-2/+2
|
* Fix redundancy max sizeChocobozzz2021-05-261-2/+2
|
* Don't guess remote tracker URLChocobozzz2021-02-181-2/+3
|
* Dissociate video file names and video uuidChocobozzz2021-02-181-6/+6
|
* Purge entire video from redundancyChocobozzz2021-02-021-1/+9
|
* Fix incorrect IDs in AP federationChocobozzz2020-11-201-18/+19
|
* Split types and typingsChocobozzz2020-06-181-1/+1
|
* Avoir some circular dependenciesChocobozzz2020-05-041-2/+2
|
* Move to eslintcontainChocobozzz2020-02-031-3/+3
|
* Add ability to list redundanciesChocobozzz2020-01-281-13/+46
|
* Don't always replace actor avatarChocobozzz2019-11-251-6/+9
|
* Add ability to disable webtorrentChocobozzz2019-11-251-3/+4
| | | | In favour of HLS
* Merge branch 'release/1.4.0' into developChocobozzz2019-09-051-1/+1
|\
| * Fix bug in redundancy evictionChocobozzz2019-09-041-1/+1
| |
* | Fix redundancy too heavy with streaming playlistsChocobozzz2019-08-301-2/+1
| |
* | Merge branch 'release/1.4.0' into developChocobozzz2019-08-221-1/+1
|\|
| * Try to fix video duplicationChocobozzz2019-08-221-1/+1
| |
* | Cleanup model typesChocobozzz2019-08-201-1/+3
| |
* | Stronger model typingsChocobozzz2019-08-191-17/+35
|/
* Fix redundancy exceeding the limitChocobozzz2019-08-121-1/+4
|
* Fix redundancy exceeded quotaChocobozzz2019-06-201-1/+4
|
* Don't expose constants directly in initializers/Chocobozzz2019-04-111-1/+1
|
* Move config in its own fileChocobozzz2019-04-111-3/+4
|
* Add hls support on serverChocobozzz2019-02-111-59/+130
|
* Optimize actor follow scores modificationsChocobozzz2018-12-201-8/+1
|
* Use move instead renameChocobozzz2018-12-111-2/+2
| | | | To avoid EXDEV errors
* Create redundancy endpointChocobozzz2018-12-041-2/+2
|
* Fix server redundancy testsChocobozzz2018-11-151-2/+3
|
* Check video exists before extending its expirationChocobozzz2018-10-031-18/+32
|
* Try to improve redundancy testsChocobozzz2018-10-021-0/+2
|
* Fix redundancy with specific videosChocobozzz2018-10-011-1/+1
|
* Add timeout on youtube dl to cleaup filesChocobozzz2018-10-011-3/+2
|
* Ensure video existence before duplicating itChocobozzz2018-10-011-4/+21
|
* Fix redundancy with videos already duplicated with another instanceChocobozzz2018-10-011-1/+1
|
* Fix redundancy totalVideos statsChocobozzz2018-09-251-1/+0
|
* Improve redundancy: add 'min_lifetime' configurationChocobozzz2018-09-241-27/+59
|
* Remove unnecessary actor existance checkChocobozzz2018-09-211-0/+2
| | | | We already checked it in the signature checker
* Add redundancy check interval in configChocobozzz2018-09-191-7/+11
|
* Put config redundancy strategies in "strategies" subkeyChocobozzz2018-09-191-4/+2
|
* Speed up overviews routeChocobozzz2018-09-141-1/+1
|
* Add recently added redundancy strategyChocobozzz2018-09-141-27/+17
|
* Add trending videos strategyChocobozzz2018-09-141-0/+2
|
* Basic video redundancy implementationChocobozzz2018-09-131-0/+161