aboutsummaryrefslogtreecommitdiffhomepage
path: root/ARCHITECTURE.md
diff options
context:
space:
mode:
authorChocobozzz <florian.bigard@gmail.com>2016-07-21 11:05:50 +0200
committerChocobozzz <florian.bigard@gmail.com>2016-07-21 11:05:50 +0200
commit164283f59c0776c2e273b4583316b7c2bcbfc9c3 (patch)
tree469f1e10f39694a768d8589a0ac0b4a81f142d9a /ARCHITECTURE.md
parent1ca81df4d8e9b4769345690e43154aeb7b6dd335 (diff)
downloadPeerTube-164283f59c0776c2e273b4583316b7c2bcbfc9c3.tar.gz
PeerTube-164283f59c0776c2e273b4583316b7c2bcbfc9c3.tar.zst
PeerTube-164283f59c0776c2e273b4583316b7c2bcbfc9c3.zip
Update Architecture document
Diffstat (limited to 'ARCHITECTURE.md')
-rw-r--r--ARCHITECTURE.md74
1 files changed, 38 insertions, 36 deletions
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 6059f4b77..65dd5b49d 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -1,21 +1,22 @@
1# Protocol (WIP, help wanted) 1# Protocol (WIP, help welcome)
2 2
3## Vocabulary 3## Vocabulary
4 4
5 - Network: several servers communicating each others with this software compose a network 5 - **Network:** several servers communicating each others with this software compose a network
6 - Pod: a server of the network (inspired from Diaspora, no really signification) 6 - **Pod:** a server of the network (inspired from Diaspora, no really signification)
7 - Friend: a pod that communicates with yours 7 - **Friend:** a pod that communicates with yours
8 - Origin pod: the pod on which the video was uploaded and which is seeding the video 8 - **Origin pod:** the pod on which the video was uploaded and which is seeding the video
9 - Make friend: the action of a server which will join a network (and so become friend with all pods that compose this network) 9 - **Make friend:** the action of a server which will join a network (and so become friend with all pods that compose this network)
10 10
11## Base 11## Base
12 12
13### The first run: join a network and make friends 13### The first run: join a network and make friends
14 * It will generate a RSA key 14 * The server generates a RSA key
15 * It will join other networks by checking the configuration file 15 * If the server administrator wants to join a network, he just has to make an http request to the API
16 * The server joins a network by checking entrypoints (server urls of the targeted network) in the configuration file
16 * If the config file doesn't specify other pods, the network will be composed by this only pod 17 * If the config file doesn't specify other pods, the network will be composed by this only pod
17 * If the config file specifies one or many pods, the server will ask them the list of the pods in the network. 18 * If the config file specifies one or more pods, the server will ask them the list of the pods in the network.
18 The server will add in its friends list all pods that are in > 50% of all other pods friends list and the pods that are asked for the list. For example if there are the following pods in a network with their following friends list: 19 The server will add in its friends list all pods that are in > 50% of all other pods friends list + the pods that are asked for the list. For example if there are the following pods in a network with their following friends list:
19 20
20 http://pod1.com 21 http://pod1.com
21 - http://pod2.com 22 - http://pod2.com
@@ -31,46 +32,47 @@
31 - http://pod5.com 32 - http://pod5.com
32 33
33 It will add: `http://pod1.com`, `http://pod2.com` and `http://pod3.com` because it asks to them the list of their friends. Then, it will add all pods that are in >= 50% of pods friends list so: `http://pod5.com`. 34 It will add: `http://pod1.com`, `http://pod2.com` and `http://pod3.com` because it asks to them the list of their friends. Then, it will add all pods that are in >= 50% of pods friends list so: `http://pod5.com`.
34 * The friend-making operation is mandatory and irreversible (we can't change the network for example, even if you are the only one pod in it) 35 * When the friends list is added, the server will present itself to all these friends ("make friend" operation) with the following informations: its **public RSA key** and its **URL**
35 * When the friends list is added, the server will present itself to all these friends with the following informations: its **public RSA key** and its **URL** 36 * Then, the pods will slowly share their videos database
36 * In returns, it will receive the list of the videos owned by the pod 37 * All the friends have the initial score of x points which represents the reliability of this friend
37 * All the friends have the initial score of 100 points (can be increased to 1000) which represents the reliability of this friend 38 * If the score reaches 0 the friend is revoked (blacklisted for the future?) and its videos are deleted
38 * If the score reaches 0 the friend is revoked (and blacklisted for the future ?) and the video deleted
39 39
40### Communications 40### Communications
41 * All the communications between pods are signed, encrypted with a RSA key and a symetric encryption 41 * All the communications between pods are signed, encrypted with a RSA key and a symetric encryption in a POST request (body)
42 * All the requests are retried if they failed 10 times with a factor of 3 (so it will finally fail in ~ 16h) 42 * All the requests are retried if they failed
43 * If a request fails the score is decreased by 10 points 43 * The requests are made at regular intervals to all pods of the network with a limit of parallel requests: if there are 1000 pods in the networks, the server won't make more than 10 requests in parallel
44 * If a request is a success the score is increased by 10 points 44 * If there are no informations to send (no video addition/removal), no requests are made
45 * The maximum of points would be 1000 (maybe more or less, depending of the server activity) 45 * The requests are grouped: for example if the interval is 1 hour and an user upload 2 videos there will be only 1 request that contains the 2 videos informations
46 * A pod which receives a request checks if the signature corresponds to the pod it has in its database. Then, it decrypts the body (or ignores it if the signature is not valid) 46 * The requests in the group are ordered: if an user add a video and remove it, we need to transmit these informations to the other pods in the same order
47 * The requests are grouped with a limit: if an user uploads 100 videos at a time, the information could be propagated in a few hours to do not make too big requests
48 * If a grouped request fails the score is decreased by x points
49 * If a grouped request is a success the score is increased by x points
50 * The maximum of points would be defined
51 * A pod which receives a request checks if the signature corresponds to the pod it has in its database. Then, it decrypts the body (or ignores it if the signature is not valid) and process the requests in the same order
47 52
48### Actions on a pod 53### Actions on a pod
49 * A pod is a tracker (websocket) which is responsible for all the video uploaded in it 54 * A pod is a websocket tracker which is responsible for all the video uploaded in it
55 * A pod has an administrator that can add/remove users, make friends and quit friends
50 * A pod has different user accounts that can upload videos 56 * A pod has different user accounts that can upload videos
51 * All pods have an index of all videos of the network (name, origin pod url, small description, uploader username, magnet Uri, thumbnail name, created date and the thumbnail file). For example, a test with 1000000 of videos with only alphanum characters and the following lengths: name = 50, author = 50, url = 25, description = 250, magnerUri = 200, thumbnail name = 50 has a mongodb size of ~ 4GB. To this, we add 1 000 000 thumbnails of 5-15 KB so 15GB maximum 57 * All pods have an index of all videos of the network (name, origin pod url, small description, uploader username, magnet Uri, thumbnail name, created date and the thumbnail file). For example, a test with 1000000 videos with alphanum characters and the following lengths: name = 50, author = 50, url = 25, description = 250, magnerUri = 200, thumbnail name = 50 has a mongodb size of ~ 4GB. To this, we add 1 000 000 thumbnails of 5-15 KB so 15GB maximum
52 * After having uploaded a video, the server seeds it, adds the meta data in its database and makes a secure request to all of its friends 58 * After having uploaded a video, the server seeds it, adds the meta data in its database and makes a secure request to all of its friends
53 * If an user wants to watch a video, he asks its pod the magnetUri and the frontend adds the torrent (with webtorrent), creates the video tag and streams the file into it 59 * If an user wants to watch a video, he asks its pod the magnetUri and the frontend adds the torrent (with WebTorrent), creates the HTML5 video tag and streams the file into it
54 * An user watching a video seeds it too (bittorent) so another user who is watching the same video can get the data from the origin server and the user 1 (etc) 60 * An user watching a video seeds it too (bittorent) so another user who is watching the same video can get the data from the origin server and the user 1 (etc)
55 61
56## Ideas 62## Ideas
57 63
58 * A video could have more information (detailed description etc) that are not sent on other pods. The user who wants to see these informations has to ask its pod: 64 * A video could have more information (detailed description etc) that are not sent on other pods. The user who wants to see these informations has to ask its pod:
59 user asks its pod -> user pod asks origin video pod -> origin video pod responds with the informations -> user pod responds to the user (and puts in cache the informations ?). We could extend this scheme with other informations (user profile etc) 65 user asks its pod -> user pod asks origin video pod -> origin video pod responds with the informations -> user pod responds to the user (and puts in cache the informations ?). We could extend this scheme with other informations
60 * Redondance: if the origin pod is down, the video is not accessible anymore. We could imagine a redondance between pods that keep seeding the video 66 * Redondance: if the origin pod is down, the video is not accessible anymore (no tracker/seeds). We could imagine a redondance between pods that keep seeding the video
61 * Server could transcode the video to lower qualities (cost in CPU and disk space) 67 * Server could transcode the video to lower qualities (cost in CPU and disk space)
62 * Server could seed at the demand: for now the server seeds all the videos but it has two drawbacks: 68 * Server could seed at the demand: for now the server seeds all the videos but it has two drawbacks:
63 - Seeding has a cost and is a long process 69 - Seeding has a cost (if a pod has 1000 videos it could be an issue)
64 - After a restart the server has to reseed all the videos (if it has 100 videos it could be very long!) 70 - After a restart the server has to reseed all the videos (with 1000 videos it could long)
65 If this solution is choosen, the frontend has to notify the origin pod that it has to seed the video it has not been seeded already 71 If this solution is choosen, the frontend has to notify the origin pod that it has to seed the video
66 * Avoid friend-making being an irreversible operation and a mandatory action at the first startup 72 * Add subtitles to videos
67 73 * Avoid stocking friends URL schemes (http/https)
68## Wanted but no idea how to implement
69
70 * Avoid URL scheme (http or https): if a server wants to implement TLS (and force it), it will break all the communication links
71 74
72## Debate 75## Debate
73 76
74 * Any user can view a video of a pod or he should have an account?
75 * Is an ex-friend should be blacklisted for the future? 77 * Is an ex-friend should be blacklisted for the future?
76 * Handle API breaks in a network. Major update always breaks the API: we need the entire network update to the same major version. Specify a limit date (2, 3 weeks after the release ?) after which all the updated pod switch to the new version of the API? The non updated pod will be ejected of the network because of they would implement the new API 78 * Handle API breaks in a network. If a major update breaks the API: we need the entire network update to the same major version. We could specify a limit date (2, 3 weeks after the release?) after which all the updated pod would switch to the new version of the API. The non updated pod will then be ejected of the network because would not implement the new API