]>
Commit | Line | Data |
---|---|---|
2e389b0b NL |
1 | Asynchronous tasks |
2 | ================== | |
3 | ||
4 | In order to launch asynchronous tasks (useful for huge imports for example), we can use RabbitMQ or Redis. | |
5 | ||
6 | Install RabbitMQ for asynchronous tasks | |
7 | --------------------------------------- | |
8 | ||
9 | Requirements | |
10 | ^^^^^^^^^^^^ | |
11 | ||
12 | You need to have RabbitMQ installed on your server. | |
13 | ||
14 | Installation | |
15 | ^^^^^^^^^^^^ | |
16 | ||
17 | .. code:: bash | |
18 | ||
19 | wget https://www.rabbitmq.com/rabbitmq-signing-key-public.asc | |
20 | apt-key add rabbitmq-signing-key-public.asc | |
21 | apt-get update | |
22 | apt-get install rabbitmq-server | |
23 | ||
24 | Configuration and launch | |
25 | ^^^^^^^^^^^^^^^^^^^^^^^^ | |
26 | ||
27 | .. code:: bash | |
28 | ||
29 | rabbitmq-plugins enable rabbitmq_management # (useful to have a web interface, available at http://localhost:15672/ (guest/guest) | |
30 | rabbitmq-server -detached | |
31 | ||
32 | Stop RabbitMQ | |
33 | ^^^^^^^^^^^^^ | |
34 | ||
35 | .. code:: bash | |
36 | ||
37 | rabbitmqctl stop | |
38 | ||
39 | ||
40 | Configure RabbitMQ in wallabag | |
41 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
42 | ||
43 | Edit your ``app/config/parameters.yml`` file to edit RabbitMQ configuration. The default one should be ok: | |
44 | ||
45 | .. code:: yaml | |
46 | ||
47 | rabbitmq_host: localhost | |
48 | rabbitmq_port: 5672 | |
49 | rabbitmq_user: guest | |
50 | rabbitmq_password: guest | |
881b0578 | 51 | rabbitmq_prefetch_count: 10 # read http://www.rabbitmq.com/consumer-prefetch.html |
2e389b0b NL |
52 | |
53 | Enable RabbitMQ in wallabag | |
54 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
55 | ||
56 | In internal settings, in the **Import** section, enable RabbitMQ (with the value 1). | |
57 | ||
58 | Launch RabbitMQ consumer | |
59 | ^^^^^^^^^^^^^^^^^^^^^^^^ | |
60 | ||
61 | Depending on which service you want to import from you need to enable one (or many if you want to support many) cron job: | |
62 | ||
63 | .. code:: bash | |
64 | ||
65 | # for Pocket import | |
66 | bin/console rabbitmq:consumer -e=prod import_pocket -w | |
67 | ||
68 | # for Readability import | |
69 | bin/console rabbitmq:consumer -e=prod import_readability -w | |
70 | ||
71 | # for Instapaper import | |
72 | bin/console rabbitmq:consumer -e=prod import_instapaper -w | |
73 | ||
74 | # for wallabag v1 import | |
75 | bin/console rabbitmq:consumer -e=prod import_wallabag_v1 -w | |
76 | ||
77 | # for wallabag v2 import | |
78 | bin/console rabbitmq:consumer -e=prod import_wallabag_v2 -w | |
79 | ||
80 | # for Firefox import | |
81 | bin/console rabbitmq:consumer -e=prod import_firefox -w | |
82 | ||
83 | # for Chrome import | |
84 | bin/console rabbitmq:consumer -e=prod import_chrome -w | |
85 | ||
86 | Install Redis for asynchronous tasks | |
87 | ------------------------------------ | |
88 | ||
89 | In order to launch asynchronous tasks (useful for huge imports for example), we can use Redis. | |
90 | ||
91 | Requirements | |
92 | ^^^^^^^^^^^^ | |
93 | ||
94 | You need to have Redis installed on your server. | |
95 | ||
96 | Installation | |
97 | ^^^^^^^^^^^^ | |
98 | ||
99 | .. code:: bash | |
100 | ||
101 | apt-get install redis-server | |
102 | ||
103 | Launch | |
104 | ^^^^^^ | |
105 | ||
106 | The server might be already running after installing, if not you can launch it using: | |
107 | ||
108 | .. code:: bash | |
109 | ||
110 | redis-server | |
111 | ||
112 | ||
113 | Configure Redis in wallabag | |
114 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
115 | ||
116 | Edit your ``app/config/parameters.yml`` file to edit Redis configuration. The default one should be ok: | |
117 | ||
118 | .. code:: yaml | |
119 | ||
120 | redis_host: localhost | |
121 | redis_port: 6379 | |
122 | ||
123 | Enable Redis in wallabag | |
124 | ^^^^^^^^^^^^^^^^^^^^^^^^ | |
125 | ||
126 | In internal settings, in the **Import** section, enable Redis (with the value 1). | |
127 | ||
128 | Launch Redis consumer | |
129 | ^^^^^^^^^^^^^^^^^^^^^ | |
130 | ||
131 | Depending on which service you want to import from you need to enable one (or many if you want to support many) cron job: | |
132 | ||
133 | .. code:: bash | |
134 | ||
135 | # for Pocket import | |
136 | bin/console wallabag:import:redis-worker -e=prod pocket -vv >> /path/to/wallabag/var/logs/redis-pocket.log | |
137 | ||
138 | # for Readability import | |
139 | bin/console wallabag:import:redis-worker -e=prod readability -vv >> /path/to/wallabag/var/logs/redis-readability.log | |
140 | ||
141 | # for Instapaper import | |
142 | bin/console wallabag:import:redis-worker -e=prod instapaper -vv >> /path/to/wallabag/var/logs/redis-instapaper.log | |
143 | ||
144 | # for wallabag v1 import | |
145 | bin/console wallabag:import:redis-worker -e=prod wallabag_v1 -vv >> /path/to/wallabag/var/logs/redis-wallabag_v1.log | |
146 | ||
147 | # for wallabag v2 import | |
148 | bin/console wallabag:import:redis-worker -e=prod wallabag_v2 -vv >> /path/to/wallabag/var/logs/redis-wallabag_v2.log | |
149 | ||
150 | # for Firefox import | |
151 | bin/console wallabag:import:redis-worker -e=prod firefox -vv >> /path/to/wallabag/var/logs/redis-firefox.log | |
152 | ||
153 | # for Chrome import | |
b46e63fc | 154 | bin/console wallabag:import:redis-worker -e=prod chrome -vv >> /path/to/wallabag/var/logs/redis-chrome.log |
2e389b0b NL |
155 | |
156 | If you want to launch the import only for some messages and not all, you can specify this number (here 12) and the worker will stop right after the 12th message : | |
157 | ||
158 | .. code:: bash | |
159 | ||
160 | bin/console wallabag:import:redis-worker -e=prod pocket -vv --maxIterations=12 |