docs/en/developer/asynchronous.rst

   1 Asynchronous tasks
   2 ==================
   3
   4 In order to launch asynchronous tasks (useful for huge imports for example), we can use RabbitMQ or Redis.
   5
   6 Install RabbitMQ for asynchronous tasks
   7 ---------------------------------------
   8
   9 Requirements
  10 ^^^^^^^^^^^^
  11
  12 You need to have RabbitMQ installed on your server.
  13
  14 Installation
  15 ^^^^^^^^^^^^
  16
  17 .. code:: bash
  18
  19   wget https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
  20   apt-key add rabbitmq-signing-key-public.asc
  21   apt-get update
  22   apt-get install rabbitmq-server
  23
  24 Configuration and launch
  25 ^^^^^^^^^^^^^^^^^^^^^^^^
  26
  27 .. code:: bash
  28
  29   rabbitmq-plugins enable rabbitmq_management # (useful to have a web interface, available at http://localhost:15672/ (guest/guest)
  30   rabbitmq-server -detached
  31
  32 Stop RabbitMQ
  33 ^^^^^^^^^^^^^
  34
  35 .. code:: bash
  36
  37   rabbitmqctl stop
  38
  39
  40 Configure RabbitMQ in wallabag
  41 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  42
  43 Edit your ``app/config/parameters.yml`` file to edit RabbitMQ configuration. The default one should be ok:
  44
  45 .. code:: yaml
  46
  47     rabbitmq_host: localhost
  48     rabbitmq_port: 5672
  49     rabbitmq_user: guest
  50     rabbitmq_password: guest
  51     rabbitmq_prefetch_count: 10 # read http://www.rabbitmq.com/consumer-prefetch.html
  52
  53 Enable RabbitMQ in wallabag
  54 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  55
  56 In internal settings, in the **Import** section, enable RabbitMQ (with the value 1).
  57
  58 Launch RabbitMQ consumer
  59 ^^^^^^^^^^^^^^^^^^^^^^^^
  60
  61 Depending on which service you want to import from you need to enable one (or many if you want to support many) cron job:
  62
  63 .. code:: bash
  64
  65   # for Pocket import
  66   bin/console rabbitmq:consumer -e=prod import_pocket -w
  67
  68   # for Readability import
  69   bin/console rabbitmq:consumer -e=prod import_readability -w
  70
  71   # for Instapaper import
  72   bin/console rabbitmq:consumer -e=prod import_instapaper -w
  73
  74   # for wallabag v1 import
  75   bin/console rabbitmq:consumer -e=prod import_wallabag_v1 -w
  76
  77   # for wallabag v2 import
  78   bin/console rabbitmq:consumer -e=prod import_wallabag_v2 -w
  79
  80   # for Firefox import
  81   bin/console rabbitmq:consumer -e=prod import_firefox -w
  82
  83   # for Chrome import
  84   bin/console rabbitmq:consumer -e=prod import_chrome -w
  85
  86 Install Redis for asynchronous tasks
  87 ------------------------------------
  88
  89 In order to launch asynchronous tasks (useful for huge imports for example), we can use Redis.
  90
  91 Requirements
  92 ^^^^^^^^^^^^
  93
  94 You need to have Redis installed on your server.
  95
  96 Installation
  97 ^^^^^^^^^^^^
  98
  99 .. code:: bash
 100
 101   apt-get install redis-server
 102
 103 Launch
 104 ^^^^^^
 105
 106 The server might be already running after installing, if not you can launch it using:
 107
 108 .. code:: bash
 109
 110   redis-server
 111
 112
 113 Configure Redis in wallabag
 114 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 115
 116 Edit your ``app/config/parameters.yml`` file to edit Redis configuration. The default one should be ok:
 117
 118 .. code:: yaml
 119
 120     redis_host: localhost
 121     redis_port: 6379
 122
 123 Enable Redis in wallabag
 124 ^^^^^^^^^^^^^^^^^^^^^^^^
 125
 126 In internal settings, in the **Import** section, enable Redis (with the value 1).
 127
 128 Launch Redis consumer
 129 ^^^^^^^^^^^^^^^^^^^^^
 130
 131 Depending on which service you want to import from you need to enable one (or many if you want to support many) cron job:
 132
 133 .. code:: bash
 134
 135   # for Pocket import
 136   bin/console wallabag:import:redis-worker -e=prod pocket -vv >> /path/to/wallabag/var/logs/redis-pocket.log
 137
 138   # for Readability import
 139   bin/console wallabag:import:redis-worker -e=prod readability -vv >> /path/to/wallabag/var/logs/redis-readability.log
 140
 141   # for Instapaper import
 142   bin/console wallabag:import:redis-worker -e=prod instapaper -vv >> /path/to/wallabag/var/logs/redis-instapaper.log
 143
 144   # for wallabag v1 import
 145   bin/console wallabag:import:redis-worker -e=prod wallabag_v1 -vv >> /path/to/wallabag/var/logs/redis-wallabag_v1.log
 146
 147   # for wallabag v2 import
 148   bin/console wallabag:import:redis-worker -e=prod wallabag_v2 -vv >> /path/to/wallabag/var/logs/redis-wallabag_v2.log
 149
 150   # for Firefox import
 151   bin/console wallabag:import:redis-worker -e=prod firefox -vv >> /path/to/wallabag/var/logs/redis-firefox.log
 152
 153   # for Chrome import
 154   bin/console wallabag:import:redis-worker -e=prod chrome -vv >> /path/to/wallabag/var/logs/redis-chrome.log
 155
 156 If you want to launch the import only for some messages and not all, you can specify this number (here 12) and the worker will stop right after the 12th message :
 157
 158 .. code:: bash
 159
 160   bin/console wallabag:import:redis-worker -e=prod pocket -vv --maxIterations=12