aboutsummaryrefslogtreecommitdiffhomepage
path: root/doc
diff options
context:
space:
mode:
authornodiscc <nodiscc@gmail.com>2018-08-30 20:03:00 +0200
committernodiscc <nodiscc@gmail.com>2018-09-09 16:21:58 +0200
commit6c44d604a1ee0360a2eaf24b9cac18ca95edcb5f (patch)
treee6c22c8144d57dc9f597e37c6486f271165aa385 /doc
parent2b4f391559586af734d5d5cb5886aa78c4f9557d (diff)
downloadShaarli-6c44d604a1ee0360a2eaf24b9cac18ca95edcb5f.tar.gz
Shaarli-6c44d604a1ee0360a2eaf24b9cac18ca95edcb5f.tar.zst
Shaarli-6c44d604a1ee0360a2eaf24b9cac18ca95edcb5f.zip
doc: server config: basic usage of robots.txt/HTML robots meta-tag/crawler control mechanisms
Diffstat (limited to 'doc')
-rw-r--r--doc/md/Server-configuration.md7
1 files changed, 7 insertions, 0 deletions
diff --git a/doc/md/Server-configuration.md b/doc/md/Server-configuration.md
index e281dc85..cf44ecf5 100644
--- a/doc/md/Server-configuration.md
+++ b/doc/md/Server-configuration.md
@@ -397,6 +397,7 @@ http {
397``` 397```
398 398
399## Proxies 399## Proxies
400
400If Shaarli is served behind a proxy (i.e. there is a proxy server between clients and the web server hosting Shaarli), please refer to the proxy server documentation for proper configuration. In particular, you have to ensure that the following server variables are properly set: 401If Shaarli is served behind a proxy (i.e. there is a proxy server between clients and the web server hosting Shaarli), please refer to the proxy server documentation for proper configuration. In particular, you have to ensure that the following server variables are properly set:
401 402
402- `X-Forwarded-Proto` 403- `X-Forwarded-Proto`
@@ -405,6 +406,12 @@ If Shaarli is served behind a proxy (i.e. there is a proxy server between client
405 406
406See also [proxy-related](https://github.com/shaarli/Shaarli/issues?utf8=%E2%9C%93&q=label%3Aproxy+) issues. 407See also [proxy-related](https://github.com/shaarli/Shaarli/issues?utf8=%E2%9C%93&q=label%3Aproxy+) issues.
407 408
409## Robots and crawlers
410
411Shaarli disallows indexing and crawling of your local documentation pages by search engines, using `<meta name="robots">` HTML tags.
412Your Shaarli instance and other pages you host may still be indexed by various robots on the public Internet.
413You may want to setup a robots.txt file or other crawler control mechanism on your server.
414See [[1]](https://en.wikipedia.org/wiki/Robots_exclusion_standard), [[2]](https://support.google.com/webmasters/answer/6062608?hl=en) and [[3]](https://developers.google.com/search/reference/robots_meta_tag)
408 415
409## See also 416## See also
410 417