Bump version to v0.8.0

[github/shaarli/Shaarli.git] / doc / Server-security.md

diff --git a/doc/Server-security.md b/doc/Server-security.md
index 0d16e2840ccd3fc2636439d561b1fa3581ece1eb..50549a214617177633fac8cf81856c093d511811 100644 (file)
--- a/doc/Server-security.md
+++ b/doc/Server-security.md
@@ -58,3 +58,17 @@ before = common.conf
 failregex = \s-\s<HOST>\s-\sLogin failed for user.*$
 ignoreregex = 
 ```
+
+## Robots - Restricting search engines and web crawler traffic
+
+Creating a `robots.txt` with the following contents at the root of your Shaarli installation will prevent _honest_ web crawlers from indexing each and every link and Daily page from a Shaarli instance, thus getting rid of a certain amount of unsollicited network traffic.
+
+```
+User-agent: *
+Disallow: /
+```
+
+See:
+- http://www.robotstxt.org/
+- http://www.robotstxt.org/robotstxt.html
+- http://www.robotstxt.org/meta.html