From fdf88d194874a533cf3a8de3d317d70018aa8a62 Mon Sep 17 00:00:00 2001 From: ArthurHoaro Date: Wed, 12 Oct 2016 12:36:59 +0200 Subject: Bump version to v0.8.0 Signed-off-by: ArthurHoaro --- doc/Server-security.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'doc/Server-security.md') diff --git a/doc/Server-security.md b/doc/Server-security.md index 0d16e284..50549a21 100644 --- a/doc/Server-security.md +++ b/doc/Server-security.md @@ -58,3 +58,17 @@ before = common.conf failregex = \s-\s\s-\sLogin failed for user.*$ ignoreregex = ``` + +## Robots - Restricting search engines and web crawler traffic + +Creating a `robots.txt` with the following contents at the root of your Shaarli installation will prevent _honest_ web crawlers from indexing each and every link and Daily page from a Shaarli instance, thus getting rid of a certain amount of unsollicited network traffic. + +``` +User-agent: * +Disallow: / +``` + +See: +- http://www.robotstxt.org/ +- http://www.robotstxt.org/robotstxt.html +- http://www.robotstxt.org/meta.html -- cgit v1.2.3