diff options
Diffstat (limited to 'doc/Server-configuration.md')
-rw-r--r-- | doc/Server-configuration.md | 12 |
1 files changed, 0 insertions, 12 deletions
diff --git a/doc/Server-configuration.md b/doc/Server-configuration.md index fd98a608..1ab57a0a 100644 --- a/doc/Server-configuration.md +++ b/doc/Server-configuration.md | |||
@@ -334,15 +334,3 @@ http { | |||
334 | } | 334 | } |
335 | } | 335 | } |
336 | ``` | 336 | ``` |
337 | |||
338 | ## Restricting search engines and web crawler traffic | ||
339 | |||
340 | Creating a `robots.txt` witht he following contents at the root of your Shaarli installation will prevent "honest" web crawlers from indexing each and every link and Daily page from a Shaarli instance, thus getting rid of a certain amount of unsollicited network traffic. | ||
341 | |||
342 | ``` | ||
343 | User-agent: * | ||
344 | Disallow: / | ||
345 | ``` | ||
346 | |||
347 | See: http://www.robotstxt.org/, http://www.robotstxt.org/robotstxt.html, http://www.robotstxt.org/meta.html | ||
348 | |||