Commit eb4cf0ae authored by echel0n's avatar echel0n

Removed authentication requirement for robots.txt handler.

parent f18ee546
......@@ -43,7 +43,6 @@ class RobotsDotTxtHandler(BaseHandler, ABC):
def initialize(self):
self.set_header('Content-Type', 'text/plain')
def get(self, *args, **kwargs):
""" Keep web crawlers out """
return self.write("User-agent: *\nDisallow: /")
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment