floor8.com
robots.txt

Robots Exclusion Standard data for floor8.com

Resource Scan

Scan Details

Site Domain floor8.com
Base Domain floor8.com
Scan Status Ok
Last Scan2024-05-18T05:03:32+00:00
Next Scan 2024-05-25T05:03:32+00:00

Last Scan

Scanned2024-05-18T05:03:32+00:00
URL https://floor8.com/robots.txt
Redirect https://www.floor8.com/robots.txt
Redirect Domain www.floor8.com
Redirect Base floor8.com
Domain IPs 35.83.190.225, 44.241.188.24
Redirect IPs 13.35.18.29, 13.35.18.7, 13.35.18.83, 13.35.18.92, 2600:9000:20c7:4600:f:8f3b:1200:93a1, 2600:9000:20c7:5e00:f:8f3b:1200:93a1, 2600:9000:20c7:7400:f:8f3b:1200:93a1, 2600:9000:20c7:9200:f:8f3b:1200:93a1, 2600:9000:20c7:9600:f:8f3b:1200:93a1, 2600:9000:20c7:9c00:f:8f3b:1200:93a1, 2600:9000:20c7:b200:f:8f3b:1200:93a1, 2600:9000:20c7:fc00:f:8f3b:1200:93a1
Response IP 13.35.18.7
Found Yes
Hash 0597cbe3b3af99893eaad5ca6bde311f92b3bd0b094e2673d88aa412bd1c3f7e
SimHash 815023146427

Groups

*

Rule Path
Disallow */xposts/

*

Rule Path
Disallow */reader/

*

Rule Path
Disallow /ads/

*

Rule Path
Disallow */editor/*
Disallow */posts/*/edit$

*

Rule Path
Disallow */posts/*/publish$

*

Rule Path
Disallow */singlepage/pipe/*
Disallow */singlepage/uncached_pipe/*
Disallow */singlepage/epipe/*

*

Rule Path
Disallow */reads/*/read$

*

Rule Path
Disallow /matches/
Disallow /es/partidos/
Disallow /it/partite/
Disallow /de/spiele/
Disallow /zh-CN/*

*

Rule Path
Disallow /ping
Disallow */sessions
Disallow */admin/*
Disallow */management
Disallow */castr
Disallow */unfeature$
Disallow */unfeature/*

*

Rule Path
Disallow /videos/
Allow /channels/videos/

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • allow adsense crawler access to all pages