Log In Sign Up

city-kinos.de
robots.txt

Robots Exclusion Standard data for city-kinos.de

Resource Scan

Scan Details

Site Domain city-kinos.de
Base Domain city-kinos.de
Scan Status Ok
Last Scan2024-11-18T09:51:43+00:00
Next Scan 2024-12-18T09:51:43+00:00

Last Scan

Scanned2024-11-18T09:51:43+00:00
URL https://city-kinos.de/robots.txt
Redirect https://www.city-kinos.de/robots.txt
Redirect Domain www.city-kinos.de
Redirect Base city-kinos.de
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.123, 76.76.21.9
Response IP 76.76.21.98
Found Yes
Hash eac1164af71e0dd3f395731f457fd66943b2808795a4962a2f18b4780c9d0e47
SimHash 6d441b13c7d1

Groups

*

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile-apps

Rule Path
Allow /
Back to top

Comments

  • Block googlebot from example.com/directory1/... and example.com/directory2/...
  • but allow access to directory2/subdirectory1/...
  • All other directories on the site are allowed by default.
  • Block the entire site from anothercrawler.
  • User-agent: googlebot
  • Disallow: /
Back to top