whereversim.de
robots.txt

Robots Exclusion Standard data for whereversim.de

Resource Scan

Scan Details

Site Domain whereversim.de
Base Domain whereversim.de
Scan Status Ok
Last Scan2024-09-07T13:56:05+00:00
Next Scan 2024-10-07T13:56:05+00:00

Last Scan

Scanned2024-09-07T13:56:05+00:00
URL https://whereversim.de/robots.txt
Domain IPs 3.233.126.24, 34.234.52.18, 52.206.163.162
Response IP 52.206.163.162
Found Yes
Hash b4dbfc872ae71c142d760dfe95766bafa3f476cc67767b9565149b6143c25030
SimHash f918180687d3

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /admin/
Disallow /temp/

webreaper
webcopier
offline explorer
httrack
microsoft.url.control
emailcollector
penthesilea

Rule Path
Disallow /

Other Records

Field Value
sitemap https://whereversim.de/sitemap.xml

Comments

  • ===================================
  • Folgende Seiten sollen nicht indexiert werden:
  • ===================================
  • ===================================
  • Schließe folgende Spider komplett aus:
  • ===================================