alltimefavorites.com
robots.txt

Robots Exclusion Standard data for alltimefavorites.com

Resource Scan

Scan Details

Site Domain alltimefavorites.com
Base Domain alltimefavorites.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-08T11:33:07+00:00
Next Scan 2024-10-06T11:33:07+00:00

Last Successful Scan

Scanned2023-06-15T08:33:50+00:00
URL https://alltimefavorites.com/robots.txt
Redirect https://www.alltimefavorites.com/robots.txt
Redirect Domain www.alltimefavorites.com
Redirect Base alltimefavorites.com
Domain IPs 104.26.8.148, 104.26.9.148, 172.67.69.62, 2606:4700:20::681a:894, 2606:4700:20::681a:994, 2606:4700:20::ac43:453e
Redirect IPs 104.26.8.148, 104.26.9.148, 172.67.69.62, 2606:4700:20::681a:894, 2606:4700:20::681a:994, 2606:4700:20::ac43:453e
Response IP 104.26.9.148
Found Yes
Hash 4e841499a9095f0237803cce8f91662e4c61d5af73d1c6407e4df333ed10005f
SimHash 171c80e2ebb5

Groups

*

Rule Path
Disallow /dm/
Disallow /riders/
Disallow /admin/
Disallow /local2/
Disallow /vendors/
Disallow /av/
Disallow /superfun/clipfed.php

xovibot

Rule Path
Disallow /

wikido

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

zoomspider

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

*

Rule Path
Disallow */wp-admin/*
Disallow */wp-login.php
Disallow */wp-register.php

Comments

  • Unidentified misbehaving bots
  • User-agent: bhc.collectionBot
  • Disallow: /
  • User-agent: msnbot
  • Crawl-delay: 5
  • Unsafe robots to keep away