Log In Sign Up

adfty.com
robots.txt

Robots Exclusion Standard data for adfty.com

Resource Scan

Scan Details

Site Domain adfty.com
Base Domain adfty.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-23T12:07:01+00:00
Next Scan 2025-06-30T12:07:01+00:00

Last Successful Scan

Scanned2025-04-13T08:36:53+00:00
URL https://adfty.com/robots.txt
Redirect https://adfty.biz/robots.txt
Redirect Domain adfty.biz
Redirect Base adfty.biz
Domain IPs 104.21.36.139, 172.67.194.216, 2606:4700:3033::6815:248b, 2606:4700:3033::ac43:c2d8
Redirect IPs 104.21.9.124, 172.67.189.67, 2606:4700:3030::6815:97c, 2606:4700:3034::ac43:bd43
Response IP 104.21.9.124
Found Yes
Hash 047d3ea194603950a05b94bd2efc550ff2fbbe11d395e966c37f5856762b2f82
SimHash 4034cd33630d

Groups

*

Rule Path
Disallow
Disallow /admin/
Disallow /admin/admin_index.php
Disallow /backup/
Disallow /cache/
Disallow /install/
Disallow /internal/
Disallow /languages/
Disallow /libs/
Disallow /live/
Disallow /LICENSE.txt
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /readme.html
Disallow /search.php
Disallow /search/
Disallow /searchurl/
Disallow /tag/
Disallow /templates/
Disallow /new/recent/
Disallow /new/yesterday/
Disallow /new/today/
Disallow /new/week/
Disallow /new/month/
Disallow /new/year/
Disallow /new/alltime/
Disallow /recent/
Disallow /yesterday/
Disallow /today/
Disallow /week/
Disallow /month/
Disallow /year/
Disallow /alltime/
Disallow /upvoted/
Disallow /downvoted/
Disallow /commented/

Other Records

Field Value
crawl-delay 5

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

riddler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

newspaper

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

semrushbot-ba

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

mj12bot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

bingbot

Rule Path
Allow /$
Disallow /

msnbot

Rule Path
Allow /$
Disallow /
Back to top

Comments

  • 1) this filename (robots.txt) must stay lowercase
  • 2) this file must be in the servers root directory
Back to top