afishing.com
robots.txt

Robots Exclusion Standard data for afishing.com

Resource Scan

Scan Details

Site Domain afishing.com
Base Domain afishing.com
Scan Status Ok
Last Scan2025-05-30T22:44:07+00:00
Next Scan 2025-06-06T22:44:07+00:00

Last Scan

Scanned2025-05-30T22:44:07+00:00
URL https://afishing.com/robots.txt
Domain IPs 175.126.226.133
Response IP 175.126.226.133
Found Yes
Hash f3b1c696b9fad42a03b41479be936caf60e46262c704d985e60c4ca49ede9bf6
SimHash cf3cc860ab21

Groups

bingbot

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

duckduckbot-https

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

simplepie

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

scmguard

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /2013_banner
Disallow /2014_banner
Disallow /2015_banner
Disallow /2016_banner
Disallow /2017_banner
Disallow /2018_banner
Disallow /2019_banner
Disallow /2020_banner
Disallow /2021_banner
Disallow /5000eging
Disallow /addons
Disallow /admin
Disallow /badadata
Disallow /config
Disallow /data
Disallow /common
Disallow /classes
Disallow /dica2008
Disallow /dica2009
Disallow /dica2010
Disallow /files
Disallow /files_back
Disallow /files_tmp
Disallow /libs
Disallow /layouts
Disallow /m.layouts
Disallow /modules
Disallow /package
Disallow /widgets
Disallow /widgetstyles

Other Records

Field Value
crawl-delay 5