ponfish.com
robots.txt

Robots Exclusion Standard data for ponfish.com

Resource Scan

Scan Details

Site Domain ponfish.com
Base Domain ponfish.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-21T07:01:42+00:00
Next Scan 2025-07-21T07:01:42+00:00

Last Successful Scan

Scanned2025-05-23T07:00:30+00:00
URL https://ponfish.com/robots.txt
Domain IPs 65.49.60.171
Response IP 65.49.60.171
Found Yes
Hash a8c3e0d99054acdf4e7b30aba591680b552ffb5d400f86f6b11fcabbd1b02ac2
SimHash 631c0802c113

Groups

*

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 18

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

extecontextcrawl

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claudebot/1.0

Rule Path
Disallow /

meta-externalagent/1.1

Rule Path
Disallow /

gptbot/1.2

Rule Path
Disallow /

amazonbot/0.1

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

dotbot/1.2

Rule Path
Disallow /

dataforseobot/1.0

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.ponfish.com/wiki/sitemap_index.xml
sitemap https://www.ponfish.com/wiki/sitemap_index.xml