pxfuel.com
robots.txt

Robots Exclusion Standard data for pxfuel.com

Resource Scan

Scan Details

Site Domain pxfuel.com
Base Domain pxfuel.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-15T22:41:04+00:00
Next Scan 2024-12-14T22:41:04+00:00

Last Successful Scan

Scanned2022-11-24T04:45:51+00:00
URL https://pxfuel.com/robots.txt
Redirect https://www.pxfuel.com/robots.txt
Redirect Domain www.pxfuel.com
Redirect Base pxfuel.com
Response IP 172.67.169.122, 104.21.71.48
Found Yes
Hash 22d6c444b6e38868a4a39331d4846dba2e1fe0c830bd6f3a804238e67495fcc1
SimHash 8045c7f86d1a

Groups

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

hyscore

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

tineye-bot

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

musobot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

qwantify/bleriot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

hubspot crawler

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

*

Rule Path
Allow /