faso.com
robots.txt

Robots Exclusion Standard data for faso.com

Resource Scan

Scan Details

Site Domain faso.com
Base Domain faso.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-21T16:07:09+00:00
Next Scan 2024-12-20T16:07:09+00:00

Last Successful Scan

Scanned2023-05-05T03:56:07+00:00
URL https://faso.com/robots.txt
Domain IPs 104.20.22.252, 104.20.23.252, 172.67.33.250
Response IP 172.67.33.250
Found Yes
Hash b81462a36711d58fe20050d0eb9fc6b8b74eb953c40fe70d7a4bf05d58db0137
SimHash 9b564b0c4ec3

Groups

*

Rule Path
Disallow /brushbuzz/links/
Disallow /brushbuzz/quotes/
Disallow /brushbuzz/tag/
Disallow /boldbrush/badges/
Disallow /search/
Disallow /art-marketing-newsletter/archives/
Disallow /static/html/

mediapartners-google*

Rule Path
Disallow

exabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

bender

Rule Path
Disallow /

discobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

Comments

  • All robots will spider the domain