finexpense.com
robots.txt

Robots Exclusion Standard data for finexpense.com

Resource Scan

Scan Details

Site Domain finexpense.com
Base Domain finexpense.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-09-30T13:57:45+00:00
Next Scan 2024-12-29T13:57:45+00:00

Last Successful Scan

Scanned2024-06-09T13:55:46+00:00
URL https://finexpense.com/robots.txt
Domain IPs 104.21.31.239, 172.67.180.204, 2606:4700:3030::6815:1fef, 2606:4700:3032::ac43:b4cc
Response IP 172.67.180.204
Found Yes
Hash 7b86082e29fff999059bf3fad97a36982b952904325743de71508c4a2081a5e3
SimHash 6471d3001597

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow
Disallow /cgi-bin/

Other Records

Field Value
sitemap https://finexpense.com/post-sitemap.xml

Warnings

  • 1 invalid line.