fanpagekarma.com
robots.txt

Robots Exclusion Standard data for fanpagekarma.com

Resource Scan

Scan Details

Site Domain fanpagekarma.com
Base Domain fanpagekarma.com
Scan Status Ok
Last Scan2024-05-31T09:48:18+00:00
Next Scan 2024-06-30T09:48:18+00:00

Last Scan

Scanned2024-05-31T09:48:18+00:00
URL https://fanpagekarma.com/robots.txt
Redirect https://www.fanpagekarma.com/robots.txt
Redirect Domain www.fanpagekarma.com
Redirect Base fanpagekarma.com
Domain IPs 34.160.107.204
Redirect IPs 34.160.107.204
Response IP 34.160.107.204
Found Yes
Hash 1792f44f8a1f5607dff934568bb0e5502df68076237afc83dfd7548a01a0119d
SimHash 990c5600fc12

Groups

*

Rule Path
Allow /
Disallow /fbImage.png
Disallow /embed/
Disallow /*.IBehaviorListener.*
Disallow /InternalError
Disallow /Error404

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

jobdiggerspider

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

petal bot

Rule Path
Disallow /