Log In Sign Up

definition-of.com
robots.txt

Robots Exclusion Standard data for definition-of.com

Resource Scan

Scan Details

Site Domain definition-of.com
Base Domain definition-of.com
Scan Status Ok
Last Scan2024-09-29T02:09:53+00:00
Next Scan 2024-10-06T02:09:53+00:00

Last Scan

Scanned2024-09-29T02:09:53+00:00
URL https://definition-of.com/robots.txt
Domain IPs 104.21.30.238, 172.67.174.47, 2606:4700:3031::6815:1eee, 2606:4700:3037::ac43:ae2f
Response IP 172.67.174.47
Found Yes
Hash 6f71cdb00ef619903e7f4d70b80d22789f757c115a798cb04c8f7a6c708bb0f2
SimHash c01544042d61

Groups

mediapartners-google*

Rule Path
Disallow

npbot

Rule Path
Disallow /

open*

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

scooter*

Rule Path
Disallow /

websearch*

Rule Path
Disallow /

zyborg*

Rule Path
Disallow /

wget*

Rule Path
Disallow /

turnitinbot*

Rule Path
Disallow /

slysearch*

Rule Path
Disallow /

seekbot*

Rule Path
Disallow /

netresearchserver*

Rule Path
Disallow /

seekbot*

Rule Path
Disallow /

mozdex*

Rule Path
Disallow /

link*

Rule Path
Disallow /

larbin*

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

*

Rule Path
Disallow /_/app/
Disallow /_/access/
Back to top

Other Records

Field Value
sitemap http://www.definition-of.com/sitemap.xml.gz
Back to top