nybizdb.com
robots.txt

Robots Exclusion Standard data for nybizdb.com

Resource Scan

Scan Details

Site Domain nybizdb.com
Base Domain nybizdb.com
Scan Status Ok
Last Scan2024-09-26T10:27:07+00:00
Next Scan 2024-10-03T10:27:07+00:00

Last Scan

Scanned2024-09-26T10:27:07+00:00
URL https://nybizdb.com/robots.txt
Redirect https://www.nybizdb.com/robots.txt
Redirect Domain www.nybizdb.com
Redirect Base nybizdb.com
Domain IPs 162.19.41.169
Redirect IPs 162.19.41.169
Response IP 162.19.41.169
Found Yes
Hash 61cf5a5ffe827b7d98eb6e8525e2348a2d40ea500c336f050711243302680fef
SimHash d35cd8505633

Groups

*

Rule Path
Allow /
Disallow /filter/

dotbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

wikido

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

exabot

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

admantx

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.nybizdb.com/sitemap.xml