aubiz.net
robots.txt

Robots Exclusion Standard data for aubiz.net

Resource Scan

Scan Details

Site Domain aubiz.net
Base Domain aubiz.net
Scan Status Ok
Last Scan2024-09-27T17:47:02+00:00
Next Scan 2024-10-04T17:47:02+00:00

Last Scan

Scanned2024-09-27T17:47:02+00:00
URL https://aubiz.net/robots.txt
Domain IPs 139.99.247.189
Response IP 139.99.247.189
Found Yes
Hash f176c075c456d6451954ccd9ad23c7d4e411ab4e9f8be1fbcd11262f9ef002a6
SimHash d35cd0405733

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow /filter/

magpie-crawler

Rule Path
Disallow /

wikido

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

exabot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

proximic

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

wesee:search

Rule Path
Disallow /

admantx

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.aubiz.net/sitemap.xml