amaster-web.dk
robots.txt

Robots Exclusion Standard data for amaster-web.dk

Resource Scan

Scan Details

Site Domain amaster-web.dk
Base Domain amaster-web.dk
Scan Status Ok
Last Scan2025-05-29T23:22:34+00:00
Next Scan 2025-06-28T23:22:34+00:00

Last Scan

Scanned2025-05-29T23:22:34+00:00
URL https://amaster-web.dk/robots.txt
Redirect https://www.amaster-web.dk/robots.txt
Redirect Domain www.amaster-web.dk
Redirect Base amaster-web.dk
Domain IPs 94.231.106.19
Redirect IPs 94.231.106.19
Response IP 94.231.106.19
Found Yes
Hash 5f27360803c24cfb51af0eb33fde8f8d5819caf6ef2470c2e1773ec1654cb1fa
SimHash 7a18cb208bf3

Groups

lwnutch

Rule Path
Disallow /*

dotbot

Rule Path
Disallow /*

genieo

Rule Path
Disallow /*

buibui-checkbot

Rule Path
Disallow /*

siteexplorer

Rule Path
Disallow /*

seznambot

Rule Path
Disallow /*

eccp

Rule Path
Disallow /*

yandexbot

Rule Path
Disallow /*

yahoo! slurp

Rule Path
Disallow /*

mj12bot

Rule Path
Disallow /*

*

Rule Path
Allow /
Disallow /admin
Disallow /search
Disallow /login
Disallow /mobile

Other Records

Field Value
sitemap https://www.amaster-web.dk/sitemap.xml

Comments

  • Stop overcrawling bots