burgbieren.nl
robots.txt

Robots Exclusion Standard data for burgbieren.nl

Resource Scan

Scan Details

Site Domain burgbieren.nl
Base Domain burgbieren.nl
Scan Status Ok
Last Scan5/23/2025, 12:25:14 AM
Next Scan 6/22/2025, 12:25:14 AM

Last Scan

Scanned5/23/2025, 12:25:14 AM
URL https://burgbieren.nl/robots.txt
Redirect https://www.burgbieren.nl/robots.txt
Redirect Domain www.burgbieren.nl
Redirect Base burgbieren.nl
Domain IPs 104.21.73.73, 172.67.158.185, 2606:4700:3030::6815:4949, 2606:4700:3031::ac43:9eb9
Redirect IPs 104.21.73.73, 172.67.158.185, 2606:4700:3030::6815:4949, 2606:4700:3031::ac43:9eb9
Response IP 172.67.158.185
Found Yes
Hash f525206500ff25efafc4e4c94fa10405e4d7170a4bc963278d3620505f6ad2d8
SimHash 6b18d800e6e1

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /getjsondata/

Other Records

Field Value
crawl-delay 5

amazonbot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

*

Rule Path
Disallow /*?

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.burgbieren.nl/sitemap_index.xml