werkenbijgroupm.nl
robots.txt

Robots Exclusion Standard data for werkenbijgroupm.nl

Resource Scan

Scan Details

Site Domain werkenbijgroupm.nl
Base Domain werkenbijgroupm.nl
Scan Status Ok
Last Scan2024-09-22T04:19:52+00:00
Next Scan 2024-10-06T04:19:52+00:00

Last Scan

Scanned2024-09-22T04:19:52+00:00
URL https://werkenbijgroupm.nl/robots.txt
Redirect https://www.werkenbijgroupm.nl/robots.txt
Redirect Domain www.werkenbijgroupm.nl
Redirect Base werkenbijgroupm.nl
Domain IPs 134.209.138.125
Redirect IPs 104.22.70.155, 104.22.71.155, 172.67.24.16, 2606:4700:10::6816:469b, 2606:4700:10::6816:479b, 2606:4700:10::ac43:1810
Response IP 104.22.71.155
Found Yes
Hash e7d81563418cce123a5136f23d76088ff7d45158b72a821fa3d478b5348cf7ae
SimHash 531dd040cff2

Groups

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow *?t=
Disallow *%26t%3D

Other Records

Field Value
sitemap https://www.werkenbijgroupm.nl/sitemap.xml