zwartecross.nl
robots.txt

Robots Exclusion Standard data for zwartecross.nl

Resource Scan

Scan Details

Site Domain zwartecross.nl
Base Domain zwartecross.nl
Scan Status Ok
Last Scan2025-04-09T01:15:42+00:00
Next Scan 2025-05-09T01:15:42+00:00

Last Scan

Scanned2025-04-09T01:15:42+00:00
URL https://zwartecross.nl/robots.txt
Redirect https://www.zwartecross.nl/robots.txt
Redirect Domain www.zwartecross.nl
Redirect Base zwartecross.nl
Domain IPs 104.26.4.219, 104.26.5.219, 172.67.73.117, 2606:4700:20::681a:4db, 2606:4700:20::681a:5db, 2606:4700:20::ac43:4975
Redirect IPs 104.26.4.219, 104.26.5.219, 172.67.73.117, 2606:4700:20::681a:4db, 2606:4700:20::681a:5db, 2606:4700:20::ac43:4975
Response IP 172.67.73.117
Found Yes
Hash ad8fe1a59c1689a2df869b7e614e3701b65ac40caf275483fd23cd522c9fb51d
SimHash 01419a134712

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin
Disallow /wp-includes