industrialrelations.nsw.gov.au
robots.txt

Robots Exclusion Standard data for industrialrelations.nsw.gov.au

Resource Scan

Scan Details

Site Domain industrialrelations.nsw.gov.au
Base Domain nsw.gov.au
Scan Status Ok
Last Scan2024-06-10T23:03:02+00:00
Next Scan 2024-07-10T23:03:02+00:00

Last Scan

Scanned2024-06-10T23:03:02+00:00
URL https://industrialrelations.nsw.gov.au/robots.txt
Redirect https://www.industrialrelations.nsw.gov.au:443/robots.txt
Redirect Domain www.industrialrelations.nsw.gov.au
Redirect Base nsw.gov.au
Domain IPs 13.54.127.0, 3.24.150.173, 52.64.26.23
Redirect IPs 104.18.10.40, 104.18.11.40, 2606:4700::6812:a28, 2606:4700::6812:b28
Response IP 104.18.11.40
Found Yes
Hash e1118530a1fd42888c93bb828539caa8891fbec15b93b163fec66ec913a69ac6
SimHash 690c9c405d93

Groups

*

Rule Path
Allow /*.jpg
Allow /*.JPG
Allow /*.jpeg
Allow /*.JPEG
Allow /*.png
Allow /*.PNG
Allow /*.gif
Allow /*.GIF
Allow /*.svg
Allow /*.SVG
Disallow /long-service-leave-calculator
Disallow /assets
Disallow /search

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.industrialrelations.nsw.gov.au/sitemap.xml