firstscene.co.nz
robots.txt
Robots Exclusion Standard data for firstscene.co.nz
Resource Scan
Scan Details
| Site Domain | firstscene.co.nz |
| Base Domain | firstscene.co.nz |
| Scan Status | Ok |
| Last Scan | 2025-06-04T06:00:52+00:00 |
| Next Scan | 2025-07-04T06:00:52+00:00 |
Last Scan
| Scanned | 2025-06-04T06:00:52+00:00 |
| URL | https://firstscene.co.nz/robots.txt |
| Redirect | https://www.firstscene.co.nz/robots.txt |
| Redirect Domain | www.firstscene.co.nz |
| Redirect Base | firstscene.co.nz |
| Domain IPs | 3.105.162.160 |
| Redirect IPs | 3.105.162.160 |
| Response IP | 3.105.162.160 |
| Found | Yes |
| Hash | af1b016e97cb9ca18e92edfecf9e841b531921c40a6e850bdd795ee86fe1deba |
| SimHash | 5bebeac4af17 |
Groups
road runner: imagescape robot (lim@cs.leidenuniv.nl)
road runner: imagescape robot
roadrunner
| Rule | Path |
|---|---|
| Disallow | /webapps/ |
nlnz_iaharvester2024
| Rule | Path |
|---|---|
| Disallow | /product/ |
| Disallow | /page/500486/shopping/cart/ |
| Disallow | /page/500486/shopping/productview/ |
| Disallow | /*?* |
| Disallow | /*/productview/ |
*
| Rule | Path |
|---|---|
| Disallow | /page/500486/shopping/cart/ |
| Disallow | /page/500486/shopping/productview/ |
| Disallow | /*?*b_id= |
| Disallow | /*?*add2cart= |
| Disallow | /*?*add2wishlist= |
| Disallow | /*/shopping/cart/ |
| Disallow | /*shopping-basket |
| Disallow | /*delete_basket |
| Disallow | /*copyStyleFromCid%3D |
| Disallow | /*new-website-contact.html |
| Disallow | /webapps/templates |
| Disallow | /webapps/templates/ |
| Disallow | /webapps/templates/track |
| Disallow | /webapps/cms/ |
| Disallow | /webapps/cid/ |
| Disallow | /thumbs/ |
| Disallow | /img/ |
| Disallow | /templates/ |
| Disallow | /banners/ |
| Disallow | /Dictionary/ |
| Disallow | /_borders/ |
| Disallow | /_fpclass/ |
| Disallow | /_overlay/ |
| Disallow | /_private/ |
| Disallow | /_themes/ |
| Disallow | /cgi |
| Disallow | /widgets/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 3 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.firstscene.co.nz/sitemap.xml |