seekbusiness.com.au
robots.txt

Robots Exclusion Standard data for seekbusiness.com.au

Resource Scan

Scan Details

Site Domain seekbusiness.com.au
Base Domain seekbusiness.com.au
Scan Status Ok
Last Scan2024-06-28T07:23:12+00:00
Next Scan 2024-07-05T07:23:12+00:00

Last Scan

Scanned2024-06-28T07:23:12+00:00
URL https://seekbusiness.com.au/robots.txt
Redirect https://www.seekbusiness.com.au/robots.txt
Redirect Domain www.seekbusiness.com.au
Redirect Base seekbusiness.com.au
Domain IPs 52.84.229.105, 52.84.229.68, 52.84.229.83, 52.84.229.86
Redirect IPs 52.84.229.105, 52.84.229.68, 52.84.229.83, 52.84.229.86
Response IP 52.84.229.105
Found Yes
Hash e245656ecb52ceb3090500b57902a16193283c745ddce97e26b5d264ebd25610
SimHash 2a5a9617c6fc

Groups

*

Rule Path
Disallow /graphql

bingbot

Rule Path
Disallow /graphql

Other Records

Field Value
crawl-delay 4

yottaamonitor

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /graphql

seekportbot

Rule Path
Disallow /graphql

Other Records

Field Value
crawl-delay 4

Comments

  • Robots.txt file for www.seekbusiness.com.au
  • NB: Urls are case sensitive
  • default
  • Bingbot can be very aggressive. Tell it to backoff a bit
  • YottaMonitor
  • Google Ad Sense
  • Seekport Bot