Log In Sign Up

securityexpo.com.au
robots.txt

Robots Exclusion Standard data for securityexpo.com.au

Resource Scan

Scan Details

Site Domain securityexpo.com.au
Base Domain securityexpo.com.au
Scan Status Ok
Last Scan2025-08-28T01:17:21+00:00
Next Scan 2025-09-27T01:17:21+00:00

Last Scan

Scanned2025-08-28T01:17:21+00:00
URL https://securityexpo.com.au/robots.txt
Redirect https://www.securityexpo.com.au/robots.txt
Redirect Domain www.securityexpo.com.au
Redirect Base securityexpo.com.au
Domain IPs 104.21.89.92, 172.67.139.137, 2606:4700:3031::6815:595c, 2606:4700:3037::ac43:8b89
Redirect IPs 104.21.89.92, 172.67.139.137, 2606:4700:3031::6815:595c, 2606:4700:3037::ac43:8b89
Response IP 104.21.89.92
Found Yes
Hash 152990916a1d80eefa9b13bfc721c3d4ff14065e4c2419c627550a6cc920851d
SimHash 625fd251e503

Groups

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

sogou spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

screaming frog seo spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Comments

  • Bing & Yahoo
  • Yandex
  • Chinese search engines
  • Aggressive SEO crawlers
  • Screaming Frog SEO Spider
  • Generic fallback
Back to top