classbento.com.au
robots.txt

Robots Exclusion Standard data for classbento.com.au

Resource Scan

Scan Details

Site Domain classbento.com.au
Base Domain classbento.com.au
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-07T02:26:46+00:00
Next Scan 2025-10-06T02:26:46+00:00

Last Successful Scan

Scanned2025-05-16T23:10:28+00:00
URL https://classbento.com.au/robots.txt
Domain IPs 172.66.41.37, 172.66.42.219
Response IP 172.66.41.37
Found Yes
Hash c1dade236de55e30e6b8b8f9891a4ffb7117a391a11a00a0f020715e54ece62c
SimHash 631c9c526293

Groups

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

awariorssbot
awariosmartbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://classbento.com.au/sitemap.xml