isitnormal.com
robots.txt

Robots Exclusion Standard data for isitnormal.com

Resource Scan

Scan Details

Site Domain isitnormal.com
Base Domain isitnormal.com
Scan Status Ok
Last Scan2024-09-19T21:32:34+00:00
Next Scan 2024-09-26T21:32:34+00:00

Last Scan

Scanned2024-09-19T21:32:34+00:00
URL https://isitnormal.com/robots.txt
Redirect https://www.isitnormal.com/robots.txt
Redirect Domain www.isitnormal.com
Redirect Base isitnormal.com
Domain IPs 104.21.47.80, 172.67.170.199, 2606:4700:3030::ac43:aac7, 2606:4700:3035::6815:2f50
Redirect IPs 104.21.47.80, 172.67.170.199, 2606:4700:3030::ac43:aac7, 2606:4700:3035::6815:2f50
Response IP 104.21.47.80
Found Yes
Hash bfda8906f8b0a8c34aa7d52028a5a2c41d3a69095f67644b8d305d7e096bf2d3
SimHash 4e059e30c182

Groups

*

Rule Path
Disallow /post*/comment*
Disallow /post*/report
Disallow /comment/*
Disallow /sign-in
Disallow /sign-up
Disallow /sign-out
Disallow /api/
Disallow /admin/
Disallow /js/
Disallow /scripts/
Disallow /posts/new
Disallow /settings
Disallow /search
Disallow /notices
Disallow /member/
Disallow /member/*
Disallow /random
Disallow /session*
Disallow /story/*/report-comment*
Disallow /poll/*/report-comment*

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15