theshayarthoughts.com
robots.txt

Robots Exclusion Standard data for theshayarthoughts.com

Resource Scan

Scan Details

Site Domain theshayarthoughts.com
Base Domain theshayarthoughts.com
Scan Status Ok
Last Scan2024-10-01T06:59:32+00:00
Next Scan 2024-10-08T06:59:32+00:00

Last Scan

Scanned2024-10-01T06:59:32+00:00
URL https://theshayarthoughts.com/robots.txt
Redirect https://www.theshayarthoughts.com/robots.txt
Redirect Domain www.theshayarthoughts.com
Redirect Base theshayarthoughts.com
Domain IPs 104.21.42.217, 172.67.166.154, 2606:4700:3030::ac43:a69a, 2606:4700:3036::6815:2ad9
Redirect IPs 104.21.42.217, 172.67.166.154, 2606:4700:3030::ac43:a69a, 2606:4700:3036::6815:2ad9
Response IP 172.67.166.154
Found Yes
Hash 826d24e81683ca6c4444b2476be951d07beaa604978cc00c0b7e28865c5979db
SimHash 4d14d6105793

Groups

*

Rule Path
Disallow /search
Disallow /category/
Disallow /tag/
Allow /

Other Records

Field Value
sitemap https://theshayarthoughts.com/atom.xml?redirect=false&start-index=1&max-results=500