themalaysiapost.com
robots.txt

Robots Exclusion Standard data for themalaysiapost.com

Resource Scan

Scan Details

Site Domain themalaysiapost.com
Base Domain themalaysiapost.com
Scan Status Ok
Last Scan4/21/2025, 5:15:58 PM
Next Scan 4/28/2025, 5:15:58 PM

Last Scan

Scanned4/21/2025, 5:15:58 PM
URL https://themalaysiapost.com/robots.txt
Domain IPs 104.21.40.110, 172.67.150.189, 2606:4700:3033::6815:286e, 2606:4700:3034::ac43:96bd
Response IP 172.67.150.189
Found Yes
Hash 15270964a2bc93312fd920795dd530efecacd2891f0fcb1d1701b5ad8b2be93e
SimHash 6b09dcc1e693

Groups

*

Rule Path
Disallow /wp-content/plugins/
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /tag/

Other Records

Field Value
sitemap https://themalaysiapost.com/sitemaps.xml

Comments

  • hestiacp autogenerated robots.txt