urbanasia.com
robots.txt

Robots Exclusion Standard data for urbanasia.com

Resource Scan

Scan Details

Site Domain urbanasia.com
Base Domain urbanasia.com
Scan Status Ok
Last Scan2024-11-18T06:26:51+00:00
Next Scan 2024-11-25T06:26:51+00:00

Last Scan

Scanned2024-11-18T06:26:51+00:00
URL https://www.urbanasia.com/robots.txt
Domain IPs 104.21.64.130, 172.67.185.214, 2606:4700:3031::6815:4082, 2606:4700:3035::ac43:b9d6
Response IP 104.21.64.130
Found Yes
Hash d4584994c0ebc039788a0d06ff9839d6c39f6cd652ac0a44b26ba494386e34f1
SimHash cd2c5d052fb3

Groups

*

Rule Path
Disallow /?*
Disallow /tag/

Other Records

Field Value
sitemap https://www.urbanasia.com/sitemap.xml
sitemap https://www.urbanasia.com/sitemap/sitemap-general.xml
sitemap https://www.urbanasia.com/sitemap/sitemap-posts.xml