aspermont.com
robots.txt

Robots Exclusion Standard data for aspermont.com

Resource Scan

Scan Details

Site Domain aspermont.com
Base Domain aspermont.com
Scan Status Ok
Last Scan2026-04-08T22:35:13+00:00
Next Scan 2026-05-08T22:35:13+00:00

Last Scan

Scanned2026-04-08T22:35:13+00:00
URL https://aspermont.com/robots.txt
Redirect https://www.aspermont.com/robots.txt
Redirect Domain www.aspermont.com
Redirect Base aspermont.com
Domain IPs 104.21.27.89, 172.67.169.22, 2606:4700:3032::6815:1b59, 2606:4700:3033::ac43:a916
Redirect IPs 104.21.27.89, 172.67.169.22, 2606:4700:3032::6815:1b59, 2606:4700:3033::ac43:a916
Response IP 104.21.27.89
Found Yes
Hash ce9c1ede0397cc0e205144bbf90671f4d54e7a702021bf5df9b7ebe78144538e
SimHash 2445df526980

Groups

*

Rule Path
Disallow /search
Disallow /search_filter
Disallow /verify-site-login
Disallow /pdf/

Other Records

Field Value
crawl-delay 120

mj12bot
search
blexbot
semrushbot
semrushbot-bm
dotbot
barkrowler
moreover
seznambot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.aspermont.com/sitemap_index.xml
sitemap https://www.aspermont.com/news-sitemap.xml
sitemap https://www.aspermont.com/videositemap.xml

Comments

  • Sitemap declarations
  • Fully exclude these robots from crawling