arsnet.org
robots.txt

Robots Exclusion Standard data for arsnet.org

Resource Scan

Scan Details

Site Domain arsnet.org
Base Domain arsnet.org
Scan Status Ok
Last Scan2025-10-01T14:12:07+00:00
Next Scan 2025-10-31T14:12:07+00:00

Last Scan

Scanned2025-10-01T14:12:07+00:00
URL https://arsnet.org/robots.txt
Redirect https://www.arsnet.org/robots.txt
Redirect Domain www.arsnet.org
Redirect Base arsnet.org
Domain IPs 185.154.138.65
Redirect IPs 185.154.138.65
Response IP 185.154.138.65
Found Yes
Hash d8633eb901bfd6353958e6c08019e29021a702c101a97a2a4a1e7c765db6bd2e
SimHash 230c88026e10

Groups

*

Rule Path
Disallow /secupress-1d22b588/

*

Rule Path
Disallow /*add-to-cart%3D*

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.arsnet.org/sitemaps.xml