ansa.org.ve
robots.txt

Robots Exclusion Standard data for ansa.org.ve

Resource Scan

Scan Details

Site Domain ansa.org.ve
Base Domain ansa.org.ve
Scan Status Ok
Last Scan5/22/2025, 1:34:39 PM
Next Scan 6/21/2025, 1:34:39 PM

Last Scan

Scanned5/22/2025, 1:34:39 PM
URL https://ansa.org.ve/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.16.1
Found Yes
Hash 041f9eb8903ce51ac99c0716db6c4ce30a45515295e8635ccefc795e54d6eb94
SimHash 7f165972fa96

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-login.php
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/ai1wm-backups/
Disallow /wp-content/
Disallow /zohoverify/
Disallow /xmlrpc.php

Other Records

Field Value
crawl-delay 5

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

bot[\s_+:,\.\;\/\\-]

Rule Path
Disallow /

python-[\s_+:,\.\;\/\\-]

Rule Path
Disallow /

python[\s_+:,\.\;\/\\-]

Rule Path
Disallow /

python-requests[\s_+:,\.\;\/\\-]

Rule Path
Disallow /

python-requests/2.18.4

Rule Path
Disallow /

python-requests/2.19.1

Rule Path
Disallow /

python-requests/2.20.0

Rule Path
Disallow /

python-requests/2.21.0

Rule Path
Disallow /

python-requests/2.22.0

Rule Path
Disallow /

python-requests/2.24.0

Rule Path
Disallow /

python-requests/2.25.1

Rule Path
Disallow /

python-requests/2.26.0

Rule Path
Disallow /

python-requests/2.27.1

Rule Path
Disallow /

python-requests/2.28.1

Rule Path
Disallow /

python-requests/2.28.2

Rule Path
Disallow /

python-requests/2.30.0

Rule Path
Disallow /

python-requests/2.31.0

Rule Path
Disallow /

internetmeasurement/1.0

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

notion

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

ms-office

Rule Path
Disallow /

got

Rule Path
Disallow /

-

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

googlebot

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-login.php
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/ai1wm-backups/
Disallow /wp-content/
Disallow /zohoverify/

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.ansa.org.ve/sitemap_index.xml

Warnings

  • 2 invalid lines.