wdtl.com
robots.txt

Robots Exclusion Standard data for wdtl.com

Resource Scan

Scan Details

Site Domain wdtl.com
Base Domain wdtl.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-02T10:16:08+00:00
Next Scan 2024-08-31T10:16:08+00:00

Last Successful Scan

Scanned2023-10-14T05:03:37+00:00
URL https://wdtl.com/robots.txt
Redirect https://www.aofrio.com/robots.txt
Redirect Domain www.aofrio.com
Redirect Base aofrio.com
Domain IPs 13.211.192.195
Redirect IPs 13.226.120.10, 13.226.120.108, 13.226.120.14, 13.226.120.77, 2600:9000:223b:1600:0:aa01:a000:93a1, 2600:9000:223b:4400:0:aa01:a000:93a1, 2600:9000:223b:4600:0:aa01:a000:93a1, 2600:9000:223b:6e00:0:aa01:a000:93a1, 2600:9000:223b:9000:0:aa01:a000:93a1, 2600:9000:223b:9c00:0:aa01:a000:93a1, 2600:9000:223b:bc00:0:aa01:a000:93a1, 2600:9000:223b:f200:0:aa01:a000:93a1
Response IP 13.33.88.101
Found Yes
Hash e7268a7bb919c1ecdf522feefd430fb4f4f7cbf1e104c778e56ed9b0c265b09d
SimHash 4910db02c78b

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php/
Allow /wp-admin/admin-ajax.php

googlebot

Rule Path
Disallow

slurp

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

bingbot

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

ninjabot

Rule Path
Disallow

msnbot

Rule Path
Disallow

yandexbot

Rule Path
Disallow

baiduspider

Rule Path
Disallow

naverbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

teoma

Rule Path
Disallow /

slurp

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

corruptbot

Rule Path
Disallow /

litefinder

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.aofrio.com/sitemap_index.xml

Warnings

  • 2 invalid lines.