aerospacedefensereview.com
robots.txt

Robots Exclusion Standard data for aerospacedefensereview.com

Resource Scan

Scan Details

Site Domain aerospacedefensereview.com
Base Domain aerospacedefensereview.com
Scan Status Ok
Last Scan2025-06-02T23:34:47+00:00
Next Scan 2025-06-09T23:34:47+00:00

Last Scan

Scanned2025-06-02T23:34:47+00:00
URL https://aerospacedefensereview.com/robots.txt
Redirect https://www.aerospacedefensereview.com/robots.txt
Redirect Domain www.aerospacedefensereview.com
Redirect Base aerospacedefensereview.com
Domain IPs 52.34.187.179
Redirect IPs 52.34.187.179
Response IP 52.34.187.179
Found Yes
Hash ef3a7f5d8026d97081093076c715472e5ac4fe7cb886021a206a6e3d89d91fd6
SimHash 3126ded3c691

Groups

*

Rule Path Comment
Disallow / Disallow all bots by default
Disallow /scripts/ -
Disallow /*.php$ -
Disallow /zlib/ -
Disallow /lib/ -
Disallow /utils/ -
Disallow /magazine/ -
Disallow /admin/ -

*

Rule Path
Disallow /magazine/

googlebot
googlebot-image
mediapartners-google
adsbot-google
googlebot-mobile
msnbot
slurp
msicrawler
baiduspider
bingbot
seositecheckupbot
screaming frog seo spider
yandexbot
linkedinbot
twitterbot
ahrefsbot

Rule Path
Allow /

ahrefssiteaudit

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.aerospacedefensereview.com/aerospace_news_sitemap.xml
sitemap https://www.aerospacedefensereview.com/aerospacedefensereviewnewnews_sitemap.xml
sitemap https://www.aerospacedefensereview.com/vendorviewpoint_sitemap.xml
sitemap https://www.aerospacedefensereview.com/vertical_sitemap.xml
sitemap https://www.aerospacedefensereview.com/all_in_one_sitemap_main.xml
sitemap https://www.aerospacedefensereview.com/contributors_sitemap.xml
sitemap https://www.aerospacedefensereview.com/aerospace_vendor_sitemap_new.xml
sitemap https://www.aerospacedefensereview.com/rankingxml_sitemap_new.xml

Comments

  • Allow major search engines
  • Allow specific tools
  • Allow AhrefsBot
  • Allow AhrefsSiteAudit