constructiontechreviewapac.com
robots.txt

Robots Exclusion Standard data for constructiontechreviewapac.com

Resource Scan

Scan Details

Site Domain constructiontechreviewapac.com
Base Domain constructiontechreviewapac.com
Scan Status Ok
Last Scan2024-10-30T01:14:07+00:00
Next Scan 2024-11-06T01:14:07+00:00

Last Scan

Scanned2024-10-30T01:14:07+00:00
URL https://constructiontechreviewapac.com/robots.txt
Redirect https://www.constructiontechreviewapac.com/robots.txt
Redirect Domain www.constructiontechreviewapac.com
Redirect Base constructiontechreviewapac.com
Domain IPs 54.71.35.46
Redirect IPs 54.71.35.46
Response IP 54.71.35.46
Found Yes
Hash de70828fb19a7fd21aa1fa76cbf1859bbf41ab096cadb6377cfb82042a980657
SimHash 2914dc13c611

Groups

*

Rule Path Comment
Disallow / Disallow all bots by default
Disallow /scripts/ -
Disallow /*.php$ -
Disallow /zlib/ -
Disallow /lib/ -
Disallow /utils/ -
Disallow /magazine/ -
Disallow /admin/ -

googlebot
googlebot-image
mediapartners-google
adsbot-google
googlebot-mobile
msnbot
slurp
msicrawler
baiduspider
bingbot
seositecheckupbot
screaming frog seo spider
yandexbot
linkedinbot
twitterbot
ahrefsbot

Rule Path
Allow /

ahrefssiteaudit

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.constructiontechreview.com/construction_news_sitemap.xml
sitemap https://www.constructiontechreview.com/constructiontechreviewnewnews_sitemap.xml
sitemap https://www.constructiontechreview.com/vendorviewpoint_sitemap.xml
sitemap https://www.constructiontechreview.com/all_in_one_sitemap_main.xml
sitemap https://www.constructiontechreview.com/contributors_sitemap.xml

Comments

  • Allow major search engines
  • Allow specific tools
  • Allow AhrefsBot
  • Allow AhrefsSiteAudit