oldhousedreams.com
robots.txt

Robots Exclusion Standard data for oldhousedreams.com

Resource Scan

Scan Details

Site Domain oldhousedreams.com
Base Domain oldhousedreams.com
Scan Status Ok
Last Scan2024-11-14T05:20:15+00:00
Next Scan 2024-11-21T05:20:15+00:00

Last Scan

Scanned2024-11-14T05:20:15+00:00
URL https://oldhousedreams.com/robots.txt
Domain IPs 104.21.93.231, 172.67.216.134, 2606:4700:3031::ac43:d886, 2606:4700:3036::6815:5de7
Response IP 104.21.93.231
Found Yes
Hash 159eaf46e34c56a7474b8188722cb9016f234e04332853bfa9f2bfbe0de433a1
SimHash 6e14de42cf8d

Groups

googlebot

Rule Path
Allow /wp-includes/

googlebot

Rule Path
Allow /wp-content/
Disallow /Trap/Bots.aspx

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

teoma

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

scrubby

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

nutch

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /readme.html
Disallow /trackback/
Disallow /xmlrpc.php

Other Records

Field Value
crawl-delay 500

*

Rule Path
Disallow /wp-content/uploads/wp-import-export-lite/

Other Records

Field Value
sitemap https://www.oldhousedreams.com/post-sitemap1.xml
sitemap https://www.oldhousedreams.com/post-sitemap2.xml
sitemap https://www.oldhousedreams.com/post-sitemap3.xml
sitemap https://www.oldhousedreams.com/post-sitemap4.xml
sitemap https://www.oldhousedreams.com/page-sitemap.xml
sitemap https://www.oldhousedreams.com/category-sitemap.xml
sitemap https://www.oldhousedreams.com/state-sitemap.xml
sitemap https://www.oldhousedreams.com/style-sitemap.xml
sitemap https://www.oldhousedreams.com/periods-sitemap.xml
sitemap https://www.oldhousedreams.com/architects-sitemap.xml
sitemap https://www.oldhousedreams.com/misc-sitemap.xml
sitemap https://www.oldhousedreams.com/region-sitemap.xml
sitemap https://www.oldhousedreams.com/features-sitemap.xml

Comments

  • robots.txt
  • WP Import Export Rule