Log In Sign Up

doitcenter.com.pa
robots.txt

Robots Exclusion Standard data for doitcenter.com.pa

Resource Scan

Scan Details

Site Domain doitcenter.com.pa
Base Domain doitcenter.com.pa
Scan Status Ok
Last Scan2024-06-25T09:00:44+00:00
Next Scan 2024-07-09T09:00:44+00:00

Last Scan

Scanned2024-06-25T09:00:44+00:00
URL https://doitcenter.com.pa/robots.txt
Redirect https://www.doitcenter.com.pa/robots.txt
Redirect Domain www.doitcenter.com.pa
Redirect Base doitcenter.com.pa
Domain IPs 18.233.19.83
Redirect IPs 13.35.18.104, 13.35.18.107, 13.35.18.28, 13.35.18.54
Response IP 13.35.18.104
Found Yes
Hash a28fe633ab19fe994208ba44b030cf789b883e6f060966247128084f1a813bfe
SimHash 6576d503d132

Groups

*

Rule Path
Allow /*graphql
Allow /*static
Allow /*media
Allow /*data
Disallow /checkout
Disallow /cart
Disallow /account
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pub/
Disallow /tag/
Disallow /*.git
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.Svn$
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Tgz$
Disallow *nosto%3D

mediapartners-google

Rule Path
Allow /*graphql
Allow /*static
Allow /*media
Allow /*data

storebot-google

Rule Path
Allow /*graphql
Allow /*static
Allow /*media
Allow /*data
Disallow *nosto%3D

adsbot-google

Rule Path
Allow /*graphql
Allow /*static
Allow /*media
Allow /*data
Disallow /checkout
Disallow /cart
Disallow /account

baiduspider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

asterias

Rule Path
Disallow /

yahoo-blogs/v3.9

Rule Path
Disallow /

yandex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-ba

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-si

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-swa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-ct

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

splitsignalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-coub

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://www.doitcenter.com.pa/sitemap.xml
Back to top

Comments

  • Allowed Resources
  • Account-Checkout-Search
  • Magneto File System
  • Nosto
  • Google's Adsense
  • Google's Storebot
  • Google's Adbot
  • Chinese Bots
  • Crawl Delays
  • XML Sitemaps
  • ===================#############################====
  • ======######=======#########################======######==
  • ========######=======########################====+####+===
  • ========######========#############################+======###+
  • ========###===========+######=======++========+
  • ========#=====#####=====+#####=====####=====
  • =============#######=====#####=====####=====
  • ========######=============#######======####=====####=====
  • ======#########+=====#=====#######+====+####=====####=====
  • ======#########+===###=====#######=====#####=====####=====
  • ==================#######=====###+====+######+====+###=====+++=
  • ================###############*+=====########==========##=====+
  • ==
  • *+===+####======#####==##+===*###=======##=====######==##=+
  • ==####===#===####===#+====###====##====###==#####===#=========
  • ====#######===========+#===#####===+#====##===========#====
  • ====#######===+#########===#####===+#====##===#########====
  • ====######====*######+#===#####===+#====##====######*#====####+=
  • ========###========##======##======#======*========#+======#=+=+
Back to top

Warnings

  • 2 invalid lines.
Back to top