bellenza.com
robots.txt

Robots Exclusion Standard data for bellenza.com

Resource Scan

Scan Details

Site Domain bellenza.com
Base Domain bellenza.com
Scan Status Ok
Last Scan2024-09-23T12:46:46+00:00
Next Scan 2024-09-30T12:46:46+00:00

Last Scan

Scanned2024-09-23T12:46:46+00:00
URL https://bellenza.com/robots.txt
Redirect https://www.bellenza.com/robots.txt
Redirect Domain www.bellenza.com
Redirect Base bellenza.com
Domain IPs 185.151.30.194
Redirect IPs 185.151.30.194
Response IP 185.151.30.194
Found Yes
Hash 1bb5894764a1d911b064c3fa521a40e7d769422bcdc4f5f2d408916bf80cc5f2
SimHash 2c382249c8f9

Groups

*

Rule Path
Disallow /mm5/
Disallow /mm5/*/
Disallow /mm5*/*/
Disallow /Merchant2/
Disallow /cgi-bin/
Disallow /manager/
Disallow /fm/
Disallow /users/
Disallow /vds-backup/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /tag/
Disallow /pma/
Disallow /wpm/
Disallow /plug-ins/
Disallow /wp-*
Disallow /search
Disallow /print
Disallow /test/
Disallow /Enlarged_Views/

googlebot

Rule Path
Disallow
Disallow /
Disallow /
Disallow /
Disallow /
Disallow /
Disallow /
Disallow /
Disallow /

baiduspider

Rule Path
Disallow /
Disallow /

sbider

Rule Path
Disallow /
Disallow /
Disallow /

surveybot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

favorstarbot

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

lwp-trivial/1.34

Rule Path
Disallow /

exabot

Rule Path
Disallow /

generic

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

mcbot

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

psbot

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

shopwiki

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

linksmanager_bot

Rule Path
Disallow /

favorstarbot

Rule Path
Disallow /

*

Rule Path
Disallow /*.php$
Disallow /*.js$
Disallow /*.inc$
Disallow /*.css$
Disallow /*.gz$
Disallow /*.cgi$

Comments

  • disallow files in these folders
  • bots to control
  • disallow files with these extensions

Warnings

  • `useragent` is not a known field.