cremicro.com
robots.txt

Robots Exclusion Standard data for cremicro.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	cremicro.com
Base Domain	cremicro.com
Scan Status	Ok
Last Scan	2024-06-22T01:28:15+00:00
Next Scan	2024-06-29T01:28:15+00:00

Last Scan

Scanned	2024-06-22T01:28:15+00:00
URL	https://cremicro.com/robots.txt
Domain IPs	104.21.38.22, 172.67.218.8, 2606:4700:3031::ac43:da08, 2606:4700:3035::6815:2616
Response IP	104.21.38.22
Found	Yes
Hash	b87ea3ee1aa4398cc8347e06b24deddc0a817a2907cca455415d7dfd7945fd85
SimHash	4b52400049b3

Groups

alexibot

Rule	Path
Disallow	/

Rule

Path

Disallow

aqua_products

Rule	Path
Disallow	/

Rule

Path

Disallow

backdoorbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

backlinkcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

bookmark search tool

Rule	Path
Disallow	/

Rule

Path

Disallow

botalot

Rule	Path
Disallow	/

Rule

Path

Disallow

bullseye/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

copyrightcheck

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

download ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack 3.0

Rule	Path
Disallow	/

Rule

Path

Disallow

keyword density/0.9

Rule	Path
Disallow	/

Rule

Path

Disallow

linkscan/8.1a unix

Rule	Path
Disallow	/

Rule

Path

Disallow

linkwalker

Rule	Path
Disallow	/

Rule

Path

Disallow

mister pix

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

mlbot

Rule	Path
Disallow	/

Rule

Path

Disallow

openbot

Rule	Path
Disallow	/

Rule

Path

Disallow

propowerbot/2.14

Rule	Path
Disallow	/

Rule

Path

Disallow

prowebwalker

Rule	Path
Disallow	/

Rule

Path

Disallow

rma

Rule	Path
Disallow	/

Rule

Path

Disallow

surveybot

Rule	Path
Disallow	/

Rule

Path

Disallow

teleport

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

true_robot

Rule	Path
Disallow	/

Rule

Path

Disallow

true_robot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

url control

Rule	Path
Disallow	/

Rule

Path

Disallow

url_spider_pro

Rule	Path
Disallow	/

Rule

Path

Disallow

webcapture 2.0

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier v.2.2

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier v3.2a

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip/4.0

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip/4.21

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip/5.0

Rule	Path
Disallow	/

Rule

Path

Disallow

xenu

Rule	Path
Disallow	/

Rule

Path

Disallow

xenuâ€™s

Rule	Path
Disallow	/

Rule

Path

Disallow

xenuâ€™s link sleuth 1.1c

Rule	Path
Disallow	/

Rule

Path

Disallow

yeti

Rule	Path
Disallow	/

Rule

Path

Disallow

ranksignals

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/wp-admin/
Allow	/wp-admin/admin-ajax.php
Allow	/wp-content/uploads/

Rule

Path

Disallow

/wp-admin/

Allow

/wp-admin/admin-ajax.php

Allow

/wp-content/uploads/

Other Records

Field	Value
sitemap	https://cremicro.com/sitemap_index.xml

Field

Value

sitemap

https://cremicro.com/sitemap_index.xml

cremicro.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

alexibot

aqua_products

backdoorbot/1.0

backlinkcrawler

bookmark search tool

botalot

bullseye/1.0

copyrightcheck

dotbot

download ninja

httrack

httrack 3.0

keyword density/0.9

linkscan/8.1a unix

linkwalker

mister pix

mj12bot

mlbot

openbot

propowerbot/2.14

prowebwalker

rma

surveybot

teleport

teleportpro

true_robot

true_robot/1.0

url control

url_spider_pro

webcapture 2.0

webcopier

webcopier v.2.2

webcopier v3.2a

webzip

webzip/4.0

webzip/4.21

webzip/5.0

xenu

xenuâ€™s

xenuâ€™s link sleuth 1.1c

yeti

ranksignals

*

Other Records

cremicro.com
robots.txt