environmentalchemistry.com
robots.txt

Robots Exclusion Standard data for environmentalchemistry.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	environmentalchemistry.com
Base Domain	environmentalchemistry.com
Scan Status	Ok
Last Scan	2024-11-07T13:44:12+00:00
Next Scan	2024-12-07T13:44:12+00:00

Last Scan

Scanned	2024-11-07T13:44:12+00:00
URL	https://environmentalchemistry.com/robots.txt
Response IP	69.48.163.61
Found	Yes
Hash	b46cb4541b4f6b5175a816100c49b6b27c4286fd952931e2223e5089b5ce294a
SimHash	ad9fd2f86b9a

Groups

*

Rule	Path
Disallow	/yogi/awards.html
Disallow	/googlereplace.html
Disallow	/users/login.html
Disallow	/links/

Rule

Path

Disallow

/yogi/awards.html

Disallow

/googlereplace.html

Disallow

/users/login.html

Disallow

/links/

Other Records

Field	Value
crawl-delay	2

Field

Value

crawl-delay

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ptd-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

ppclabs_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

rorrimbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mlbot

Rule	Path
Disallow	/

Rule

Path

Disallow

nutch

Rule	Path
Disallow	/

Rule

Path

Disallow

converacrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

shopwiki

Rule	Path
Disallow	/

Rule

Path

Disallow

irlbot

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

webaroobot

Rule	Path
Disallow	/

Rule

Path

Disallow

fasterfox

Rule	Path
Disallow	/

Rule

Path

Disallow

sbider

Rule	Path
Disallow	/

Rule

Path

Disallow

bruinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

psbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

becomebot

Rule	Path
Disallow	/

Rule

Path

Disallow

dittospyder

Rule	Path
Disallow	/

Rule

Path

Disallow

psbot

Rule	Path
Disallow	/

Rule

Path

Disallow

msiecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

sapphirewebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

teleport

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

webstripper

Rule	Path
Disallow	/

Rule

Path

Disallow

websauger

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

netants

Rule	Path
Disallow	/

Rule

Path

Disallow

mister pix

Rule	Path
Disallow	/

Rule

Path

Disallow

webauto

Rule	Path
Disallow	/

Rule

Path

Disallow

thenomad

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

url_spider_pro

Rule	Path
Disallow	/

Rule

Path

Disallow

cherrypicker

Rule	Path
Disallow	/

Rule

Path

Disallow

emailcollector

Rule	Path
Disallow	/

Rule

Path

Disallow

emailsiphon

Rule	Path
Disallow	/

Rule

Path

Disallow

webbandit

Rule	Path
Disallow	/

Rule

Path

Disallow

emailwolf

Rule	Path
Disallow	/

Rule

Path

Disallow

extractorpro

Rule	Path
Disallow	/

Rule

Path

Disallow

sitesnagger

Rule	Path
Disallow	/

Rule

Path

Disallow

prowebwalker

Rule	Path
Disallow	/

Rule

Path

Disallow

cugillionbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yacybot

Rule	Path
Disallow	/

Rule

Path

Disallow

Warnings

2 invalid lines.

environmentalchemistry.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

Other Records

ahrefsbot

ptd-crawler

ppclabs_bot

rorrimbot

mlbot

nutch

converacrawler

dotbot

shopwiki

irlbot

httrack

webaroobot

fasterfox

sbider

bruinbot

psbot

ia_archiver

becomebot

dittospyder

psbot

msiecrawler

sapphirewebcrawler

teleport

teleportpro

webzip

webstripper

websauger

webcopier

netants

mister pix

webauto

thenomad

webzip

url_spider_pro

cherrypicker

emailcollector

emailsiphon

webbandit

emailwolf

extractorpro

sitesnagger

prowebwalker

cugillionbot

yacybot

Warnings

environmentalchemistry.com
robots.txt