reviewments.com
robots.txt

Robots Exclusion Standard data for reviewments.com

Resource Scan

Scan Details

Site Domain reviewments.com
Base Domain reviewments.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-07T09:51:28+00:00
Next Scan 2024-12-06T09:51:28+00:00

Last Successful Scan

Scanned2024-08-09T09:41:18+00:00
URL https://reviewments.com/robots.txt
Domain IPs 2a02:4780:84:ee63:20aa:62cc:e4d9:d6ba, 93.127.201.11
Response IP 77.37.75.215
Found Yes
Hash d32403245e6bc59cb48bbca3d45b198653f8c75d45bd15c72360f155a91c4efa
SimHash 58269930c1e7

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

googlebot

Rule Path
Allow /

googleother

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

bingbot

Rule Path
Allow /

google-extended

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

applebot

Rule Path
Allow /

applebot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

facebot

Rule Path
Allow /

slurp

Rule Path
Allow /

gptbot

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

oncrawl

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bytespider

Rule Path
Disallow /
Disallow /?blackhole

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://reviewments.com/sitemap_index.xml