listafirme.ro
robots.txt

Robots Exclusion Standard data for listafirme.ro

Resource Scan

Scan Details

Site Domain listafirme.ro
Base Domain listafirme.ro
Scan Status Ok
Last Scan2024-08-31T22:39:56+00:00
Next Scan 2024-09-30T22:39:56+00:00

Last Scan

Scanned2024-08-31T22:39:56+00:00
URL https://listafirme.ro/robots.txt
Redirect https://www.listafirme.ro/robots.txt
Redirect Domain www.listafirme.ro
Redirect Base listafirme.ro
Domain IPs 86.35.0.61
Redirect IPs 86.35.0.61
Response IP 86.35.0.61
Found Yes
Hash dd08900c30717eb6cb04fb51bc649bc1fa89782c1a0ec8d4bbafdc90c352b630
SimHash 7316d0d0cba1

Groups

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

seekbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkdex

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

aipbot

Rule Path
Disallow /

spider traficdublu

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

paracrawl

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /*/

slurp

Rule Path
Disallow /*/

applebot

Rule Path
Disallow /*/

exabot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /*/

gptbot

Rule Path
Disallow /*/

oai-searchbot

Rule Path
Disallow /*/

ia_archiver

Rule Path
Disallow /*/

amazonbot

Rule Path
Disallow /*/

*

Rule Path
Disallow /inc/
Disallow /frontdesk/
Disallow /ajax/
Disallow /api/

Warnings

  • 4 invalid lines.