Log In Sign Up

sinonim.org
robots.txt

Robots Exclusion Standard data for sinonim.org

Resource Scan

Scan Details

Site Domain sinonim.org
Base Domain sinonim.org
Scan Status Ok
Last Scan2024-11-16T09:16:42+00:00
Next Scan 2024-11-23T09:16:42+00:00

Last Scan

Scanned2024-11-16T09:16:42+00:00
URL https://sinonim.org/robots.txt
Response IP 46.36.217.250
Found Yes
Hash c355b51823f2fd843fd7b3cfc0337cad96d0750157d382747312c60287b9a9b6
SimHash 0b14106760ba

Groups

*

Rule Path
Disallow /contact
Disallow /privacy
Disallow /oferta
Disallow /switcher.php?opr
Disallow /switcher.php?ne_nashli
Disallow /switcher.php?opr&*
Disallow /switcher.php?predlTwo

mediapartners-google

Rule Path
Disallow

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

spbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

keys-so-bot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

semrushbot

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

spiderling

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

hubspot webcrawler

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

siteexplorer

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

cliqzbot

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

bluechipbacklinks

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

seekport crawler

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/

petalbot

Rule Path
Disallow /s/
Disallow /a/
Disallow /p/
Disallow /f/
Disallow /mo/
Disallow /as/

uptime

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000
Back to top

Warnings

  • `host` is not a known field.
Back to top