Log In Sign Up

almancaabc.com
robots.txt

Robots Exclusion Standard data for almancaabc.com

Resource Scan

Scan Details

Site Domain almancaabc.com
Base Domain almancaabc.com
Scan Status Ok
Last Scan2024-09-18T23:56:13+00:00
Next Scan 2024-09-25T23:56:13+00:00

Last Scan

Scanned2024-09-18T23:56:13+00:00
URL https://almancaabc.com/robots.txt
Domain IPs 185.149.100.50
Response IP 185.149.100.50
Found Yes
Hash 580f10d63cab42406dac39c4ddba40f6356bed82268abcfd619dbd6bdace076b
SimHash 0f74d345c556

Groups

anthropic-ai

Rule Path
Disallow /

bloodhound

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

cydralspider

Rule Path
Disallow /

downloadexpress

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

gammaspider

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

pimptrain

Rule Path
Disallow /

raven

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

wapspider

Rule Path
Disallow /

webzinger

Rule Path
Disallow /

googlebot

Rule Path
Disallow */*/feed
Disallow *?share=

mediapartners-google

Rule Path
Disallow */*/feed
Disallow *?share=

slurp

Rule Path
Disallow */*/feed
Disallow *?share=

bingbot

Rule Path
Disallow */*/feed
Disallow *?share=

msnbot

Rule Path
Disallow */*/feed
Disallow *?share=

baiduspider

Rule Path
Disallow */*/feed
Disallow *?share=

naverbot

Rule Path
Disallow */*/feed
Disallow *?share=

seznambot

Rule Path
Disallow */*/feed
Disallow *?share=

yandex

Rule Path
Disallow */*/feed
Disallow *?share=

ia_archiver

Rule Path
Disallow

twitterbot

Rule Path
Disallow

facebookexternalhit/1.0

Rule Path
Disallow

facebookexternalhit/1.1

Rule Path
Disallow

facebot

Rule Path
Disallow

linkedinbot/1.0

Rule Path
Disallow

*

Rule Path
Disallow /
Allow /wp-content/uploads/
Disallow /readme.html
Disallow /oneri
Disallow /sorular/
Disallow /?ref=
Disallow /*ref%3D
Disallow /refer/
Disallow /geh/
Disallow /wp-admin/
Disallow /rel/
Back to top

Other Records

Field Value
sitemap https://almancaabc.com/sitemap_index.xml
sitemap https://almancaabc.com/post-sitemap.xml
sitemap https://almancaabc.com/page-sitemap.xml
sitemap https://almancaabc.com/glossary-sitemap.xml
sitemap https://almancaabc.com/video-sitemap.xml
sitemap https://almancaabc.com/category-sitemap.xml
Back to top

Comments

  • Sitemaps
  • Specific bots with Disallow rules
  • Google bots
  • Yahoo bot
  • Bing/MSN bots
  • Baidu bot
  • Naver bot
  • Seznam bot
  • Yandex bot
  • Internet Archive
  • Twitter bot
  • Facebook bots
  • LinkedIn bot
  • General bots and rules
Back to top