discountsex24.com
robots.txt

Robots Exclusion Standard data for discountsex24.com

Resource Scan

Scan Details

Site Domain discountsex24.com
Base Domain discountsex24.com
Scan Status Ok
Last Scan2024-10-17T04:51:40+00:00
Next Scan 2024-11-16T04:51:40+00:00

Last Scan

Scanned2024-10-17T04:51:40+00:00
URL https://discountsex24.com/robots.txt
Redirect https://www.discountsex24.com/robots.txt
Redirect Domain www.discountsex24.com
Redirect Base discountsex24.com
Domain IPs 185.52.189.101
Redirect IPs 185.52.189.101
Response IP 185.52.189.101
Found Yes
Hash 58ffb40e7911d31fa7ccb86f2c1d691417de4354c68f41e8b7f75eaf7b8eecfd
SimHash de1f0121a89b

Groups

amazonbot

Rule Path
Disallow *

yandex

Rule Path
Disallow *

blexbot

Rule Path
Disallow *

dataforseobot

Rule Path
Disallow *

dotbot

Rule Path
Disallow *

ahrefsbot

Rule Path
Disallow *

mj12bot

Rule Path
Disallow *

semrushbot

Rule Path
Disallow *

barkrowler

Rule Path
Disallow *

gptbot

Rule Path
Disallow *

claudebot

Rule Path
Disallow *

seekportbot

Rule Path
Disallow *

timpibot

Rule Path
Disallow *

metajobbot

Rule Path
Disallow *

awariobot

Rule Path
Disallow *

applebot

Rule Path
Disallow *

mediatoolkitbot

Rule Path
Disallow *

bitsightbot

Rule Path
Disallow *

serpstatbot

Rule Path
Disallow *

ccbot

Rule Path
Disallow *

seznambot

Rule Path
Disallow *

domainstatsbot

Rule Path
Disallow *

friendly_crawler

Rule Path
Disallow *

facebookexternalhit

Rule Path
Disallow *

go-http-client

Rule Path
Disallow *

yandexbot

Rule Path
Disallow *

yandex

Rule Path
Disallow *

amazonbot

Rule Path
Disallow *

*

Rule Path
Disallow /wp-admin/*
Disallow /wp-json/*
Disallow /wp-login.php
Disallow /*/feed/
Disallow /my-account/
Disallow /login/
Disallow /cart/
Disallow /registrierung/
Disallow /sitemap/
Disallow /impressum/
Disallow /checkout/
Disallow /mein-konto/
Disallow /wp-admin/
Disallow /registrierung/
Disallow /impressum/
Disallow /mein-konto/
Disallow /kasse/
Disallow /warenkorb/
Disallow /shop/
Allow /media/
Allow /media/bildergalerien/
Allow /media/models
Allow /media/videos
Allow /sexkontakte/kontaktanzeigen
Allow /sexkontakte/prostituierte
Allow /trends
Allow /verzeichnis/locations
Allow /verzeichnis/events
Allow /*

Other Records

Field Value
sitemap https://www.discountsex24.com/sitemap_index.xml

Comments

  • This is technically redundant but can be kept for clarity if you prefer