desifakes.com
robots.txt

Robots Exclusion Standard data for desifakes.com

Resource Scan

Scan Details

Site Domain desifakes.com
Base Domain desifakes.com
Scan Status Ok
Last Scan2024-09-20T16:01:48+00:00
Next Scan 2024-10-20T16:01:48+00:00

Last Scan

Scanned2024-09-20T16:01:48+00:00
URL https://desifakes.com/robots.txt
Domain IPs 104.21.20.67, 172.67.191.217, 2606:4700:3030::6815:1443, 2606:4700:3033::ac43:bfd9
Response IP 104.21.20.67
Found Yes
Hash 903e328ae849c50f4a3ccb9aa27f4e38dfdcab76fff385cf8e51cbd21cccdc8a
SimHash 024b56200e21

Groups

*

Rule Path
Disallow /whats-new/
Disallow /account/
Disallow /attachments/
Disallow /goto/
Disallow /posts/
Disallow /temp/
Disallow /login/
Disallow /admin.php
Allow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

linux wget

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot0.5.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

proximic

Rule Path
Disallow /

trendictionbot0.5.0

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

ttd-content

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

Other Records

Field Value
sitemap https://desifakes.com/sitemap.xml