Log In Sign Up

streamingmafia.com
robots.txt

Robots Exclusion Standard data for streamingmafia.com

Resource Scan

Scan Details

Site Domain streamingmafia.com
Base Domain streamingmafia.com
Scan Status Ok
Last Scan2025-08-22T20:56:30+00:00
Next Scan 2025-09-21T20:56:30+00:00

Last Scan

Scanned2025-08-22T20:56:30+00:00
URL https://streamingmafia.com/robots.txt
Domain IPs 152.228.155.65, 152.228.155.66, 152.228.155.67, 152.228.155.69, 152.228.155.70, 152.228.155.71, 152.228.155.73, 152.228.155.74, 152.228.155.76, 152.228.155.79, 188.165.29.223
Response IP 152.228.155.71
Found Yes
Hash 7e0b344e2d94d88ed1d427104cd73ddfded4307323cfa7c86c30256b3103fa3d
SimHash 511cdfd0cd11

Groups

*

Rule Path
Disallow /api
Disallow /2fa
Disallow /confirm_signup
Disallow /example.txt
Disallow /signup

googlebot
bingbot
duckduckgo
yandex
baiduspider
applebot
gptbot
claudebot
perplexitybot
ccbot
chatgpt-user
youbot
neevabot
ai-searcher
google-extended
openai

Rule Path
Allow /

ahrefsbot
semrushbot
mj12bot
dotbot
dataforseobot
seomozbot
uptimerobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://streamingmafia.com/sitemap.xml
Back to top

Comments

  • Allow major search engines and AI bots
  • Block known bad bots and scrapers
  • Sitemap location
Back to top