Log In Sign Up

bigfile.top
robots.txt

Robots Exclusion Standard data for bigfile.top

Resource Scan

Scan Details

Site Domain bigfile.top
Base Domain bigfile.top
Scan Status Ok
Last Scan2025-04-21T06:40:01+00:00
Next Scan 2025-04-28T06:40:01+00:00

Last Scan

Scanned2025-04-21T06:40:01+00:00
URL https://bigfile.top/robots.txt
Domain IPs 104.21.9.176, 172.67.161.36, 2606:4700:3032::6815:9b0, 2606:4700:3034::ac43:a124
Response IP 172.67.161.36
Found Yes
Hash 26ab0a2816858b9bcf9ef2a9ea8ff8622eb4c4f3d800abaab1a26b9f2e94f7e6
SimHash ba1e7106c9e0

Groups

awariorssbot
awariosmartbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

amazonadbot

Rule Path
Disallow /
Back to top