Log In Sign Up

worldstar.com
robots.txt

Robots Exclusion Standard data for worldstar.com

Resource Scan

Scan Details

Site Domain worldstar.com
Base Domain worldstar.com
Scan Status Ok
Last Scan2024-05-02T20:27:17+00:00
Next Scan 2024-05-09T20:27:17+00:00

Last Scan

Scanned2024-05-02T20:27:17+00:00
URL https://worldstar.com/robots.txt
Domain IPs 151.101.0.193, 151.101.128.193, 151.101.192.193, 151.101.64.193
Response IP 151.101.0.193
Found Yes
Hash 17f0a67f1563305906ba3970cce74deec5913e31495727a499adb344930b5b16
SimHash 4401ed0bc0fb

Groups

*

Rule Path
Allow /
Disallow /embed/*
Disallow /u.php/*
Disallow /utils/*
Disallow /api/v1/*
Disallow /video.php
Disallow /videos/e/*
Disallow /recaptcha/*
Disallow /videos/js/*
Disallow /video-c.php
Disallow /video/*
Disallow /videos/assets/*
Disallow /videos/images/*
Disallow /videos/search.php
Allow /videos/search.php?s=fight%2Bcompilation$
Allow /videos/search.php?s=fight$
Allow /videos/search.php?s=fight%2Bcomp$
Allow /videos/search.php?s=detroitfights$
Allow /videos/search.php?s=fight%2Bcomp$
Allow /videos/search.php?s=Fight%2BCompilation$
Allow /videos/search.php?s=girl%2Bfights$
Allow /videos/search.php?s=woman%2Bfight$
Allow /videos/search.php?s=girls%2Bfighting$
Allow /videos/search.php?s=girls%2Bfighting$
Allow /videos/search.php?s=hood%2Bfight$
Allow /videos/search.php?s=shooting$
Allow /videos/search.php?s=street%2Bfight$
Allow /videos/search.php?filter=*&s=fight%2Bcompilation*&start=*$
Allow /videos/search.php?filter=*&s=fight*&start=*$
Allow /videos/search.php?filter=*&s=fight%2Bcomp*&start=*$
Allow /videos/search.php?filter=*&s=detroit%2Bfights*&start=*$
Allow /videos/search.php?filter=*&s=fight%2Bcomp*&start=*$
Allow /videos/search.php?filter=*&s=Fight%2BCompilation*&start=*$
Allow /videos/search.php?filter=*&s=girl%2Bfights*&start=*$
Allow /videos/search.php?filter=*&s=woman%2Bfight*&start=*$
Allow /videos/search.php?filter=*&s=girls%2Bfighting*&start=*$
Allow /videos/search.php?filter=*&s=hood%2Bfight*&start=*$
Allow /videos/search.php?filter=*&s=shooting*&start=*$
Allow /videos/search.php?filter=*&s=street%2Bfight*&start=*$
Allow /videos/search.php?s=fight%2Bcompilation*&start=*$
Allow /videos/search.php?s=fight*&start=*$
Allow /videos/search.php?s=fight%2Bcomp*&start=*$
Allow /videos/search.php?s=detroit%2Bfights*&start=*$
Allow /videos/search.php?s=fight%2Bcomp*&start=*$
Allow /videos/search.php?s=Fight%2BCompilation*&start=*$
Allow /videos/search.php?filter=*&s=girl%2Bfights*&start=*$
Allow /videos/search.php?s=woman%2Bfight*&start=*$
Allow /videos/search.php?s=girls%2Bfighting*&start=*$
Allow /videos/search.php?s=hood%2Bfight*&start=*$
Allow /videos/search.php?s=shooting*&start=*$
Allow /videos/search.php?s=street%2Bfight*&start=*$
Allow /videos/search.php?filter=*&s=fight%2Bcompilation$
Allow /videos/search.php?filter=*&s=fight$
Allow /videos/search.php?filter=*&s=fight%2Bcomp$
Allow /videos/search.php?filter=*&s=detroit%2Bfights$
Allow /videos/search.php?filter=*&s=fight%2Bcomp$
Allow /videos/search.php?filter=*&s=Fight%2BCompilation$
Allow /videos/search.php?filter=*&s=girl%2Bfights$
Allow /videos/search.php?filter=*&s=woman%2Bfight$
Allow /videos/search.php?filter=*&s=girls%2Bfighting$
Allow /videos/search.php?filter=*&s=hood%2Bfight$
Allow /videos/search.php?filter=*&s=shooting$
Allow /videos/search.php?filter=*&s=street%2Bfight$
Back to top

Other Records

Field Value
sitemap https://worldstar.com/sitemap.xml
Back to top

Comments

  • allow all paths
  • block all search pages
  • allow specific search pages
  • allow specific search pages with any trailing parameters
  • allow specific search pages with any trailing parameters
  • sitemap URLs
Back to top