Log In Sign Up

bet.com
robots.txt

Robots Exclusion Standard data for bet.com

Resource Scan

Scan Details

Site Domain bet.com
Base Domain bet.com
Scan Status Ok
Last Scan2024-10-30T01:21:57+00:00
Next Scan 2024-11-06T01:21:57+00:00

Last Scan

Scanned2024-10-30T01:21:57+00:00
URL https://bet.com/robots.txt
Redirect https://www.bet.com/robots.txt
Redirect Domain www.bet.com
Redirect Base bet.com
Domain IPs 100.28.104.175, 2600:1f18:297:ba0e:7900:f622:48a8:f266, 2600:1f18:297:ba18:6ddc:5d8f:6053:f003, 2600:1f18:297:ba24:b508:76de:f4a:df34, 54.157.108.158, 98.83.184.133
Redirect IPs 23.36.50.136, 2600:1413:1:991::2215
Response IP 23.54.58.160
Found Yes
Hash 9a960f12e4088d4a74999aae358d0c3414ed8ff0a6b295420fa7278d8b133819
SimHash de1c5274df1a

Groups

*

Rule Path
Disallow /api/

petalbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.bet.com/xmlsitemap/us/en-us/episode
sitemap https://www.bet.com/xmlsitemap/us/en-us/movie
sitemap https://www.bet.com/xmlsitemap/us/en-us/season
sitemap https://www.bet.com/xmlsitemap/us/en-us/show
sitemap https://www.bet.com/xmlsitemap/us/en-us/video
sitemap https://www.bet.com/xmlsitemap/us/en-us/photogallery
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/all-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/celebs-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/music-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/news-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/lifestyle-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/all-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/celebrity-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/music-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/news-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/lifestyle-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/videofeed/videofeed-sitemap.xml
Back to top