Log In Sign Up

bet.com
robots.txt

Robots Exclusion Standard data for bet.com

Resource Scan

Scan Details

Site Domain bet.com
Base Domain bet.com
Scan Status Ok
Last Scan2024-04-30T15:58:30+00:00
Next Scan 2024-05-07T15:58:30+00:00

Last Scan

Scanned2024-04-30T15:58:30+00:00
URL https://bet.com/robots.txt
Redirect https://www.bet.com/robots.txt
Redirect Domain www.bet.com
Redirect Base bet.com
Domain IPs 34.213.106.51, 54.68.182.72
Redirect IPs 23.52.114.111, 2600:1413:b000:38e::2215, 2600:1413:b000:397::2215
Response IP 23.54.58.160
Found Yes
Hash 9a960f12e4088d4a74999aae358d0c3414ed8ff0a6b295420fa7278d8b133819
SimHash de1c5274df1a

Groups

*

Rule Path
Disallow /api/

petalbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.bet.com/xmlsitemap/us/en-us/episode
sitemap https://www.bet.com/xmlsitemap/us/en-us/movie
sitemap https://www.bet.com/xmlsitemap/us/en-us/season
sitemap https://www.bet.com/xmlsitemap/us/en-us/show
sitemap https://www.bet.com/xmlsitemap/us/en-us/video
sitemap https://www.bet.com/xmlsitemap/us/en-us/photogallery
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/all-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/celebs-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/music-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/news-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/news/lifestyle-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/all-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/celebrity-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/music-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/news-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/newsfeed/lifestyle-category-sitemap.xml
sitemap https://www.bet.com/xmlsitemap/us/en-us/videofeed/videofeed-sitemap.xml
Back to top