news-en.bestdeals.today
robots.txt

Robots Exclusion Standard data for news-en.bestdeals.today

Resource Scan

Scan Details

Site Domain news-en.bestdeals.today
Base Domain bestdeals.today
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-03-26T16:16:50+00:00
Next Scan 2024-06-24T16:16:50+00:00

Last Successful Scan

Scanned2023-08-29T07:23:43+00:00
URL https://news-en.bestdeals.today/robots.txt
Domain IPs 172.64.100.38, 172.64.101.38, 2606:4700:e0::ac40:6426, 2606:4700:e0::ac40:6526
Response IP 172.64.100.38
Found Yes
Hash 46c540754e9bf28b731b3cab9fa4f5ecc93ce9904041ad516fb2251e3f18d80f
SimHash 21041b60c711

Groups

adsbot-google

Rule Path
Disallow

grapeshot

Rule Path
Disallow

gptbot

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.bestdeals.today/sitemap.xml