news-pt.bestdeals.today
robots.txt

Robots Exclusion Standard data for news-pt.bestdeals.today

Resource Scan

Scan Details

Site Domain news-pt.bestdeals.today
Base Domain bestdeals.today
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-03-20T18:24:05+00:00
Next Scan 2024-06-18T18:24:05+00:00

Last Successful Scan

Scanned2023-08-23T07:42:42+00:00
URL https://news-pt.bestdeals.today/robots.txt
Domain IPs 104.21.67.176, 172.67.179.3, 2606:4700:3034::6815:43b0, 2606:4700:3035::ac43:b303
Response IP 104.21.67.176
Found Yes
Hash 46c540754e9bf28b731b3cab9fa4f5ecc93ce9904041ad516fb2251e3f18d80f
SimHash 21041b60c711

Groups

adsbot-google

Rule Path
Disallow

grapeshot

Rule Path
Disallow

gptbot

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.bestdeals.today/sitemap.xml