drudgenow.com
robots.txt

Robots Exclusion Standard data for drudgenow.com

Resource Scan

Scan Details

Site Domain drudgenow.com
Base Domain drudgenow.com
Scan Status Ok
Last Scan2024-11-13T13:24:29+00:00
Next Scan 2024-11-20T13:24:29+00:00

Last Scan

Scanned2024-11-13T13:24:29+00:00
URL https://drudgenow.com/robots.txt
Domain IPs 2001:19f0:5:3595:5400:4ff:fed8:68c3, 207.246.84.225
Response IP 207.246.84.225
Found Yes
Hash 214d13be1869869e84c3bda98cfb4406e56294fb247e8af6773226f7143aa58f
SimHash 29144191e910

Groups

ahrefsbot

Rule Path
Disallow /

expo9

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://thereport.be/sitemap.txt
sitemap https://thereport.be/news/sitemap.xml