verilynews.com
robots.txt

Robots Exclusion Standard data for verilynews.com

Resource Scan

Scan Details

Site Domain verilynews.com
Base Domain verilynews.com
Scan Status Ok
Last Scan2026-04-08T18:05:39+00:00
Next Scan 2026-04-15T18:05:39+00:00

Last Scan

Scanned2026-04-08T18:05:39+00:00
URL https://verilynews.com/robots.txt
Domain IPs 104.21.75.217, 172.67.182.62, 2606:4700:3035::6815:4bd9, 2606:4700:3037::ac43:b63e
Response IP 104.21.75.217
Found Yes
Hash 4e7bded7eab068480444ef16022eef9248e77544ec2293d35e2b9e42f9b4d617
SimHash d9075080eb9a

Groups

oai-searchbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

applebot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

*

Rule Path Comment
Disallow /wp-admin/ -
Allow /wp-admin/admin-ajax.php -
Disallow /cgi-bin/ -
Disallow /?s= Block search results indexing
Disallow /wp-login.php -
Disallow /wp-json/ -

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://verilynews.com/sitemap_index.xml