Log In Sign Up

euromaidanpress.com
robots.txt

Robots Exclusion Standard data for euromaidanpress.com

Resource Scan

Scan Details

Site Domain euromaidanpress.com
Base Domain euromaidanpress.com
Scan Status Ok
Last Scan2024-09-24T10:30:36+00:00
Next Scan 2024-10-01T10:30:36+00:00

Last Scan

Scanned2024-09-24T10:30:36+00:00
URL https://euromaidanpress.com/robots.txt
Domain IPs 104.21.80.72, 172.67.175.224, 2606:4700:3031::ac43:afe0, 2606:4700:3036::6815:5048
Response IP 172.67.175.224
Found Yes
Hash e6be4c071c974ad59a818b5e398cd0a5a783db9b6686aceb8107e23472052008
SimHash 4f10c9d6d2b1

Groups

*

Rule Path
Disallow /*.php$
Disallow /*?s=*
Disallow /wp-admin/
Disallow /comments/feed/
Disallow /preview/
Disallow /search
Disallow /cdn-cgi/
Disallow /?blackhole

Other Records

Field Value
crawl-delay 40

googlebot-news

Rule Path
Allow /

mediapartners-google*

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot
yandex
slurp

Rule Path
Disallow /?*s=*
Disallow /wp-admin/
Disallow /*.php$

Other Records

Field Value
crawl-delay 40

the knowledge ai

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://euromaidanpress.com/sitemap_index.xml
Back to top

Comments

  • Don't request php pages:
  • honeypot
Back to top