romaniapolitica.ro
robots.txt

Robots Exclusion Standard data for romaniapolitica.ro

Resource Scan

Scan Details

Site Domain romaniapolitica.ro
Base Domain romaniapolitica.ro
Scan Status Ok
Last Scan2024-11-12T07:33:31+00:00
Next Scan 2024-11-19T07:33:31+00:00

Last Scan

Scanned2024-11-12T07:33:31+00:00
URL https://romaniapolitica.ro/robots.txt
Domain IPs 104.21.8.219, 172.67.140.72, 2606:4700:3035::6815:8db, 2606:4700:3035::ac43:8c48
Response IP 172.67.140.72
Found Yes
Hash 24d1c34e43fefa553995f723bab48a8f9ccd6575cb48c8014db8b4e0bdb02994
SimHash 4a14d83aef83

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /trackback/
Disallow /comments/
Disallow /search/
Disallow /author/
Disallow /?s=*
Disallow /?attachment_id=*
Allow /wp-content/uploads/
Allow /wp-content/cache/

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

bingbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

ahrefsbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

semrushbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

googlebot-image

Rule Path
Allow /wp-content/uploads/
Disallow /wp-admin/
Disallow /wp-includes/

google-inspectiontool

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

mj12bot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

screaming frog seo spider

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

dotbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

ccbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

duckduckbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

slurp

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

facebook crawler

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

twitterbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

pinterestbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

linkedinbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

botify

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

oncrawl

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

sitebulb

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

netpeak spider

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

deepcrawl

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

lumar

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

scrapebox

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

dyno mapper

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

chatgptcrawler

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

Other Records

Field Value
sitemap https://romaniapolitica.ro/sitemap_index.xml

Comments

  • Googlebot
  • Bingbot
  • AhrefsBot
  • SemrushBot
  • Googlebot-Image
  • Google-InspectionTool
  • Sitemap