techtrend.ro
robots.txt

Robots Exclusion Standard data for techtrend.ro

Resource Scan

Scan Details

Site Domain techtrend.ro
Base Domain techtrend.ro
Scan Status Ok
Last Scan2024-11-12T07:38:29+00:00
Next Scan 2024-11-19T07:38:29+00:00

Last Scan

Scanned2024-11-12T07:38:29+00:00
URL https://techtrend.ro/robots.txt
Domain IPs 45.129.66.57
Response IP 45.129.66.57
Found Yes
Hash c94eddd145c22e7b32c10f604781ee4d7110ca55237b53ffa652675fcf32c260
SimHash 4a16d93b6f93

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /trackback/
Disallow /comments/
Disallow /search/
Disallow /author/
Disallow /?s=*
Disallow /?attachment_id=*
Allow /wp-content/uploads/
Allow /wp-content/cache/

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

bingbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

ahrefsbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

semrushbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

googlebot-image

Rule Path
Allow /wp-content/uploads/
Disallow /wp-admin/
Disallow /wp-includes/

google-inspectiontool

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

mj12bot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

screaming frog seo spider

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

dotbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

ccbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

duckduckbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

slurp

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

facebook crawler

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

twitterbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

pinterestbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

linkedinbot

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

botify

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

oncrawl

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

sitebulb

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

netpeak spider

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

deepcrawl

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

lumar

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

scrapebox

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

dyno mapper

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

chatgptcrawler

Rule Path
Allow /
Disallow /wp-admin/
Disallow /wp-includes/

Other Records

Field Value
sitemap https://techtrend.ro/sitemap_index.xml

Comments

  • Googlebot
  • Bingbot
  • AhrefsBot
  • SemrushBot
  • Googlebot-Image
  • Google-InspectionTool
  • Sitemap