Log In Sign Up

ocsaingatlan.ingatlantajolo.hu
robots.txt

Robots Exclusion Standard data for ocsaingatlan.ingatlantajolo.hu

Resource Scan

Scan Details

Site Domain ocsaingatlan.ingatlantajolo.hu
Base Domain ingatlantajolo.hu
Scan Status Ok
Last Scan2024-05-21T11:57:38+00:00
Next Scan 2024-06-20T11:57:38+00:00

Last Scan

Scanned2024-05-21T11:57:38+00:00
URL https://ocsaingatlan.ingatlantajolo.hu/robots.txt
Domain IPs 104.21.28.111, 172.67.145.224, 2606:4700:3034::6815:1c6f, 2606:4700:3036::ac43:91e0
Response IP 172.67.145.224
Found Yes
Hash 5bc367ebe1e3b4cfb4cccec95478bb2db1f819087f00590636c14267f0ff17cf
SimHash 38003f57e4e0

Groups

*

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-bm

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.ingatlantajolo.hu/sitemap/sitemap-index-v3.xml
Back to top

Comments

  • To block SEMrushBot from crawling your site for different SEO and technical issues:
  • To block SEMrushBot from crawling your site for Backlink Audit tool
  • To slow down SEMrushBot from crawling your site for On Page SEO Checker tool and similar tools
  • To block SEMrushBot from checking URLs your site for SWA tool
  • To slow down SEMrushBot from crawling your site for Content Analyzer and Post Tracking tools
  • To block SEMrushBot from crawling your site for Brand Monitoring
Back to top

Warnings

  • 3 invalid lines.
Back to top