paralel.az
robots.txt

Robots Exclusion Standard data for paralel.az

Resource Scan

Scan Details

Site Domain paralel.az
Base Domain paralel.az
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan3/24/2025, 9:17:31 PM
Next Scan 6/22/2025, 9:17:31 PM

Last Successful Scan

Scanned11/2/2024, 7:54:00 PM
URL https://paralel.az/robots.txt
Domain IPs 104.21.47.46, 172.67.144.117, 2606:4700:3031::6815:2f2e, 2606:4700:3036::ac43:9075
Response IP 172.67.144.117
Found Yes
Hash fc7ac991c24a730f3d548a99b11d268d4a26ead8839f604da37d5cde4ff5d28d
SimHash ef08cc44c6b7

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://paralel.az/az/sitemap/article

Comments

  • Added by add_robots.sh
  • Crawl-delay: 10
  • Request-rate: 1/10

Warnings

  • `host` is not a known field.