tap.az
robots.txt

Robots Exclusion Standard data for tap.az

Resource Scan

Scan Details

Site Domain tap.az
Base Domain tap.az
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-01T19:56:09+00:00
Next Scan 2024-07-30T19:56:09+00:00

Last Successful Scan

Scanned2023-01-04T11:56:50+00:00
URL https://tap.az/robots.txt
Domain IPs 104.26.12.26, 104.26.13.26, 172.67.75.5, 2606:4700:20::681a:c1a, 2606:4700:20::681a:d1a, 2606:4700:20::ac43:4b05
Response IP 172.67.75.5
Found Yes
Hash 947dd2e58fc42a05aaf96712a7a2e38f1a64a5bf7be4ef771e9e85602caf5b14
SimHash 4944cc564f92

Groups

*

Rule Path
Disallow /pages/advertising
Disallow /auth/
Disallow /pages/rules
Disallow /bookmarks

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://tap.azstatic.com/uploads/attachment/sitemap_az.xml