techtribune.net
robots.txt

Robots Exclusion Standard data for techtribune.net

Resource Scan

Scan Details

Site Domain techtribune.net
Base Domain techtribune.net
Scan Status Ok
Last Scan2024-09-15T16:44:39+00:00
Next Scan 2024-09-22T16:44:39+00:00

Last Scan

Scanned2024-09-15T16:44:39+00:00
URL https://techtribune.net/robots.txt
Domain IPs 104.21.33.140, 172.67.145.203, 2606:4700:3033::ac43:91cb, 2606:4700:3035::6815:218c
Response IP 104.21.33.140
Found Yes
Hash a3eaf33df6e545787093d58b4cc218cede41e7c712d00d6c86db07b5254b64cc
SimHash 8b0dfc63e6a3

Groups

*

Rule Path
Disallow */wp-admin/
Disallow */?s=*
Disallow */search/
Disallow */sysadmin/
Allow *search/?s=*
Allow *wp-admin/admin-ajax.php

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://fr.techtribune.net/sitemap_index.xml
sitemap https://fr.techtribune.net/news-sitemap.xml
sitemap https://fr.techtribune.net/d2/sitemap_index.xml
sitemap https://fr.techtribune.net/d2/news-sitemap.xml