techtribune.net
robots.txt

Robots Exclusion Standard data for techtribune.net

Resource Scan

Scan Details

Site Domain techtribune.net
Base Domain techtribune.net
Scan Status Ok
Last Scan2024-11-18T01:56:17+00:00
Next Scan 2024-11-25T01:56:17+00:00

Last Scan

Scanned2024-11-18T01:56:17+00:00
URL https://techtribune.net/robots.txt
Domain IPs 104.21.33.140, 172.67.145.203, 2606:4700:3033::ac43:91cb, 2606:4700:3035::6815:218c
Response IP 172.67.145.203
Found Yes
Hash 78ef6dec494806feb5be8baf8c6ec10b32791ad2c0a5c68b9bbc95118f1abc5d
SimHash 8b0df823c623

Groups

*

Rule Path
Disallow */wp-admin/
Disallow */?s=*
Disallow */search/
Disallow */sysadmin/
Allow *search/?s=*
Allow *wp-admin/admin-ajax.php

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

siteauditbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

oai-searchbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://fr.techtribune.net/sitemap_index.xml
sitemap https://fr.techtribune.net/news-sitemap.xml
sitemap https://fr.techtribune.net/d2/sitemap_index.xml
sitemap https://fr.techtribune.net/d2/news-sitemap.xml