f1technical.net
robots.txt

Robots Exclusion Standard data for f1technical.net

Resource Scan

Scan Details

Site Domain f1technical.net
Base Domain f1technical.net
Scan Status Ok
Last Scan2024-09-20T12:55:45+00:00
Next Scan 2024-09-27T12:55:45+00:00

Last Scan

Scanned2024-09-20T12:55:45+00:00
URL https://f1technical.net/robots.txt
Redirect https://www.f1technical.net/robots.txt
Redirect Domain www.f1technical.net
Redirect Base f1technical.net
Domain IPs 37.48.120.233
Redirect IPs 37.48.120.233
Response IP 37.48.120.233
Found Yes
Hash 56613992b0dcc086137ea307e92c655e027015daccb0877a2a7f8d9872c9b077
SimHash 0a04fc52e3c3

Groups

mediapartners-google*

Rule Path
Disallow

twitterbot

Rule Path
Disallow

ahrefsbot

Product Comment
ahrefsbot https://ahrefs.com/robot/
Rule Path
Disallow /

megaindex

Product Comment
megaindex https://www.megaindex.ru/?tab=linkAnalyze
Rule Path
Disallow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /error.php
Disallow /cron.php
Disallow /cgi-bin/
Disallow /admin/
Disallow /images/
Disallow /includes/
Disallow /stats/
Disallow /sitestats/
Disallow /language/
Disallow /pr/
Disallow /forum/adm/
Disallow /forum/includes/
Disallow /forum/images/
Disallow /forum/language/
Disallow /forum/templates/
Disallow /forum/common.php
Disallow /forum/config.php
Disallow /forum/cron.php
Disallow /forum/download.php
Disallow /forum/mcp.php
Disallow /forum/memberlist.php
Disallow /forum/posting.php
Disallow /forum/report.php
Disallow /forum/search.php
Disallow /forum/ucp.php
Disallow /forum/viewonline.php
Disallow /gallery/lib/
Disallow /gallery/var/logs/
Disallow /gallery/var/modules/
Disallow /gallery/var/albums/
Disallow /gallery/var/tmp/
Disallow /gallery/var/uploads/
Disallow /gallery/var/fullsizes/

Other Records

Field Value
sitemap https://www.f1technical.net/sitemap/sitemap.xml.gz
sitemap https://www.f1technical.net/newest.xml