aifirsasims.ru
robots.txt

Robots Exclusion Standard data for aifirsasims.ru

Resource Scan

Scan Details

Site Domain aifirsasims.ru
Base Domain aifirsasims.ru
Scan Status Ok
Last Scan2024-10-05T22:59:48+00:00
Next Scan 2024-10-12T22:59:48+00:00

Last Scan

Scanned2024-10-05T22:59:48+00:00
URL https://aifirsasims.ru/robots.txt
Domain IPs 104.21.85.246, 172.67.212.250, 2606:4700:3032::ac43:d4fa, 2606:4700:3034::6815:55f6
Response IP 172.67.212.250
Found Yes
Hash dfecf0a465a247a99f6d07364eefe58a2731ea2506b773c1d7bace5a74bceee0
SimHash 4b445c6aabb2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /dlm_uploads/
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Allow /wp-includes/js/
Allow /wp-includes/images/
Disallow /trackback/
Disallow /wp-login.php
Disallow /wp-register.php

Other Records

Field Value
sitemap http://aifirsasims.ru/sitemap.xml