pikabu.ru
robots.txt

Robots Exclusion Standard data for pikabu.ru

Resource Scan

Scan Details

Site Domain pikabu.ru
Base Domain pikabu.ru
Scan Status Ok
Last Scan2024-05-11T15:57:36+00:00
Next Scan 2024-05-18T15:57:36+00:00

Last Scan

Scanned2024-05-11T15:57:36+00:00
URL https://pikabu.ru/robots.txt
Domain IPs 185.178.210.201
Response IP 185.178.210.201
Found Yes
Hash 6933d4b874612eb604dbba45b01117018d1dbf7fc9395ff73d4789916c9d411f
SimHash 3a47c84f6a34

Groups

*

Rule Path
Allow */community/edits/new
Allow /search?n=16&r=3
Allow /original-stories
Disallow /generate_xml.php
Disallow /restore.php
Disallow /ajax/*
Disallow /api/*
Disallow /api2/*
Disallow /assets/*
Disallow /cron/*
Disallow /data_vars/*
Disallow /external/*
Disallow /files/*
Disallow /staff/*
Disallow /pds/*
Disallow *username%3D*
Disallow /offer-redirect/
Disallow /*?r=
Disallow */edits/*
Disallow /*?n=*
Disallow /admitad/finance/redirect
Disallow /page/misc/adstat.php*
Disallow /.well-known/*
Disallow /search?*
Disallow /tag/*st%3D*
Disallow /tag/*n%3D*
Disallow /tag/*et%3D*
Disallow /tag/*d%3D*
Disallow /tag/*cl%3D*
Disallow /tag/*vh%3D*
Disallow /%40*/*
Disallow /community/*/*
Disallow /communities/*
Disallow /communities?t=*
Disallow /communities?q=*
Disallow /add?community=*
Disallow /information/docs

yandex

Rule Path
Allow */community/edits/new
Allow /search?n=16&r=3
Allow /original-stories
Disallow /generate_xml.php
Disallow /restore.php
Disallow /ajax/*
Disallow /api/*
Disallow /api2/*
Disallow /assets/*
Disallow /cron/*
Disallow /data_vars/*
Disallow /external/*
Disallow /files/*
Disallow /rabbit_workers/*
Disallow /staff/*
Disallow /pds/*
Disallow *username%3D*
Disallow /offer-redirect/
Disallow /*?r=
Disallow */edits/*
Disallow /*?n=*
Disallow /admitad/finance/redirect
Disallow /page/misc/adstat.php*
Disallow /.well-known/*
Disallow /search?*
Disallow /tag/*st%3D*
Disallow /tag/*n%3D*
Disallow /tag/*et%3D*
Disallow /tag/*d%3D*
Disallow /tag/*cl%3D*
Disallow /tag/*vh%3D*
Disallow /%40*/*
Disallow /community/*/*
Disallow /communities/*
Disallow /communities?t=*
Disallow /communities?q=*
Disallow /add?community=*

Other Records

Field Value
crawl-delay 3

mail.ru_bot

Rule Path
Allow */community/edits/new
Allow /search?n=16&r=3
Allow /original-stories
Disallow /generate_xml.php
Disallow /restore.php
Disallow /WT4fgrvixo2F/*
Disallow /ajax/*
Disallow /api/*
Disallow /api2/*
Disallow /assets/*
Disallow /cron/*
Disallow /data_vars/*
Disallow /external/*
Disallow /files/*
Disallow /rabbit_workers/*
Disallow /staff/*
Disallow /pds/*
Disallow *username%3D*
Disallow /offer-redirect/
Disallow /*?r=
Disallow */edits/*
Disallow /*?n=*
Disallow /admitad/finance/redirect
Disallow /page/misc/adstat.php*
Disallow /.well-known/*
Disallow /search?*
Disallow /tag/*st%3D*
Disallow /tag/*n%3D*
Disallow /tag/*et%3D*
Disallow /tag/*d%3D*
Disallow /tag/*cl%3D*
Disallow /tag/*vh%3D*
Disallow /%40*/*
Disallow /community/*/*
Disallow /communities/*
Disallow /communities?t=*
Disallow /communities?q=*
Disallow /add?community=*

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://pikabu.ru/sitemap_index.xml

Warnings

  • `clean-param` is not a known field.