Log In Sign Up

promods.net
robots.txt

Robots Exclusion Standard data for promods.net

Resource Scan

Scan Details

Site Domain promods.net
Base Domain promods.net
Scan Status Ok
Last Scan2024-09-27T07:16:34+00:00
Next Scan 2024-10-27T07:16:34+00:00

Last Scan

Scanned2024-09-27T07:16:34+00:00
URL https://promods.net/robots.txt
Domain IPs 104.26.2.60, 104.26.3.60, 172.67.72.12, 2606:4700:20::681a:23c, 2606:4700:20::681a:33c, 2606:4700:20::ac43:480c
Response IP 172.67.72.12
Found Yes
Hash c8008388a4f879700daef333c628695835b385960df07653c6ad3017b190c74e
SimHash d24c49d2c21b

Groups

*

Rule Path
Disallow /cgi-bin/

megaindex.ru

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sch-fast-se-crawl02.osl.basefarm.net

Rule Path
Disallow /

sch-fast-se-crawl04.osl.basefarm.net

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

yandex

Rule Path
Disallow /
Disallow /ucp.php
Disallow /members/
Disallow /faq.php
Disallow /search.php
Disallow /posting.php
Disallow /styles/
Disallow /includes/
Disallow /language/
Disallow /mcp.php
Disallow /ucp.php
Disallow /adm/
Back to top

Comments

  • Standard information
  • Block specific bots
  • Block Yandex completely
  • phpBB specific directories
Back to top

Warnings

  • 4 invalid lines.
Back to top