phpprobid.com
robots.txt

Robots Exclusion Standard data for phpprobid.com

Resource Scan

Scan Details

Site Domain phpprobid.com
Base Domain phpprobid.com
Scan Status Ok
Last Scan2025-05-23T15:18:01+00:00
Next Scan 2025-06-22T15:18:01+00:00

Last Scan

Scanned2025-05-23T15:18:01+00:00
URL https://phpprobid.com/robots.txt
Domain IPs 104.21.57.156, 172.67.146.225, 2606:4700:3031::6815:399c, 2606:4700:3037::ac43:92e1
Response IP 172.67.146.225
Found Yes
Hash 58ffcb41b1fe8924f7bb556ef5411ced3c9fdc2f36316f9de607d29bdab127a3
SimHash a90768f4c761

Groups

*

Rule Path
Disallow /cache/
Disallow /config/
Disallow /library/
Disallow /module/
Disallow /SQL/
Disallow /dev/
Disallow /cron.php
Disallow /install.php
Disallow /error_log
Disallow /*?SID=
Disallow /*?selected_currency=
Disallow /admin/
Disallow /administration/
Disallow /install/
Disallow /rss/
Disallow /advert/
Disallow /play-video/
Disallow /app/async*
Disallow /sign-in-modal/
Disallow /sign-up-modal/
Disallow /forgot-username/
Disallow /forgot-password/
Disallow /bid/
Disallow /buy/
Disallow /offer/
Disallow /cart/
Disallow /listings/cart/
Disallow /listings/listing/watch/
Disallow /advanced-search/
Disallow /sell/

Other Records

Field Value
sitemap https://www.phpprobid.com/sitemap.xml

Comments

  • Crawl-delay: 10
  • Do not crawl application folders
  • Do not crawl root files
  • Do not crawl links with session ids
  • Do not crawl the admin module
  • Do not crawl the install module
  • Do not crawl selected app module pages
  • - rss pages
  • - advert related pages
  • - play video action
  • async controller
  • Do not crawl selected members module pages
  • Disallow: /members-area/
  • Disallow: /members/
  • Do not crawl selected listings module pages
  • - bid/purchase/cart related pages
  • OTHER PAGES
  • - search pages
  • - sell pages
  • Uncomment if you do not wish to allow Google and Bing to index your images
  • User-agent: Googlebot-Image
  • Disallow: /
  • User-agent: msnbot-media
  • Disallow: /
  • Disallow: /*.php$