bit4id.com
robots.txt

Robots Exclusion Standard data for bit4id.com

Resource Scan

Scan Details

Site Domain bit4id.com
Base Domain bit4id.com
Scan Status Ok
Last Scan2025-08-29T14:25:42+00:00
Next Scan 2025-09-12T14:25:42+00:00

Last Scan

Scanned2025-08-29T14:25:42+00:00
URL https://bit4id.com/robots.txt
Domain IPs 18.161.111.2, 18.161.111.47, 18.161.111.66, 18.161.111.67
Response IP 18.244.87.33
Found Yes
Hash 5b98257122d1c77fa7309d3961ccea2c61f24a6efef4a7b7fab58bc8ba3c0cad
SimHash 8d20190122b2

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-includes
Disallow /wp-content/plugins
Disallow /wp-content/cache
Disallow /*?
Disallow /?*
Disallow /wp-admin
Disallow */comments
Disallow */trackback
Disallow /cgi-bin
Disallow /*.php$
Disallow /*.inc$
Disallow /*.gz
Disallow /*.cgi
Disallow /*lang%3D*
Disallow /*?preview_id=*
Disallow /*?post_type=*
Disallow /*?page_id=*
Disallow */feed
Disallow *?s=*
Disallow /search/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.bit4id.com/sitemap.xml

Comments

  • Disallow sensitive files on WP
  • Limit useless crawl
  • But allow
  • Sitemap