Log In Sign Up

searchbug.com
robots.txt

Robots Exclusion Standard data for searchbug.com

Resource Scan

Scan Details

Site Domain searchbug.com
Base Domain searchbug.com
Scan Status Ok
Last Scan2025-10-11T01:51:46+00:00
Next Scan 2025-10-18T01:51:46+00:00

Last Scan

Scanned2025-10-11T01:51:46+00:00
URL https://searchbug.com/robots.txt
Redirect https://www.searchbug.com/robots.txt
Redirect Domain www.searchbug.com
Redirect Base searchbug.com
Domain IPs 104.26.2.239, 104.26.3.239, 172.67.71.204, 2606:4700:20::681a:2ef, 2606:4700:20::681a:3ef, 2606:4700:20::ac43:47cc
Redirect IPs 104.26.2.239, 104.26.3.239, 172.67.71.204, 2606:4700:20::681a:2ef, 2606:4700:20::681a:3ef, 2606:4700:20::ac43:47cc
Response IP 104.26.3.239
Found Yes
Hash df481421214c5acb490ea9d98021aadc89c24a14a8b11509d9cfc5385dea0710
SimHash 28049901a1f5

Groups

*

Rule Path
Allow /info/wp-admin-ajax.php
Allow /info/wp-content/uploads
Disallow /info/wp-admin/
Disallow /info/wp-content/plugins/kalins-pdf-creation-station/kalins_pdf_create.php?singlepost=*
Disallow /info/out-date/?*
Disallow /info/tag/*
Disallow /info/category/*
Disallow /tools/zip-code-lookup.aspx?*
Disallow /*?s
Disallow /*?HOW=
Disallow /*?TYPE=
Disallow /*?WHAT=
Disallow /*?RATE=
Disallow /*?BG=
Disallow /*?FNX=
Disallow /info/tag/
Disallow /info/category/
Disallow /info/author/

googlebot

Rule Path
Disallow /?s=

gptbot

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /

anthropicbot

Rule Path
Allow /

xaibot

Rule Path
Allow /

bingbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.searchbug.com/info/sitemap_index.xml
sitemap https://www.searchbug.com/info/post-sitemap.xml
sitemap https://www.searchbug.com/info/page-sitemap.xml
sitemap https://www.searchbug.com/sitemap.xml
Back to top

Comments

  • BLOCK dynamic search and filter parameters
  • BLOCK WordPress-style archive paths
Back to top

Warnings

  • 2 invalid lines.
Back to top