Log In Sign Up

gaports.com
robots.txt

Robots Exclusion Standard data for gaports.com

Resource Scan

Scan Details

Site Domain gaports.com
Base Domain gaports.com
Scan Status Ok
Last Scan2025-11-21T13:06:47+00:00
Next Scan 2025-12-21T13:06:47+00:00

Last Scan

Scanned2025-11-21T13:06:47+00:00
URL https://gaports.com/robots.txt
Domain IPs 104.18.28.247, 104.18.29.247, 2606:4700::6812:1cf7, 2606:4700::6812:1df7
Response IP 104.18.29.247
Found Yes
Hash f6d4854d9de09815a2704eadb351c539d02996d916b9df875eb782c1920f3038
SimHash 6b749a3105e7

Groups

*

Rule Path Comment
Disallow /wp-admin/ -
Disallow /wp-includes/ -
Disallow /xmlrpc.php -
Disallow /?s= Block internal search pages
Disallow /*?*filter=* Block filter parameter URLs
Disallow /*?*orderby=* Block sorting parameter URLs
Disallow /*?*utm_* Block tracking URLs (Google Analytics, etc.)
Disallow /*?*replytocom Block WordPress comment reply links
Disallow /wp-content/uploads/bulk-media-register-tmp/ -
Allow /wp-admin/admin-ajax.php -

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

chatgpt-user

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

google-extended

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

ccbot

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

facebookbot

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

bingai

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

google-extended

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

anthropic

Rule Path
Disallow /?s=
Disallow /*?*filter=*
Disallow /*?*orderby=*
Allow /

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://gaports.com/sitemap_index.xml
Back to top

Comments

  • GENERAL SETTINGS
  • Optional: reduce crawl load if server is sensitive
  • AI & SCRAPER BOT RULES
  • Allow known AI data collection bots
  • IMPORTANT SEO DIRECTIVES
  • Allow major search engines full access
  • SITEMAP
Back to top