Log In Sign Up

digitalbang.gr
robots.txt

Robots Exclusion Standard data for digitalbang.gr

Resource Scan

Scan Details

Site Domain digitalbang.gr
Base Domain digitalbang.gr
Scan Status Ok
Last Scan2025-11-28T02:56:36+00:00
Next Scan 2025-12-05T02:56:36+00:00

Last Scan

Scanned2025-11-28T02:56:36+00:00
URL https://digitalbang.gr/robots.txt
Domain IPs 104.21.52.28, 172.67.194.155, 2606:4700:3032::ac43:c29b, 2606:4700:3034::6815:341c
Response IP 104.21.52.28
Found Yes
Hash 677c433236dcf43cf9302268d7f41dcfb9871a6e3e26aae853adb024d5774ff3
SimHash 61125212cc2b

Groups

*

Rule Path Comment
Disallow /wp-admin/ -
Allow /wp-admin/admin-ajax.php -
Disallow /feed/ -
Disallow /?s= -
Disallow /*?* Blocks query parameter URLs

oai-search

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://digitalbang.gr/sitemap_index.xml
Back to top

Comments

  • Allow OpenAI's ChatGPT web crawler
  • Allow Anthropic's ClaudeBot
  • Allow Perplexity AI crawler
  • Allow Google's AI (Gemini and Bard)
  • Allow Common Crawl (used in many LLM training datasets)
Back to top