Log In Sign Up

datagaps.com
robots.txt

Robots Exclusion Standard data for datagaps.com

Resource Scan

Scan Details

Site Domain datagaps.com
Base Domain datagaps.com
Scan Status Ok
Last Scan2025-11-05T23:45:46+00:00
Next Scan 2025-12-05T23:45:46+00:00

Last Scan

Scanned2025-11-05T23:45:46+00:00
URL https://datagaps.com/robots.txt
Domain IPs 104.21.68.129, 172.67.195.139, 2606:4700:3031::ac43:c38b, 2606:4700:3037::6815:4481
Response IP 104.21.68.129
Found Yes
Hash 2ca9f5c85fb7124ed273486f7033e2c8cc6325d7e3c5e2fcfb4061472bb5c0a6
SimHash 9f54f911e434

Groups

*

Rule Path
Disallow /?s=*
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

bingbot

Rule Path
Disallow /?s=*
Disallow /wp-admin/
Allow /

oai-searchbot

Rule Path
Disallow /wp-admin/

gptbot

Rule Path
Disallow /wp-admin/
Allow /

claudebot

Rule Path
Allow /

claude-web

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

perplexity-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

amazonbot

Rule Path
Allow /

qwenbot

Rule Path
Allow /
Disallow /blog/category/
Disallow /blog/case-study-category/
Disallow /blog/*/page/
Disallow /blog/category/*/page/
Disallow /blog/category/*
Disallow /blog/2/
Disallow /blog/3/
Disallow /blog/4/
Disallow /blog/5/
Disallow /feed/
Disallow /category/
Disallow /tag/
Disallow /blog/tag/
Disallow /author/
Disallow /blog/tag/*/
Disallow /event-category/*/*/
Disallow /case-studies-2/
Disallow /case-study/page/*/
Disallow /post_tag-sitemap.xml
Disallow /blog/tag/etl-testing-process/
Disallow /blog/tag/tableau-testing-tools/
Disallow /blog/tag/power-bi-for-higher-education/
Disallow /blog/tag/regression-testing-tableau/
Disallow /blog/tag/functional-testing-tableau/
Disallow /blog/tag/automated-testing-for-tableau/
Disallow /blog/tag/healthcare-insurance/
Disallow /blog/tag/healthcare-data-quality/
Disallow /blog/tag/bi-testing-automation/
Disallow /blog/tag/data-validation/
Disallow /wp-content/uploads/*.pdf
Disallow /wp-content/uploads/*.xlsx
Disallow /blog/category/cloud-data-migration/
Disallow /blog/category/dataops/
Disallow /blog/category/etl-testing/
Disallow /event/
Disallow /event/*
Disallow /tds/
Disallow /blog/tag/etl-testing-process/
Disallow /blog/category/uncategorized/
Disallow /industries/$
Disallow /thank-you/$
Disallow /*.pdf$
Disallow /blog/case-study-category/others-it/
Disallow /blog/category/data-observability/
Disallow /blog/tag/bi-analytics-testing-tools/
Disallow /blog/tag/tableau-reports-testing/
Disallow /blog/tag/etl-testing-process/
Disallow /blog/tag/*
Disallow /blog/category/databricks/
Disallow /blog/tag/bi-testing-automation/
Disallow /blog/author/*
Disallow /.well-known/captcha/*
Disallow /.well-known/sgcaptcha/*
Disallow /*/feed/
Disallow /webinars/page/
Disallow /*?paged=
Back to top

Other Records

Field Value
sitemap https://www.datagaps.com/sitemap_index.xml
sitemap https://www.datagaps.com/page-sitemap.xml
sitemap https://www.datagaps.com/post-sitemap.xml
sitemap https://www.datagaps.com/post_tag-sitemap.xml
Back to top