datadobi.com
robots.txt

Robots Exclusion Standard data for datadobi.com

Resource Scan

Scan Details

Site Domain datadobi.com
Base Domain datadobi.com
Scan Status Ok
Last Scan2024-09-11T04:39:47+00:00
Next Scan 2024-10-11T04:39:47+00:00

Last Scan

Scanned2024-09-11T04:39:47+00:00
URL https://datadobi.com/robots.txt
Domain IPs 104.26.8.127, 104.26.9.127, 172.67.73.115, 2606:4700:20::681a:87f, 2606:4700:20::681a:97f, 2606:4700:20::ac43:4973
Response IP 172.67.73.115
Found Yes
Hash 32a632dc4f891813345b8e491c87a3e6c586c74a56593092e4a44f9a89944a3d
SimHash dfc7dc7599f3

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /readme.html
Disallow /license.txt
Disallow /wp-login.php
Disallow /wp-login.php*
Disallow /?wordfence_logHuman=*
Disallow /?wordfence_logHuman
Disallow /wp-content/uploads/Protect_Your_Unstructured_Data_on_Any_NAS_or_Cloud.pdf
Disallow /wp-content/uploads/Protect-Your-Unstructured-Data-on-Any-NAS-or-Cloud.pdf
Disallow /wp-content/uploads/protect-your-unstructured-data-on-any-nas-or-cloud-v2a.pdf
Disallow /new-hp-9-4-2024/
Disallow /wp-json/
Disallow /wp-json*/
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/
Disallow /?s=*
Disallow /search/*

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://datadobi.com/sitemap_index.xml

Comments

  • From Yoast