asdaaarab.com
robots.txt

Robots Exclusion Standard data for asdaaarab.com

Resource Scan

Scan Details

Site Domain asdaaarab.com
Base Domain asdaaarab.com
Scan Status Ok
Last Scan2024-09-27T01:20:52+00:00
Next Scan 2024-10-04T01:20:52+00:00

Last Scan

Scanned2024-09-27T01:20:52+00:00
URL https://asdaaarab.com/robots.txt
Redirect https://fortelese.com/robots.txt
Redirect Domain fortelese.com
Redirect Base fortelese.com
Domain IPs 2a02:4780:9:962:0:982:174f:1, 45.84.206.216
Redirect IPs 104.21.40.93, 172.67.183.207, 2606:4700:3031::ac43:b7cf, 2606:4700:3033::6815:285d
Response IP 172.67.183.207
Found Yes
Hash b698187cd7e7947001cc91966d181f9753031f8a48789bc4afd21eaa8b48d6d8
SimHash e6f75c00f992

Groups

*

Rule Path
Disallow /index.php
Disallow /api.php
Disallow /new/index.php
Disallow /wp-json/
Disallow /tag/
Disallow /page/
Disallow /?s=*
Disallow /search/*
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /trackback/
Disallow /wp-login.php
Disallow /wp-register.php
Allow /wp-admin/admin-ajax.php
Allow /wp-includes/js/
Allow /wp-includes/images/

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://fortelese.com/sitemap_index.xml

Comments

  • If you want to learn about why our robots.txt looks like this, read this post: https://fortelese.com/robots-txt