afzargostar.com
robots.txt

Robots Exclusion Standard data for afzargostar.com

Resource Scan

Scan Details

Site Domain afzargostar.com
Base Domain afzargostar.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-12-14T09:44:42+00:00
Next Scan 2026-02-12T09:44:42+00:00

Last Successful Scan

Scanned2025-09-22T21:23:27+00:00
URL https://afzargostar.com/robots.txt
Domain IPs 195.28.10.55
Response IP 195.28.10.55
Found Yes
Hash 055b162253110b8c8b61e9a9cd826495ea586f7102c59cd0e83dc9413d1f1c00
SimHash e50d58f3c3b0

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /feed/
Disallow /?s*
Disallow /wp-login*
Disallow /tag/
Disallow /?utm*
Disallow /wpfb*

*

Rule Path
Disallow /

bingbot
googlebot
slurp

Rule Path
Allow /

*

Rule Path
Disallow /

bingbot
googlebot
slurp

Rule Path
Allow /

*

Rule Path
Disallow /

bingbot
googlebot
slurp

Rule Path
Allow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://afzargostar.com/sitemap_index.xml
sitemap https://afzargostar.com/post-sitemap.xml
sitemap https://www.afzargostar.com/post-sitemap.xml
sitemap https://afzargostar.com:443/post-sitemap.xml