observalgerie.com
robots.txt

Robots Exclusion Standard data for observalgerie.com

Resource Scan

Scan Details

Site Domain observalgerie.com
Base Domain observalgerie.com
Scan Status Ok
Last Scan2024-05-22T09:52:56+00:00
Next Scan 2024-05-29T09:52:56+00:00

Last Scan

Scanned2024-05-22T09:52:56+00:00
URL https://observalgerie.com/robots.txt
Domain IPs 104.16.205.47, 104.18.153.16, 2606:4700::6810:cd2f, 2606:4700::6812:9910
Response IP 104.18.153.16
Found Yes
Hash 315cc5c8d14c51734c27ac9022809b9f52616641c404f1db1bce9efeea746987
SimHash 73275c626ab5

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

ahrefsbot
aspiegelbot
blexbot
barkrowler
dotbot
mj12bot
mauibot
nimbostratus-bot
petalbot
semrushbot
seznambot
sogou
serpstatbot
trendiction
textbulkerbot

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 180

gptbot
google-extended
anthropic-ai
bytespider
ccbot
chatgpt-user
cohere-ai
facebookbot
omgilibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://observalgerie.com/sitemap_index.xml

Warnings

  • 1 invalid line.