Log In Sign Up

projetweb.com
robots.txt

Robots Exclusion Standard data for projetweb.com

Resource Scan

Scan Details

Site Domain projetweb.com
Base Domain projetweb.com
Scan Status Ok
Last Scan2025-10-29T11:17:31+00:00
Next Scan 2025-11-05T11:17:31+00:00

Last Scan

Scanned2025-10-29T11:17:31+00:00
URL https://projetweb.com/robots.txt
Domain IPs 94.23.174.87
Response IP 94.23.174.87
Found Yes
Hash 884dac9470e86cfff0dd2eda6c2ce1eeb05acb1671c0cb48bb1ea54b05e29ddd
SimHash 001d84704f33

Groups

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

ia_archiver

Rule Path
Disallow /

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

oncrawl

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap http://projetweb.com/sitemap_mag.txt
sitemap http://projetweb.com/sitemap.txt
Back to top