lapauseinfo.fr
robots.txt
Robots Exclusion Standard data for lapauseinfo.fr
Resource Scan
Scan Details
| Site Domain | lapauseinfo.fr |
| Base Domain | lapauseinfo.fr |
| Scan Status | Ok |
| Last Scan | 2024-11-08T22:43:59+00:00 |
| Next Scan | 2024-11-15T22:43:59+00:00 |
Last Scan
| Scanned | 2024-11-08T22:43:59+00:00 |
| URL | https://lapauseinfo.fr/robots.txt |
| Domain IPs | 104.21.95.233, 172.67.149.14, 2606:4700:3031::6815:5fe9, 2606:4700:3031::ac43:950e |
| Response IP | 104.21.95.233 |
| Found | Yes |
| Hash | 99006c369e286fb2216d6d36abc36f2f4eb4f57841863da2ae8997cbae1d04d1 |
| SimHash | 7b15584a2ab1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-content/plugins/link-juice-optimizer/public/js/link-juice-optimizer.js |
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /wp-login.php |
| Disallow | /wp-signup.php |
| Disallow | /press-this.php |
| Disallow | /remote-login.php |
| Disallow | /activate/ |
| Disallow | /cgi-bin/ |
| Disallow | /mshots/v1/ |
| Disallow | /next/ |
| Disallow | /public.api/ |
| Allow | /*css?* |
| Allow | /*js?* |
| Disallow | /*.json$ |
awariosmartbot
aspiegelbot
blexbot
barkrowler
dotbot
mj12bot
mauibot
nimbostratus-bot
petalbot
seznambot
sogou
serpstatbot
trendiction
criteobot
twitterbot
aspiegelbot
blexbot
barkrowler
dotbot
mj12bot
mauibot
nimbostratus-bot
petalbot
seznambot
sogou
serpstatbot
trendiction
textbulkerbot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 180 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://lapauseinfo.fr/sitemaps.xml |
| sitemap | https://lapauseinfo.fr/sitemap-news.xml |
Warnings
- 1 invalid line.