grands-reportages.com
robots.txt

Robots Exclusion Standard data for grands-reportages.com

Resource Scan

Scan Details

Site Domain grands-reportages.com
Base Domain grands-reportages.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-12T01:51:54+00:00
Next Scan 2024-05-26T01:51:54+00:00

Last Successful Scan

Scanned2024-04-27T01:49:01+00:00
URL https://grands-reportages.com/robots.txt
Redirect https://www.grands-reportages.com/robots.txt
Redirect Domain www.grands-reportages.com
Redirect Base grands-reportages.com
Domain IPs 104.21.66.11, 172.67.198.12, 2606:4700:3034::ac43:c60c, 2606:4700:3035::6815:420b
Redirect IPs 104.21.66.11, 172.67.198.12, 2606:4700:3034::ac43:c60c, 2606:4700:3035::6815:420b
Response IP 104.21.66.11
Found Yes
Hash e5a01bbfef7a3de63633f289b6974c93dd98feb93332ee79e5c3c0d961dcbd86
SimHash cd6c5c400543

Groups

*

Rule Path
Disallow /admin/
Disallow /widget/
Disallow /xwidget/
Disallow *sort%3D*
Disallow *resultat-recherche*
Disallow *resultats-recherche*
Disallow /portqr-usa-sanfrancisco
Disallow /portqr-usa-cowboy
Disallow /portqr-usa-navajos
Disallow /portqr-usa-universal
Disallow /portqr-usa-newyork
Disallow /portqr-usa-philadelphie
Disallow /portqr-usa-lasvegas
Disallow /maintenance
Disallow /concours-dites-nous-ou-cette-photo-prise
Disallow /actu-directravel-salon-voyage-direct-22-24-septembre-paris
Disallow /app-store
Disallow /voyage-annee-votes

nutch

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.grands-reportages.com/sitemap.xml

Comments

  • User rules