dlapiper.com
robots.txt

Robots Exclusion Standard data for dlapiper.com

Resource Scan

Scan Details

Site Domain dlapiper.com
Base Domain dlapiper.com
Scan Status Ok
Last Scan2024-05-25T03:10:48+00:00
Next Scan 2024-06-24T03:10:48+00:00

Last Scan

Scanned2024-05-25T03:10:48+00:00
URL https://dlapiper.com/robots.txt
Redirect https://www.dlapiper.com/robots.txt
Redirect Domain www.dlapiper.com
Redirect Base dlapiper.com
Domain IPs 20.50.2.40
Redirect IPs 125.56.219.26, 125.56.219.34, 2600:1413:b000:14::b857:c153, 2600:1413:b000:14::b857:c159
Response IP 125.56.219.64
Found Yes
Hash 1b9a3a0f66e96eaa2e688fbb03e500f73476515469c50de746b564d471689719
SimHash 2380b8f4c414

Groups

*

Rule Path
Disallow /*?format=vCard
Disallow /*?ajax=1
Disallow /sitecore
Disallow /sitecore_files/
Disallow /App_Browsers/
Disallow /App_config/
Disallow /App_Data/
Disallow /temp/
Disallow /upload/
Disallow /xsl/
Disallow /Tasks/
Disallow /SitecoreDesktopSearchWeb/

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

linguee bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://www.dlapiper.com/sitemap.xml

Comments

  • Baiduspider
  • Yandex
  • Linguee
  • rogerbot
  • Bingbot
  • Slurp

Warnings

  • 1 invalid line.