dsservices.com
robots.txt

Robots Exclusion Standard data for dsservices.com

Resource Scan

Scan Details

Site Domain dsservices.com
Base Domain dsservices.com
Scan Status Ok
Last Scan2024-09-15T05:05:24+00:00
Next Scan 2024-10-15T05:05:24+00:00

Last Scan

Scanned2024-09-15T05:05:24+00:00
URL https://www.dsservices.com/robots.txt
Domain IPs 45.33.69.166
Response IP 45.33.69.166
Found Yes
Hash 4eea0cd91a6fd297a6bf129e726cc09a8c0a7f0dbf31c513e291eea53f0e06ee
SimHash 4055d45355b1

Groups

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googlebot-image

Rule Path
Disallow
Disallow /*.pdf

Other Records

Field Value
crawl-delay 1

googlebot-mobile

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

w3c-checklink

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

twitterbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

accessiblewebbot

Rule Path
Disallow

facebookexternalhit

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

dotbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

powermapper

Rule Path
Allow /

rogerbot

Rule Path
Disallow

nerdybot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

scrapybot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dsservices.com/sitemap/sitemap.xml