gifts.com
robots.txt

Robots Exclusion Standard data for gifts.com

Resource Scan

Scan Details

Site Domain gifts.com
Base Domain gifts.com
Scan Status Ok
Last Scan2024-05-10T12:41:52+00:00
Next Scan 2024-05-24T12:41:52+00:00

Last Scan

Scanned2024-05-10T12:41:52+00:00
URL https://gifts.com/robots.txt
Redirect https://www.gifts.com/robots_https.txt
Redirect Domain www.gifts.com
Redirect Base gifts.com
Domain IPs 18.155.68.126, 18.155.68.20, 18.155.68.41, 18.155.68.53
Redirect IPs 18.155.68.126, 18.155.68.20, 18.155.68.41, 18.155.68.53
Response IP 18.155.68.41
Found Yes
Hash c68f9a243bfaf647689678030a85731857257e922648b4464bb8645d5ab81eab
SimHash 68545f51e002

Groups

googlebot

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

spinn3r

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

applebot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://www.gifts.com/sitemap.xml
sitemap https://www.gifts.com/blog/sitemap_index.xml
sitemap https://www.gifts.com/categories-sitemap.xml
sitemap https://www.gifts.com/products-sitemap.xml

Warnings

  • 2 invalid lines.