marqeta.com
robots.txt

Robots Exclusion Standard data for marqeta.com

Resource Scan

Scan Details

Site Domain marqeta.com
Base Domain marqeta.com
Scan Status Ok
Last Scan2024-05-18T12:44:05+00:00
Next Scan 2024-06-17T12:44:05+00:00

Last Scan

Scanned2024-05-18T12:44:05+00:00
URL https://marqeta.com/robots.txt
Redirect https://www.marqeta.com/robots.txt
Redirect Domain www.marqeta.com
Redirect Base marqeta.com
Domain IPs 15.197.167.90, 3.33.186.135
Redirect IPs 13.215.31.72, 175.41.180.202, 2406:da18:b3d:e201::1f4, 2406:da18:b3d:e202::1f4
Response IP 175.41.180.202
Found Yes
Hash 27c5f58321899dd75b01ed0c0c6a982ca868d7574e0514b96bf452f360857600
SimHash 0c70997a2303

Groups

*

Rule Path
Allow /

Comments

  • Allow all user agents.
  • User agent "Googlebot" should not crawl the `http://example.com/nogooglebot/` folder or any of its subdirectories.
  • User-agent: Googlebot
  • Disallow: /nogooglebot/