flippa.com
robots.txt

Robots Exclusion Standard data for flippa.com

Resource Scan

Scan Details

Site Domain flippa.com
Base Domain flippa.com
Scan Status Ok
Last Scan2024-11-16T13:16:30+00:00
Next Scan 2024-11-23T13:16:30+00:00

Last Scan

Scanned2024-11-16T13:16:30+00:00
URL https://flippa.com/robots.txt
Domain IPs 104.16.22.179, 104.16.23.179, 104.16.24.179, 104.16.25.179, 104.16.26.179
Response IP 104.16.22.179
Found Yes
Hash 5e7030037ef2f35a6f8840229db929033660a993be6d36b2090119f8352f1c6a
SimHash 6c0158049113

Groups

*

Rule Path
Disallow /users/
Disallow /auctions/*/buy-it-now$
Disallow /auctions/*/offers/make-an-offer$
Disallow /auctions/*/bids$

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.2

Other Records

Field Value
sitemap http://flippa-filestore-production.s3.amazonaws.com/sitemaps/sitemap.xml.gz