manuf-rubber.com
robots.txt

Robots Exclusion Standard data for manuf-rubber.com

Resource Scan

Scan Details

Site Domain manuf-rubber.com
Base Domain manuf-rubber.com
Scan Status Ok
Last Scan2024-09-10T06:32:37+00:00
Next Scan 2024-10-10T06:32:37+00:00

Last Scan

Scanned2024-09-10T06:32:37+00:00
URL https://manuf-rubber.com/robots.txt
Redirect https://www.manuf-rubber.com/robots.txt
Redirect Domain www.manuf-rubber.com
Redirect Base manuf-rubber.com
Domain IPs 23.185.0.2, 2620:12a:8000::2
Redirect IPs 23.185.0.2, 2620:12a:8001::2
Response IP 23.185.0.2
Found Yes
Hash 95744f5fbdf83301c580a29c5f6d9686a2786bdea9df4311eeb1864eeeac5a9a
SimHash 68005f36e713

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /contact/request-a-quote/request-thank-you/
Disallow /contact/thank-you/
Disallow /quick-quote-thank-you/

Other Records

Field Value
sitemap https://www.manuf-rubber.com/sitemap_index.xml