forexbitcoin.info
robots.txt

Robots Exclusion Standard data for forexbitcoin.info

Resource Scan

Scan Details

Site Domain forexbitcoin.info
Base Domain forexbitcoin.info
Scan Status Ok
Last Scan2024-05-25T02:08:33+00:00
Next Scan 2024-06-24T02:08:33+00:00

Last Scan

Scanned2024-05-25T02:08:33+00:00
URL https://forexbitcoin.info/robots.txt
Domain IPs 104.21.37.10, 172.67.202.44, 2606:4700:3033::6815:250a, 2606:4700:3034::ac43:ca2c
Response IP 172.67.202.44
Found Yes
Hash a29cfd374bf22537b8b8b23737467370b9936e7ef92ce272c7d0a22eba0a56f8
SimHash 48048a008521

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /cgi-bin
Disallow /wp-
Disallow /*?*
Allow */uploads
Allow /wp-*/*.css
Allow /wp-*/*.png
Allow /wp-*/*.gif
Allow /wp-*/*.js
Allow /wp-*/*.ttf
Allow /wp-*/*.woff
Allow */jquery/*.js

Other Records

Field Value
sitemap https://forexbitcoin.info/sitemap_index.xml