goldposter.com
robots.txt

Robots Exclusion Standard data for goldposter.com

Resource Scan

Scan Details

Site Domain goldposter.com
Base Domain goldposter.com
Scan Status Ok
Last Scan2024-09-20T17:31:11+00:00
Next Scan 2024-09-27T17:31:11+00:00

Last Scan

Scanned2024-09-20T17:31:11+00:00
URL https://goldposter.com/robots.txt
Domain IPs 104.21.8.129, 172.67.157.93, 2606:4700:3034::6815:881, 2606:4700:3035::ac43:9d5d
Response IP 172.67.157.93
Found Yes
Hash 3730713597ed9add33b3920dbbc8c5edd0ca66d913e4a0f6123250a9ed6219a6
SimHash 7b5ed072ea23

Groups

*

Rule Path
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-content/cache/wp-rocket/
Disallow /author/
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow */trackback/
Disallow */feed/
Disallow */comments/
Disallow /attachment/
Disallow /?s=*
Disallow /?q=*
Disallow /?p=*
Disallow /en/
Disallow /*/?s=*%5C
Disallow /wp-json/
Disallow /?wpfpaction=*
Disallow /?gpaction=*
Disallow /sitemap_index.xml
Disallow /search-goldposter/

ahrefsbot

Rule Path
Disallow /

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

sogou web spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

extlinksbot

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

leiki

Rule Path
Disallow /

webmeup

Rule Path
Disallow /