clickworker.de
robots.txt

Robots Exclusion Standard data for clickworker.de

Resource Scan

Scan Details

Site Domain clickworker.de
Base Domain clickworker.de
Scan Status Ok
Last Scan2024-11-19T15:40:26+00:00
Next Scan 2024-12-03T15:40:26+00:00

Last Scan

Scanned2024-11-19T15:40:26+00:00
URL https://clickworker.de/robots.txt
Redirect https://www.clickworker.de/robots.txt
Redirect Domain www.clickworker.de
Redirect Base clickworker.de
Domain IPs 104.21.41.116, 172.67.146.230, 2606:4700:3031::6815:2974, 2606:4700:3031::ac43:92e6
Redirect IPs 104.21.41.116, 172.67.146.230, 2606:4700:3031::6815:2974, 2606:4700:3031::ac43:92e6
Response IP 104.21.41.116
Found Yes
Hash 75e1925da237cfb3b4da7b46cff49473732fa19651639de3fee576ff65280e79
SimHash 4895d8120933

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-admin/admin-ajax.php
Disallow /apple-app-site-association
Disallow /.well-known
Disallow /product-wizard
Disallow /tagesdateien01
Disallow /community_advent_calendar/

surveybot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

semager

Rule Path
Disallow /

linguee

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

zeusbot

Rule Path
Disallow /

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.clickworker.de/sitemap_index.xml/