crowdworks.jp
robots.txt
Robots Exclusion Standard data for crowdworks.jp
Resource Scan
Scan Details
| Site Domain | crowdworks.jp |
| Base Domain | crowdworks.jp |
| Scan Status | Ok |
| Last Scan | 2025-12-14T02:36:34+00:00 |
| Next Scan | 2025-12-28T02:36:34+00:00 |
Last Scan
| Scanned | 2025-12-14T02:36:34+00:00 |
| URL | https://crowdworks.jp/robots.txt |
| Domain IPs | 18.181.1.211, 57.182.96.232 |
| Response IP | 57.182.96.232 |
| Found | Yes |
| Hash | 083b9dac53b807b599ab0c9fb18923728af34cbed93e424f8329fe8299592363 |
| SimHash | 482cfa148797 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api/ |
| Allow | /api/v3/public/ |
| Disallow | /attachments/ |
| Disallow | /oauth_clients/ |
| Disallow | /admin/ |
| Disallow | /cases/?attachment_id= |
| Disallow | /articles/2012/ |
| Disallow | /articles/2013/ |
| Disallow | /internal/ |
| Disallow | /deeplink/ |
| Disallow | /identification_request_* |
| Disallow | /public/proposal_products/winners/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://crowdworks.jp/sitemaps/public_employee_category/employee_caregory.xml.gz |
| sitemap | https://crowdworks.jp/sitemaps/public_employee_occupation/employee_occupation.xml.gz |
| sitemap | https://crowdworks.jp/sitemaps/public_employee_skill/employee_skill.xml.gz |
| sitemap | https://crowdworks.jp/sitemaps/public_job_category/job_category.xml.gz |
| sitemap | https://crowdworks.jp/sitemaps/public_job_skill/job_skill.xml.gz |
| sitemap | https://crowdworks.jp/sitemaps/sitemap_worker_detail/worker_detail.xml.gz |
| sitemap | https://crowdworks.jp/sitemaps/sitemap_client_detail/client_detail.xml.gz |
Comments