ditto.jobs
robots.txt
Robots Exclusion Standard data for ditto.jobs
Resource Scan
Scan Details
| Site Domain | ditto.jobs |
| Base Domain | ditto.jobs |
| Scan Status | Ok |
| Last Scan | 2024-09-23T21:02:48+00:00 |
| Next Scan | 2024-09-30T21:02:48+00:00 |
Last Scan
| Scanned | 2024-09-23T21:02:48+00:00 |
| URL | https://ditto.jobs/robots.txt |
| Redirect | https://www.ditto.jobs/robots.txt |
| Redirect Domain | www.ditto.jobs |
| Redirect Base | ditto.jobs |
| Domain IPs | 76.76.21.21 |
| Redirect IPs | 76.76.21.142, 76.76.21.93 |
| Response IP | 76.76.21.142 |
| Found | Yes |
| Hash | 700f5d61059d60a0c8e835d0fc3280df43bf440b359845e372e5525a56c80c8c |
| SimHash | 2c01b95477f4 |
Groups
googlebot
adsbot-google
adsbot-google-mobile
mediapartners-google
bingbot
bingbot
bingpreview
slurp
duckduckbot
baiduspider
baidu
yandexbot
facebot
facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
msnbot
msnbot
mozilla/5.0 (compatible; jooblebot/2.0; windows nt 6.1; wow64; +http://jooble.org/jooble-bot) applewebkit/537.36 (khtml, like gecko) safari/537.36
mozilla/5.0 (compatible; jooblebot/2.0; windows nt 6.1; wow64; +http://jooble.org/jooblebot) applewebkit/537.36 (khtml, like gecko) safari/537.36
twitterbot
| Rule | Path |
|---|---|
| Allow | /api/* |
| Disallow | /candidate-profile |
| Disallow | /user/profile |
| Disallow | /user/settings |
| Disallow | /user/redirect |
| Disallow | |
| Disallow |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.ditto.jobs/sitemap.xml |