tokyo.com
robots.txt
Robots Exclusion Standard data for tokyo.com
Resource Scan
Scan Details
| Site Domain | tokyo.com |
| Base Domain | tokyo.com |
| Scan Status | Ok |
| Last Scan | 2025-02-22T12:54:40+00:00 |
| Next Scan | 2025-03-24T12:54:40+00:00 |
Last Scan
| Scanned | 2025-02-22T12:54:40+00:00 |
| URL | https://tokyo.com/robots.txt |
| Domain IPs | 104.26.14.117, 104.26.15.117, 172.67.73.247, 2606:4700:20::681a:e75, 2606:4700:20::681a:f75, 2606:4700:20::ac43:49f7 |
| Response IP | 104.26.14.117 |
| Found | Yes |
| Hash | a36603ba3e82a58fd4a1f44ac4500c7b3121408bc52043bc8f97bca74e8967e4 |
| SimHash | fa58dc8d01b2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-login.php |
| Disallow | /jp/wp-login.php |
| Disallow | /sc/wp-login.php |
| Disallow | /wp-admin/ |
| Disallow | /wp-register.php |
| Disallow | /wp-login.php?action=lostpassword |
| Disallow | /index.php?paged |
| Disallow | /index.php?s |
| Disallow | /wp-trackback |
| Disallow | /xmlrpc |
| Disallow | /?ref |
| Disallow | /?author |
| Disallow | /sc/itinerary |
| Disallow | /jp/itinerary |
| Disallow | /itinerary |
| Disallow | /jp/listing-tag |
| Disallow | /sc/listing-tag |
| Disallow | /listing-tag |
| Disallow | /?post_type=job_listing |
| Disallow | /jp/?post_type=job_listing |
| Disallow | /sc/?post_type=job_listing |
| Disallow | /jp/tag/ |
| Disallow | /sc/tag/ |
| Disallow | /tag/ |
| Disallow | /search/ |
| Disallow | /jp/search/ |
| Disallow | /sc/search/ |
googlebot
| Rule | Path |
|---|---|
| Disallow | /wp-login.php |
| Disallow | /jp/wp-login.php |
| Disallow | /sc/wp-login.php |
| Disallow | /wp-admin/ |
| Disallow | /wp-register.php |
| Disallow | /wp-login.php?action=lostpassword |
| Disallow | /index.php?paged |
| Disallow | /index.php?s |
| Disallow | /wp-trackback |
| Disallow | /xmlrpc |
| Disallow | /*%26action%3Dprint |
| Disallow | /*%26ref |
| Disallow | /?ref |
| Disallow | /?author |
| Disallow | /sc/itinerary |
| Disallow | /jp/itinerary |
| Disallow | /itinerary |
| Disallow | /jp/listing-tag |
| Disallow | /sc/listing-tag |
| Disallow | /listing-tag |
| Disallow | /jp/tag/ |
| Disallow | /sc/tag/ |
| Disallow | /tag/ |
| Disallow | /?post_type=job_listing |
| Disallow | /jp/?post_type=job_listing |
| Disallow | /sc/?post_type=job_listing |
| Disallow | /search/ |
| Disallow | /jp/search/ |
| Disallow | /sc/search/ |
slurp
| Rule | Path |
|---|---|
| Disallow | /wp-login.php |
| Disallow | /jp/wp-login.php |
| Disallow | /sc/wp-login.php |
| Disallow | /wp-admin/ |
| Disallow | /wp-register.php |
| Disallow | /wp-login.php?action=lostpassword |
| Disallow | /index.php?paged |
| Disallow | /index.php?s |
| Disallow | /wp-trackback |
| Disallow | /xmlrpc |
| Disallow | /*%26action%3Dprint |
| Disallow | /*%26ref |
| Disallow | /?ref |
| Disallow | /?author |
| Disallow | /sc/itinerary |
| Disallow | /jp/itinerary |
| Disallow | /itinerary |
| Disallow | /jp/listing-tag |
| Disallow | /sc/listing-tag |
| Disallow | /listing-tag |
| Disallow | /?post_type=job_listing |
| Disallow | /jp/?post_type=job_listing |
| Disallow | /sc/?post_type=job_listing |
| Disallow | /jp/tag/ |
| Disallow | /sc/tag/ |
| Disallow | /tag/ |
| Disallow | /search/ |
| Disallow | /jp/search/ |
| Disallow | /sc/search/ |
msnbot
| Rule | Path |
|---|---|
| Disallow | /wp-login.php |
| Disallow | /jp/wp-login.php |
| Disallow | /sc/wp-login.php |
| Disallow | /wp-admin/ |
| Disallow | /wp-register.php |
| Disallow | /wp-login.php?action=lostpassword |
| Disallow | /index.php?paged |
| Disallow | /index.php?s |
| Disallow | /wp-trackback |
| Disallow | /xmlrpc |
| Disallow | /*%26action%3Dprint |
| Disallow | /*%26ref |
| Disallow | /?ref |
| Disallow | /*comment |
| Disallow | /?author |
| Disallow | /sc/itinerary |
| Disallow | /jp/itinerary |
| Disallow | /itinerary |
| Disallow | /jp/listing-tag |
| Disallow | /sc/listing-tag |
| Disallow | /listing-tag |
| Disallow | /?post_type=job_listing |
| Disallow | /jp/?post_type=job_listing |
| Disallow | /sc/?post_type=job_listing |
| Disallow | /jp/tag/ |
| Disallow | /sc/tag/ |
| Disallow | /tag/ |
| Disallow | /search/ |
| Disallow | /jp/search/ |
| Disallow | /sc/search/ |