webcalist.com
robots.txt
Robots Exclusion Standard data for webcalist.com
Resource Scan
Scan Details
| Site Domain | webcalist.com |
| Base Domain | webcalist.com |
| Scan Status | Ok |
| Last Scan | 2025-11-20T19:54:07+00:00 |
| Next Scan | 2025-12-20T19:54:07+00:00 |
Last Scan
| Scanned | 2025-11-20T19:54:07+00:00 |
| URL | https://webcalist.com/robots.txt |
| Domain IPs | 104.21.43.6, 172.67.215.13, 2606:4700:3033::ac43:d70d, 2606:4700:3035::6815:2b06 |
| Response IP | 104.21.43.6 |
| Found | Yes |
| Hash | 62fdb89818d3db29b4e582ca4488c9121afba00de4c4b4dcf6f815a2fdc786f6 |
| SimHash | 4f09670086b0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | |
| Disallow | /users/ |
| Disallow | /my/ |
| Disallow | /register/ |
| Disallow | /logout/ |
| Disallow | /search/video* |
| Disallow | /goto/ |
| Disallow | /genders/ |
| Disallow | /best/ |
| Disallow | /most-viewed/ |
| Disallow | /most-commented/ |
| Disallow | /rooms/*/best/daily |
| Disallow | /rooms/*/best/weekly |
| Disallow | /rooms/*/best/monthly |
| Disallow | /rooms/*/best/year-20 |
| Disallow | /rooms/*/most-viewed |
| Disallow | /rooms/*/most-commented |
| Allow | /rooms/* |
| Allow | /rooms/*/best |
| Disallow | /tags/*/best |
| Disallow | /tags/*/most-viewed |
| Disallow | /tags/*/most-commented |
| Allow | /tags/* |
| Disallow | /info/faq |
| Disallow | /categories |
| Disallow | /_debugbar |
Other Records
| Field | Value |
|---|---|
| sitemap | https://webcalist.com/sitemap.xml |