Log In Sign Up

xhamster.com
robots.txt

Robots Exclusion Standard data for xhamster.com

Resource Scan

Scan Details

Site Domain xhamster.com
Base Domain xhamster.com
Scan Status Ok
Last Scan2024-06-10T22:35:39+00:00
Next Scan 2024-07-10T22:35:39+00:00

Last Scan

Scanned2024-06-10T22:35:39+00:00
URL https://xhamster.com/robots.txt
Domain IPs 104.17.34.109, 104.17.35.109, 2606:4700::6811:226d, 2606:4700::6811:236d
Response IP 104.17.35.109
Found Yes
Hash 8643fee792ce3f57345f4b083c0917ae5d68be7200cb2afc151ea49ef6dc7a8d
SimHash 2f00eb03e41a

Groups

*

Rule Path
Disallow *light%3D*
Disallow /movies/*/download/
Disallow /premium/
Disallow /users/
Disallow /posts/
Disallow /embed/
Allow /users/creators
Allow /user/xHamster/blog/*
Allow /user/xHamster/posts/*
Disallow /stories/*/print
Disallow *embedded_video*
Disallow /photos/gallery/*/
Allow /photos/gallery/*/2$
Allow /photos/gallery/*/3$
Allow /photos/gallery/*/4$
Allow /photos/gallery/*/5$
Allow /photos/gallery/*.html
Disallow /ab.log
Disallow /hot-7
Disallow /hot-30
Disallow /hot-7-pro
Disallow /hot-30-pro
Disallow /switch
Disallow *?max-duration=
Disallow *?min-duration=
Disallow *%26max-duration%3D
Disallow *%26min-duration%3D
Disallow *?geo=
Disallow */categories/*/best/daily
Disallow */categories/*/best/monthly
Disallow */categories/*/best/weekly
Disallow */categories/*/best/year-20
Disallow */categories/*/most-commented
Disallow */categories/*/most-viewed
Disallow */categories/*/producer
Disallow */categories/*/creator
Disallow */channels/*/newest
Disallow */channels/*/best
Disallow */channels/*/most-commented
Disallow */channels/*/most-viewed
Disallow */pornstars/*/newest
Disallow */pornstars/*/best
Disallow */pornstars/*/most-commented
Disallow */pornstars/*/most-viewed
Disallow */pornstars/*/exclusive
Disallow */creators/*/newest
Disallow */creators/*/best
Disallow */creators/*/most-commented
Disallow */creators/*/most-viewed
Disallow */creators/*/exclusive
Disallow */creators/*/full-length
Disallow */celebrities/*/newest
Disallow */celebrities/*/best
Disallow */celebrities/*/most-commented
Disallow */celebrities/*/most-viewed
Disallow */tags/*/newest
Disallow */tags/*/best
Disallow */tags/*/most-commented
Disallow */tags/*/most-viewed
Disallow */tags/*/producer
Disallow */tags/*/creator
Allow */categories/*/best
Allow */pornstars/all/*/best
Allow */creators/all/*/best
Allow */celebrities/all/*/best
Disallow */categories/*/best*?max-duration=
Disallow */categories/*/best*?min-duration=
Disallow */categories/*/best*%26max-duration%3D
Disallow */categories/*/best*%26min-duration%3D
Disallow /exit.php
Disallow */*/*/vr?*dimension=
Disallow */*/*/vr?*angle=
Disallow */categories/*fps%3D
Disallow */tags/*fps%3D
Disallow */channels/*fps%3D
Disallow */pornstars/*fps%3D
Disallow */creators/*fps%3D
Disallow */celebrities/*fps%3D
Disallow *%3Dproducer
Disallow *%3Dcreator
Disallow /signup?
Disallow /api/
Disallow */search/*cats%3D
Disallow */search/*correction%3D
Disallow */search/*date%3D
Disallow */search/*length%3D
Disallow */search/*orientations%3D
Disallow */search/*prod%3D
Disallow */search/*revert%3D
Disallow */search/*sort%3Ddate
Disallow */search/*sort%3Dlongest
Disallow */search/*sort%3Dviews
Disallow */search/*page%3D
Allow */search/*?page=2$
Allow */search/*?page=3$
Allow */search/*?page=4$
Allow */search/*?page=5$
Allow */search/*?page=6$
Allow */search/*?page=7$
Allow */search/*?page=8$
Allow */search/*?page=9$
Allow */search/*?page=10$
Allow */search/*?sort=best&page=2$
Allow */search/*?sort=best&page=3$
Allow */search/*?sort=best&page=4$
Allow */search/*?sort=best&page=5$
Allow */search/*?sort=best&page=6$
Allow */search/*?sort=best&page=7$
Allow */search/*?sort=best&page=8$
Allow */search/*?sort=best&page=9$
Allow */search/*?sort=best&page=10$
Disallow *?skip=
Disallow *%26skip%3D
Disallow /webcams/
Disallow /tracking?
Disallow *?x_platform_switch=*
Disallow /xembed.php
Disallow /top/*
Disallow /*/top/*
Disallow /livevideoamp/

ia_archiver

Rule Path
Disallow /
Back to top

Comments

  • Disallow all pagination
  • Allow search pages from 2 to 10
  • Allow best search pages from 2 to 10
Back to top