boafoda.com
robots.txt

Robots Exclusion Standard data for boafoda.com

Resource Scan

Scan Details

Site Domain boafoda.com
Base Domain boafoda.com
Scan Status Ok
Last Scan2024-05-10T04:22:35+00:00
Next Scan 2024-06-09T04:22:35+00:00

Last Scan

Scanned2024-05-10T04:22:35+00:00
URL https://boafoda.com/robots.txt
Redirect https://www.boafoda.com/robots.txt
Redirect Domain www.boafoda.com
Redirect Base boafoda.com
Domain IPs 104.18.4.72, 104.18.5.72
Redirect IPs 104.18.4.72, 104.18.5.72
Response IP 104.18.5.72
Found Yes
Hash 4d07b710f24cb013e68980e765b1826a0d97dbbe7c42900caf05f44b5bc54e8d
SimHash df0baa40ef18

Groups

archive.org_bot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

*

Rule Path
Disallow /comments/
Disallow /external/
Disallow /pornstars/load_more_pornstars
Disallow /pornstars/return_rendered_pornstars
Disallow /posts/load_more_posts
Disallow /posts/return_rendered_posts
Disallow /studios/load_more_studios
Disallow /studios/return_rendered_studios
Disallow /posts/load_related_posts
Disallow /posts/load_related_miss_tubesite_videos
Disallow /celebrity/load_more_media
Disallow /celebrity/load_more_feeds
Disallow /celebrity/return_rendered_celebrities
Disallow /celebrity/return_rendered_feeds
Disallow /celebrity/return_rendered_medias
Disallow /api/video_tag_specific_video/
Disallow /api/video_get_info/
Disallow /user/
Disallow /offline/
Disallow /show_404/
Disallow /*?utm*
Disallow /disabled_videos/
Disallow /disabled_videos
Disallow /obfs/
Disallow /adv/
Disallow /videos/0/
Disallow /videos/-*
Disallow /services/user_info
Disallow */galleries/
Disallow /video_studio
Disallow /ads.txt
Disallow /random
Disallow /home/load_more_text/
Disallow /.well-known/
Disallow /video_pornstar
Disallow /geo/mobile_menu/
Disallow */upload
Disallow */activity
Disallow */likes
Disallow */history
Disallow */photo-gif
Disallow /swipe/
Disallow /cdn-cgi/*
Disallow /webcams/full_webcam_page
Disallow /webcams/full_dating_page
Disallow /webcams/interstitial_page
Disallow /webcams/full_webcam_page/*
Disallow /webcams/full_dating_page/*
Disallow /webcams/interstitial_page/*