factual.com
robots.txt

Robots Exclusion Standard data for factual.com

Resource Scan

Scan Details

Site Domain factual.com
Base Domain factual.com
Scan Status Ok
Last Scan2024-11-15T16:44:17+00:00
Next Scan 2024-11-22T16:44:17+00:00

Last Scan

Scanned2024-11-15T16:44:17+00:00
URL http://factual.com/robots.txt
Redirect https://foursquare.com/robots.txt
Redirect Domain foursquare.com
Redirect Base foursquare.com
Domain IPs 146.75.30.132
Redirect IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Response IP 151.101.66.132
Found Yes
Hash bb5a00ee5639027d474ffa7ff4db956948d03b53b70a9877a819affdc4f127c0
SimHash 5a21ce304592

Groups

*

Rule Path
Disallow /search
Disallow /search?
Disallow /login?
Disallow /login/*
Disallow /signup?
Disallow /signup/*
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /signup/$
Disallow /mobile/
Disallow /touch/login
Disallow /mobile/search
Disallow /user/*/checkin/
Disallow /*/checkin/
Allow /v/checkin*
Disallow /private/wtrack
Disallow /l/
Disallow /*/badge/
Disallow /*/badges
Allow /v/badge*
Disallow /oauth2/
Disallow /device/
Disallow /venue/claim
Disallow /app/
Disallow /go/
Disallow /*/lists/edited$
Disallow /*/lists/followed$
Disallow /*/lists/friends$
Disallow /*/list/todos$
Disallow /*/list/tips$
Disallow /*/list/venuelikes$

branch metrics api
branch api

Rule Path
Allow /v/*

piplbot
mj12bot
ccbot
seznambot
exabot
netseer
mappy
crawler4j
gigabot
zoombot
bubing
getintent crawler
blexbot
trendictionbot
hyscore
magpie-crawler
rytebot

Rule Path
Disallow /v/*

gptbot
chatgpt-user

Rule Path
Disallow /

Other Records

Field Value
sitemap https://4sq-sitemap.s3.amazonaws.com/sitemap_index.xml