Log In Sign Up

motopinas.com
robots.txt

Robots Exclusion Standard data for motopinas.com

Resource Scan

Scan Details

Site Domain motopinas.com
Base Domain motopinas.com
Scan Status Ok
Last Scan2024-10-04T19:25:19+00:00
Next Scan 2024-10-11T19:25:19+00:00

Last Scan

Scanned2024-10-04T19:25:19+00:00
URL https://motopinas.com/robots.txt
Redirect https://www.motopinas.com/robots.txt
Redirect Domain www.motopinas.com
Redirect Base motopinas.com
Domain IPs 104.21.87.185, 172.67.145.154, 2606:4700:3035::ac43:919a, 2606:4700:3036::6815:57b9
Redirect IPs 104.21.87.185, 172.67.145.154, 2606:4700:3035::ac43:919a, 2606:4700:3036::6815:57b9
Response IP 104.21.87.185
Found Yes
Hash 50703933bbe8f56e98ab8acb52d984ab017d7d499df2ea13672f46dc17bc3123
SimHash 80135a82e971

Groups

twitterbot

Rule Path
Allow /images/

facebookexternalhit

Rule Path
Allow /

googlebot-image

Rule Path
Allow /images/

googlebot

Rule Path
Allow /images/

spider

Rule Path
Disallow /

bot-

Rule Path
Disallow /

bot/

Rule Path
Disallow /

linkchecker

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

java

Rule Path
Disallow /

nicebot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

powermarks

Rule Path
Disallow /

missigua_locator

Rule Path
Disallow /

web downloader

Rule Path
Disallow /

lanshanbot

Rule Path
Disallow /

custo

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

httrack off-line browser

Rule Path
Disallow /

nutchcvs

Rule Path
Disallow /

t-h-u-n-d-e-r-s-t-o-n-e

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

htmlparser

Rule Path
Disallow /

crawl

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

larbin

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

npbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

Rule Path
Disallow /cron.php
Back to top

Other Records

Field Value
sitemap https://www.motopinas.com/sitemap.xml
Back to top