what is robots.txt  | | 
December 3rd, 2005, 04:16 PM
|
#1 (permalink)
| | Member
Join Date: Nov 2005 Location: Denver, Colorado
Posts: 244
|  what is robots.txt
I heard the tem robots.txt, I searched it up on Google but still don’t quite understand what it dose and how it works. Can someone, in plain English, please explain this to me? And how to implement it to my site. | 
|  | |
December 3rd, 2005, 11:03 PM
|
#2 (permalink)
| | Member
Join Date: Nov 2005 Location: Denver, Colorado
Posts: 244
|
i dont usually do this but.... bump.... please help. | |
| | |
December 3rd, 2005, 11:14 PM
|
#3 (permalink)
| | Real gangstas sip on Yacc
Join Date: Oct 2001 Location: Suckas-ville
Posts: 4,552
|
__________________
Signatures blow hard
If your signature contains an ad of any kind, congratulations, you're on my ignore list.
| |
| | |
December 3rd, 2005, 11:17 PM
|
#4 (permalink)
| | Member
Join Date: Nov 2005 Location: Denver, Colorado
Posts: 244
|
this is kind of the exact opposit of what i'm looking for. this link decribed how to stop search engins from finding your site... but i am looking for a way to get search engins to post my site. | |
| | |
December 3rd, 2005, 11:43 PM
|
#5 (permalink)
| | Real gangstas sip on Yacc
Join Date: Oct 2001 Location: Suckas-ville
Posts: 4,552
|
Robots.txt tells search engines that spider your site which pages NOT to include. If you want to get your site into search engines in the first place either link it somewhere or manually submit it. | |
| | |
December 3rd, 2005, 11:45 PM
|
#6 (permalink)
| | Member
Join Date: Jun 2004 Location: Utah
Posts: 423
|
I don't think the robot.txt helps any webcrawlers find your site; it just tells them how to behave when crawling your site.
If you want though, Here's links to the Google and Yahoo site submission pages, as well as some tips from google on how to best submit your site. | |
| | |
December 3rd, 2005, 11:52 PM
|
#7 (permalink)
| | :slack: strong
Join Date: Jan 2002 Location: MI
Posts: 17,395
|
Before you help him too much...
*cough* Quote: |
Originally Posted by lost-and-found |
from this thread...
http://www.techimo.com/forum/t157810.html
I believe he is the admin from the other forum... | |
| | |
December 4th, 2005, 12:33 AM
|
#8 (permalink)
| | Indispensable Member
Join Date: Oct 2001 Location: YeeHaw! Dallas
Posts: 18,650
|  Boy Det, you're a party pooper! He's trying to get his site set up! | |
| | |
December 4th, 2005, 12:34 AM
|
#9 (permalink)
| | :slack: strong
Join Date: Jan 2002 Location: MI
Posts: 17,395
|
Aren't I a nice guy...  | |
| | |
December 4th, 2005, 05:08 AM
|
#10 (permalink)
| | Member
Join Date: Aug 2004 Location: Hyderabad,India
Posts: 242
| Quote: |
Originally Posted by Lazywolf I don't think the robot.txt helps any webcrawlers find your site; it just tells them how to behave when crawling your site. | I am using robots.txt on my website. By using robots.txt, we can set some parameters for web crawlers of search engines like I have set them to visit after every 45 days 
[COPYRIGHTED: Indian
NO PART OF THIS POST SHOULD BE REPRODUCED WITHOUT MY PERMISSION]  | |
| | | Thread Tools | Search this Thread | | | | 
Similar Threads | | Thread | Thread Starter | Forum | Replies | Last Post | | Robots.txt? | Blazer06 | Webmastering and Programming | 3 | July 21st, 2004 06:24 AM | | Robots are cool | CERuppel | IMO Community | 1 | June 16th, 2004 03:07 AM | | We Are Robots | sixf00t4 | IMO Community | 2 | September 25th, 2003 09:23 AM | | Snake robots | Theophylact | Tech News Discussion | 1 | May 1st, 2003 12:07 PM | | Robots!! | brandon184 | IMO Community | 5 | August 29th, 2002 09:59 AM | |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | | | | Most Active Discussions | | | | |   Recent Discussions  | | | | |
 |
hardware
prices 
