home
hardware prices
news
articles
forums
photos
user reviews
Tech Support Forums - TechIMO.com
>
PC Hardware and Tech
>
Webmastering and Programming
Many Search Engine Crawlers on Site
Join TechIMO for Free!
User Name
Remember Me?
Password
Register
Blogs
FAQ
Members List
Calendar
Search
Today's Posts
Mark Forums Read
Get bargains at
»
Thread Tools
Currently Active Users
: 1900
Discussions: 188,398, Posts: 2,243,591, Members: 232,627
December 31st, 2004, 06:04 AM
#
1
(
permalink
)
ClubMed
Registered User
Join Date: Oct 2001
Posts: 4,097
Many Search Engine Crawlers on Site
Since about 2 days the search engine crawlers seem to be going nuts, I constantly have at least 10 going through the entire site.
Is it normal for this to happen during this time of year, or should I be worried?
Thanks in advance.
TechIMO.com Ads - Login or
register
for less ads.
Guest
,
Register Free!
to remove this ad and get your tech support questions answered in minutes!
January 2nd, 2005, 02:33 AM
#
2
(
permalink
)
large_nostril
Ultimate Member
Join Date: Dec 2004
Posts: 1,558
You shouldn't ever be worried about crawlers as long as they are from reputable search engines.
<EDIT>
You can always edit the robots.txt file if you find their behavior suspicious.
</EDIT>
«
Another PHP / Logging Question
|
Need Some Javascript Help!
»
Currently Active Users Viewing This Thread: 1
(0 members and 1 guests)
Thread Tools
Show Printable Version
Email this Page
Posting Rules
You
may
post new threads
You
may
post replies
You
may not
post attachments
You
may not
edit your posts
vB code
is
On
Smilies
are
On
[IMG]
code is
On
HTML code is
Off
Trackbacks
are
Off
Pingbacks
are
On
Refbacks
are
Off
Similar Threads
Thread
Thread Starter
Forum
Replies
Last Post
SHTML and PHP variables
FatalException
Webmastering and Programming
6
October 3rd, 2004
01:08 PM
I think this is the right place for this post???
Iknownothing
Technical Support
22
July 20th, 2004
11:12 AM
yw metasearch engine
kohsis
Tech News Discussion
2
August 26th, 2003
06:22 PM
Local search engine!
KenKun
Webmastering and Programming
7
June 30th, 2003
12:13 AM
Searching the forums
Theophylact
Suggestion Box
18
October 16th, 2002
09:32 AM
Most Active Discussions
Is It Just Me?
(
2904
)
Unarmed man on his stomach shot by ..
(
6
)
3-days in and no threads about Gaza
(
161
)
New Build ( Finally )
(
6
)
CPU wont boot
(
7
)
Building a gaming computer advice
(
5
)
I think I just killed my computer w..
(
24
)
Folderchat Weekday thread
(
444
)
Upgrading RAM
(
6
)
Recent Discussions
Left 4 Dead Small Freezes
(
3
)
RCA 52Inch HDTV wont turn on
(
4
)
wishin i could edit my aol prof..
(
0
)
For cheap price and good qualit..
(
0
)
Sporadic internet connectivity
(
2
)
Assassins Creed PC Problems
(
40
)
I think my PSU is dieing
(
1
)
building a gaming computer, inp..
(
0
)
Iming and surfing slowed down
(
3
)
Blackberry Storm, Gears of War ..
(
1
)
Core 2 Quad Q9550 system
(
3
)
COWBOOM Ripoff! Used Laptop w/$..
(
4
)
All times are GMT -4. The time now is
03:16 AM
.
TechIMO Copyright 2008
All Enthusiast, Inc.
Contact Us
-
Tech Support Forums - TechIMO.com
-
Archive
-
Privacy Statement
-
Terms of Use
-
Coupons and Deals and Dealighted
-
Store Ratings and Reviews
-
Web Business Blog
-
Top
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28