home hardware prices news articles forums photos user reviews
Go Back   Tech Support Forums - TechIMO.com > PC Hardware and Tech > Webmastering and Programming
Ask a Tech Support Question (free)!

DB Index optimization

Reply
Get bargains at  »  Dealighted.com
 
Thread Tools Search this Thread
Currently Active Users: 2178
Discussions: 200,919, Posts: 2,379,027, Members: 246,286
Old October 10th, 2002, 05:44 PM   Digg it!   #1 (permalink)
Not Really a Member
 
Join Date: Oct 2001
Posts: 25,380
DB Index optimization

I have a whopper of a table that is roughly 10-11 GIGS
(one table) in sql server.
No I can't parse out the data into more tables because its not really a relational database its just used to track netstat information.

Ok my problem is I'm trying to run a query to clean up the data but its going god awful slow.

I put an index on the columns I'm using but that didn't help so I'm wondering if its because I concatenate the columns together in the query.

My query
select connectionid
from connections
where servername + RemoteAddress + LocalPort = 'servernameipinfolocalport'

Now my index is...

CREATE
INDEX [idx_CleanupIndex] ON [dbo].[Connections] ([ServerName], [RemoteAddress], [LocalPort])
WITH
DROP_EXISTING
ON [PRIMARY]

Would it help if I changed the index to

CREATE
INDEX [idx_CleanupIndex] ON [dbo].[Connections] ([ServerName] + [RemoteAddress] + [LocalPort])
WITH
DROP_EXISTING
ON [PRIMARY]

I don't just want to whack it and give it a shot because it will take roughly 20-30 mins to create the stupid index and HD space is quite limited (because of this Db lol)

Now this is on a 4 proc server with 1.5 gigs of RAM.. this should NOT take that long!!! lol

This thing has roughly 63,952,419 rows of data so yea it will take some time but jeeeeez.
SQL Server 2k isn't THAT pathetic is it?!
vass0922 is online now   Reply With Quote
Old October 10th, 2002, 06:27 PM     #2 (permalink)
Not Really a Member
 
Join Date: Oct 2001
Posts: 25,380
Ok so
CREATE
INDEX [idx_CleanupIndex] ON [dbo].[Connections] ([ServerName] + [RemoteAddress] + [LocalPort])
WITH
DROP_EXISTING
ON [PRIMARY]

doesn't work
It doesn't like the Pluses

If I do a query of
select connectionid
from connections
where servername = 'servername' and RemoteAddress = 'ipinfo'
and LocalPort = 65465 it comes up real fast like it should!
But concatenated its too slow
*sigh* well if anybody has any ideas, I am gonna see if I can restructure my stored procedure a bit.
__________________
Helicopters don't fly; they vibrate so much and make so much noise that the earth rejects them.
vass0922 is online now   Reply With Quote
Old October 10th, 2002, 08:10 PM     #3 (permalink)
Not Really a Member
 
Join Date: Oct 2001
Posts: 25,380
I was able to re-structure the queries and the Stored Procedure to take better advantage of the Index so it may work out after all. Works MUCH better now
however, if you know how to do the above (index for concatenated columns) would still be quite interested in that knowledge
vass0922 is online now   Reply With Quote
Old October 10th, 2002, 11:50 PM     #4 (permalink)
Member
 
Join Date: Sep 2002
Posts: 364
I was going to suggest trying an SP. Do you always create your indexes in code? I just make them in the designer....
Creosote is offline   Reply With Quote
Old October 11th, 2002, 01:03 AM     #5 (permalink)
Not Really a Member
 
Join Date: Oct 2001
Posts: 25,380
I created them in the designer but all that does is write the sql for you
I'm out of habit so can't create them by code by myself anymore lol

And even when I ran it as a compiled SP it only went through 14k rows in 18 hours lol
NOT GOOD!

I changed my methodology a little, so now I populate a table with Id's that I can delete out of the table and already up to 176k
only 63.6 million left LOL
vass0922 is online now   Reply With Quote
Reply
Thread Tools Search this Thread
Search this Thread:

Advanced Search


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Most Active Discussions
Is It Just Me? (2839)
Why is Khalid Sheikh Mohammed even .. (9)
Obama the Muslim (9)
Is the PSU I received dead? (10)
windows vista security holes (7)
Install XP pro and a Vista laptop ?.. (9)
HIS HD5770 graphic card question (15)
Foreign voltage (8)
A good PSU? (10)
Dept. of HS: NSA 'Helped' Develop V.. (15)
Print spooler problem (10)
New Computer wont recognize XP disc (7)
Ideal cheap graph card for PC-Gamin.. (15)
EVGA 9800 gtx help with finding a g.. (8)
Recent Discussions
Browsers wont load websites (1)
Help getting around port 80 for camer.. (1)
[F@H SPAM 11/16/09] ! 1/2 months to r.. (31)
Foreign voltage (8)
Dept. of HS: NSA 'Helped' Develop Vis.. (15)
windows vista security holes (7)
Install XP pro and a Vista laptop ?? (9)
Virus Doctor Popup? (0)
EVGA 9800 gtx help with finding a goo.. (8)
Modern Warfare For the PC (32)
Problem with speed step/turbo boost? (1)
monitor will not turn on at all, (0)
Modern Warfare 2: Who Bought It? (61)
World's largest Monopoly Game using G.. (330)
Print spooler problem (10)
SIS 740 and Widescreen (8)
Baffling Problem with my CPU/MoBo's. .. (0)
Display shows 3x5 inch in middle of s.. (0)
HIS HD5770 graphic card question (15)
Best file format to play on Windows H.. (0)
PSP Go bought in Japan (0)
Asus P4G8X Mobo (3)
Need hard disk drivers (4)
windows 7 internet problem (4)
What OS for a home server? (other tha.. (1)


All times are GMT -4. The time now is 04:35 PM.
TechIMO Copyright 2009 All Enthusiast, Inc.



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28