I have recently run into a problem with SQL tables where I had a mistake in my code that dropped a column in the table and after that recreated the column. This process was repeated many times until I discovered it.
But as a result it appears that the SQL table haven't properly deleted the columns. For instance I get this message when I want to add a new column:
"Warning: The table "Matches" has been created, but its maximum row size exceeds the allowed maximum of 8060 bytes. INSERT or UPDATE to this table will fail if the resulting row exceeds the size limit."
And I also have trouble updating column values because they exceed the maximum limits.
And to be clear, the table does not have more than 30 (visible) rows and I only made each column a maximum of 40 varchars, so it appears that the dropped rows still exist somewhere. But how do I delete them?
Thanks in advance.
With the H2 database, suppose there is a SUMS table that has a key and several count fields and there is an UPDATES table which has the same key and count fields. The keys in the UPDATES table may or may not exist in the SUMS table.
What is the most efficient way to add all the counts for each key from the UPDATES table to the SUM table, or insert a row with those counts if the SUMS table does not yet have it?
Of course I could always process the result set of a select on the UPDATES table and then one-by-one update or insert into the SUMS table, but this feels like there should be a more efficient way to do it.
If it is not possible in H2 but possible in some other Java-embeddable solution I would be interested in this too, because this processing is just an intermediate step for processing a larger number of these counts (a couple of dozen million keys and a couple of billion rows for updating them).
What is a best practice to get around SSAS 2 billion distinct value limitation on a column. My data set grows by 2 billions rows every 10 month and one of the measures in the cube is a row count that runs on the PK. Because adding partitions does not help resolve the problem, would creating new cubes with identical info be the right approach?
Do you have to run the count on a column? This should only be necessary if you have duplicates in your table. Could you not just count rows? Then just omit the primary key column.
My table has the following fields:
Date (integer)
State (integer)
ProductId (integer)
ProductName (integer)
Description (text) (maximum text lenght
3000 characters)
There will be more than 8 million rows. I need to decide whether I should put the product description in another table. My main goal is to have this statement very fast:
SELECT Date,State,ProductId,ProductName FROM tablename ORDER BY DATE desc LIMIT 100
The SQL result will not fetch the Description field value in the above statement. The user will see the description only when the row is selected in the application (new query).
I would really want to have the product Description in the same table, but I'm not sure how SQLite scans the rows. If Date value doesn't match I would assume that SQLite can quickly skip to the next row. Or maybe it needs to scan all fields of the row till it gets to the end of the Description field value in order to know that the row has ended? If it needs to scan all fields to get to the next row will the value of 3000 characters in the Description field decrease the speed a lot?
EDIT: No indexing should be used since INSERT speed is important.
EDIT: The only reason of trying to have it all in one table is that I want to do INSERTs and UPDATEs in one transaction of hundreds of items. The same item could be inserted and later updated in the same transaction, so I can not know the last insert id per item.
When you use that query and do not have an index on the Date column, SQLite will read all records from the table, and use a temporary table to sort the result.
When you have an index on the Date column, SQLite will look up the last 100 records in the index, then read all the data of those records from the table.
When you have a covering index, i.e., one index with the four columns Date, State, ProductId, and ProductName, SQLite will just read the last 100 entries from the index.
Whenever SQLite reads from the database file, it does not read values or records, but entire pages (typically, 1 KB or 4 KB).
In case 1, SQLite will read all pages of the table.
In case 2, SQLite will read the last page of the index (because the 100 dates will fit into one page), and 100 pages of the table (one for each record, assuming that no two of these records happen to be in the same page).
In case 3, SQLite will read the last few pages of the index.
Case 2 will be much faster than case 1; case 3 will be faster still, but probably not enough to be noticeable.
I would suggest to rely on good old database normalization rules, in this case specifically 1NF. If that Description (same goes for the ProductName) is going to be repeated, you have a database design issue, and it being in SQLite or other has little to do with it. CL is right with his indexes, mind you, proper indexing will still matter.
Review your model, make a table for products and another for inventory.
I am trying to create a number of rows in a table based on the value of record in a different table.
i.e Table A has two columns NumberID, Number and TableB has MonthID, Month, Amount
When TableA.Number gets a value of say 5 I want TableB to be updated with 5 new rows. The values of TableB.Month and TableB.Amount will then be entered manually. Can this be done?
Depending upon what database you are using, this can be done using triggers.
Alternately, you could do this in the application layer.