I want to create a temp table so as to be able to join it to a few tables because joining those tables with the content of the proposed temporary table takes a lot of time (fetching the content of the temporary table is time consuming.Repeating it over and over takes more and more time). I am dropping the temporary table when my needs are accomplished.
I want to know if these temporary tables would be visible over other client session(my requirement is to make them visible only for current client session). I am using postgresql. It would be great if you could suggest better alternatives to the solution I am thinking of.
PostgreSQL then is the database for you. Temporary tables done better than the standard. From the docs,
Although the syntax of CREATE TEMPORARY TABLE resembles that of the SQL standard, the effect is not the same. In the standard, temporary tables are defined just once and automatically exist (starting with empty contents) in every session that needs them. PostgreSQL instead requires each session to issue its own CREATE TEMPORARY TABLE command for each temporary table to be used. This allows different sessions to use the same temporary table name for different purposes, whereas the standard's approach constrains all instances of a given temporary table name to have the same table structure.
Pleas read the documentation.
Temporary tables are only visible in the current session and are automatically dropped when the database session ends.
If you specify ON COMMIT, the temporary table will automatically be dropped at the end of the current transaction.
If you need good table statistics on a temporary table, you have to call ANALYZE explicitly, as these statistics are not collected automatically.
By default , temporary tables are visible for current session only and temporary tables are dropped automatically after commit or closing that transaction, so you don't need to drop explicitly.
The auto vacuum daemon cannot access and therefore cannot vacuum or analyze temporary tables. For this reason, appropriate vacuum and analyze operations should be performed via session SQL commands. For example, if a temporary table is going to be used in complex queries, it is wise to run ANALYZE on the temporary table after it is populated.
Optionally, GLOBAL or LOCAL can be written before TEMPORARY or TEMP. This presently makes no difference in PostgreSQL and is deprecated
Related
I am creating a large intermediate data set (millions of records, half hour runtime) that is used as parts of several other different queries. Ideally this would be stored as a materialized view or cached table but I don't have DDL access on the server.
Subqueries and CTE's are inefficent because they must be re-executed every time and can't be indexed. Local/global temporary tables work well but get lost when I disconnect.
Is there any way to create a more persistent temporary table (or equivalent) without DDL access?
Is there a way in Oracle to create a table that only exists while the database is running and is only stored in memory? So if the database is restarted I will have to recreate the table?
Edit:
I want the data to persist across sessions. The reason being that the data is expensive to recreate but is also highly sensitive.
Using a temporary table would probably help performance compared to what happens today, but its still not a great solution.
You can create a 100% ephemeral table that is usable for the duration of a session (typically shorter than the duration than the database run time) called a TEMPORARY table. The entire purpose of a table in memory is to make it faster for reading from. You will have to re-populate the table for each session as the table will be forgotten (both structure and data) once the session completes.
No exactly, no.
Oracle has the concept of a "global temporary table". With a global temporary table, you create the table once, as with any other table. The table definition will persist permanently, as with any other table.
The contents of the table, however, will will not be permanent. Depending on how you define it, the contents will persist for either the life of the session (on commit perserve rows) or the life of the transaction (on commit delete rows).
See the documentation for all the details:
http://docs.oracle.com/cd/E11882_01/server.112/e25494/tables003.htm#ADMIN11633
Hope that helps.
You can use Oracle's trigger mechanism to invoke a stored procedure when the database starts up or shuts down.
That way you could have the startup trigger create the table, and the shutdown trigger drop it.
You'd probably also want the startup trigger to handle cases where the table exists and truncate it just in case the server stopped suddenly and the shutdown trigger wasn't called.
Oracle trigger documentation
Using Oracle's Global Temporary Tables, you can create a table in memory and have it delete the data at the end of the transaction, or the end of the session.
If I understand correctly, you have some data that needs to be processed when the database is brought online and left available only as long as the database is online. The only use-case I can think of that would require this is if you're encrypting some data and you want to ensure that the unencrypted data is never written to disk.
If this is actually your use-case, I would recommend forgetting about trying to create your own solution for this and, instead, make use of Oracle's encrypted tablespaces or Transparent Data Encryption.
Does the time it takes to drop a table in SQL reflect the quantity of data within the table?
Lets say for dropping an identical table one with 100000 rows and 1000 rows.
Is this MySQL? The ext3 Linux filesystem is known to be slow to drop large tables. The time to drop will have to do more with the size of the table, more than the rows in the table.
http://www.mysqlperformanceblog.com/2009/06/16/slow-drop-table/
It will definitely depend on the dbserver and on the specific storage params in that db server and upon the existence of LOBs in the table. For MOST scenarios, it's very fast.
In Oracle, dropping a table is very quick and unlikely to matter compared to other operations you would be performing (creating the table and populating it).
It would not be idomatic to be creating and dropping tables enough in Oracle for this to be any sort of a performance factor. You would instead consider using global temporary tables.
From http://download.oracle.com/docs/cd/B28359_01/server.111/b28318/schema.htm
In addition to permanent tables,
Oracle Database can create temporary
tables to hold session-private data
that exists only for the duration of a
transaction or session.
The CREATE GLOBAL TEMPORARY TABLE
statement creates a temporary table
that can be transaction-specific or
session-specific. For
transaction-specific temporary tables,
data exists for the duration of the
transaction. For session-specific
temporary tables, data exists for the
duration of the session. Data in a
temporary table is private to the
session. Each session can only see and
modify its own data. DML locks are not
acquired on the data of the temporary
tables. The LOCK statement has no
effect on a temporary table, because
each session has its own private data.
I read about temporary tables, global temporary tables and table variables. I understood it but could not imagine a condition when I have to use this. Please elaborate on when I should use the temporary table.
Most common scenario for using temporary tables is from within a stored procedure.
If there is logic inside a stored procedure which involves manipulation of data that cannot be done within a single query, then in such cases, the output of one query / intermediate results can be stored in a temporary table which then participates in further manipulation via joins etc to achieve the final result.
One common scenario in using temporary tables is to store the results of a SELECT INTO statement
The table variable is relatively new (introduced in SQL Server 2005 - as far as i can remember ) can be used instead of the temp table in most cases. Some differences between the two are discussed here
In a lot of cases, especially in OLTP applications, usage of temporary tables within your procedures means that you MAY possibly have business processing logic in your database and might be a consideration for you to re-look your design - especially in case of n tier systems having a separate business layer in their application.
The main difference between the three is a matter of lifetime and scope.
By a global table, I am assuming you mean a standard, run of the mill, table. Tables are used for storing persistent data. They are accessible to all logged in users. Any changes you make are visible to other users and vice versa.
A temporary table exist solely for storing data within a session. The best time to use temporary tables are when you need to store information within SQL server for use over a number of SQL transactions. Like a normal table, you'll create it, interact with it (insert/update/delete) and when you are done, you'll drop it. There are two differences between a table and a temporary table.
The temporary table is only visible to you. Even if someone else creates a temporary table with the same name, no one else will be able to see or affect your temporary table.
The temporary table exists for as long as you are logged in, unless you explicitly drop it. If you log out or are disconnected SQL Server will automatically clean it up for you. This also means the data is not persistent. If you create a temporary table in one session and log out, it will not be there when you log back in.
A table variable works like any variable within SQL Server. This is used for storing data for use in a single transaction. This is a relatively new feature of TSQL and is generally used for passing data between procedures - like passing an array. There are three differences between a table and a table variable.
Like a temporary table, it is only visible to you.
Because it is a variable, it can be passed around between stored procedures.
The temporary table only exists within the current transaction. Once SQL Server finishes a transaction (with the GO or END TRANSACTION statements) or it goes out of scope, it will be deallocated.
I personally avoid using temporary tables and table variables, for a few reasons. First, the syntax for them is Microsoft specific. If your program is going to interact with more than one RDBMS, don't use them. Also, temporary tables and table variables have a tendency to increase the complexity of some SQL queries. If your code can be accomplished using a simpler method, I'd recommend going with simple.
I am trying to understand the tempDB and following are the doubts popping in my mind.
What is the lifetime of data in tempDB? Say a query is doing some Order By and uses tempDB for performing that. After this query finishes, someone else also executes a query which utilizes the tempDB. Will the second query find records written by first query in the tempDB or will they be deleted?
Are there any visible tables created inside the tempDB by the Sql Engine? How can I know which temporary table is created because of this query? Is there any naming convention followed by the Sql engine for naming these temporary tables?
I am new to tempDB so please pardon me for asking such silly (if at all) questions :-)
It will be very nice if someone can point me to a good resource which can help me learn about tempDB.
Temp table is stored in tempdb until the connection is dropped (or in the case of a global temp tables when the last connection using it is dropped). You can also (and it is a good practice to do so) manually drop the table when you are finished using it with a drop table statement.
No, others cannot see your temp tables if they are local temp tables (They can see and use global temp tables) Multiple people can run commands which use the same temp table name but they will not be overlapping in a local temp table and so you can have a table named #test and so can 10,000 other users, but each one has its own structure and data.
You don't want to generally look up temp tables in tempdb. It is possible to check for existence, but that is the only time I have ever referenced tempdb directly. Simply use your temp table name. Example below of checking for existence
IF OBJECT_ID('TempDB.dbo.#DuplicateAssignments') IS NOT NULL
BEGIN
DROP TABLE #DuplicateAssignments
END
You name temp tables by prefacing the name with # (for local tables the ones you would use 999.9% of the time) and ## for global temp tables, then the rest of the name you want.
There's a few MSDN articles that are probably the best source of information on the tempDB database in SQL Server.
tempdb Database
The tempdb system database is a global
resource that is available to all
users connected to the instance of SQL
Server and is used to hold the
following:
Temporary user objects that are explicitly created, such as: global or
local temporary tables, temporary
stored procedures, table variables, or
cursors.
Internal objects that are created by the SQL Server Database Engine, for
example, work tables to store
intermediate results for spools or
sorting.
Row versions that are generated by data modification transactions in a
database that uses read-committed
using row versioning isolation or
snapshot isolation transactions.
Row versions that are generated by data modification transactions for
features, such as: online index
operations, Multiple Active Result
Sets (MARS), and AFTER triggers.
Operations within tempdb are minimally
logged. This enables transactions to
be rolled back. tempdb is re-created
every time SQL Server is started so
that the system always starts with a
clean copy of the database. Temporary
tables and stored procedures are
dropped automatically on disconnect,
and no connections are active when the
system is shut down. Therefore, there
is never anything in tempdb to be
saved from one session of SQL Server
to another. Backup and restore
operations are not allowed on tempdb.
There's also tempdb and Index Creation, this blog post along with Working with tempdb in SQL Server 2005 which states:
The SQL Server system database, tempdb, has undergone a number of changes in SQL Server 2005. There are new tempdb usages and internal optimizations in SQL Server 2005; tempdb architecture is mostly unchanged since SQL Server 2000.
The tempdb system database is very similar to a user database. The main difference is that data in tempdb does not persist after SQL Server shuts down.
The temporary tables created in TempDB are dropped when the query is completed.
I'm not sure on this (I would have to try it), but I think theoretically ALL tables created in TempDB are visible, although only the user that created the table has permission to access it.