in Postgres DB, found a simple select statement cost 1 days+ and seems will not finished. currently DB resource consumption is normal and I execute that SQL manually, but it completed in several seconds successfully.
and I executed SQL "SELECT relation::regclass AS relname, * FROM pg_locks where pid = '2066446'; The results found, there are 8 "AccessShareLock" types and 1 "ExclusiveLock" type.
I am not sure this could help on debug this long running SQLs. Could you please give me some directions on how to debug further for this issue?
add query plan here:
Related
On attempting to execute a simple query on a table (dimensions: 1,131,714,069 rows by 22 columns), I am running into the error:
[Amazon][JDBC](12080) Statement object has been closed.
Research online has unfortunately not provided much insight into this error.
I will not encounter this error each time I execute a query; so far it seems that its occurrence is unpredictable. The query that most recently caused this error looked was a very simple SELECT ... FROM ... WHERE with no subqueries and only one condition in the WHERE clause.
The query was busy for about 22 minutes before failing, however after waiting a few minutes, then running it again, it completed successfully in a matter of seconds. That being said, this kind of unpredictability and unreliability is exactly what I'm trying to prevent against.
If it helps, the IDE that I am using to connect to my Redshift database is TeamSQL.
What could be causing this error, and what steps could I take to prevent it?
I use an open source application (Maarch) and after a migration from older version to newer, I have a postgresql problem.
There is two actions, involved pgsql, that use 100% of the CPU and block the application.
I have no apache2 error, no pgsql error neither. I'm currently block for now, cause I don't know where I can search the source of the problem.
The server is under debian 8, with PHP 7.0 and POSTGRESQL 9.4
Any idea ?
Please consider:
enabling logging of all statements eg http://cheng.logdown.com/posts/2016/04/08/enable-logging-in-postgresql
Please note that the above mentioned will slow down the application and may eat a lot of hdd for logs. Please browse / grep the logs.
executing a few times select * from pg_stat_activity this will let you know what queries are running under the hood.
execute select * from pg_stat_user_tables in search for last_vacuum or autovac dates - if correlates to dates of your slowdowns you gov the issue solved.
execute SELECT now() - query_start, usename, datname, waiting, state, query FROM pg_stat_activity WHERE now() - query_start > '5 seconds'::interval ORDER BY 1 DESC; to determine long lasting queries (above 5 secs.
If the above mentioned will not help please analyze if queries that are being executed are complex and if fields in joins / where sections of the running queries are indexed.
alternatively running a vacuum full with analyze may help a bit (long lasting operation).
Hi i connected Hive using DB visualizer and fired a simple join query to fetch two columns according to the filter applied. But the query was running for more than an hour with the status "Executing". I fired the same query in Hive logging through Putty and got the result in less than 20 seconds.
Can anyone help me to understand why the query in DB visualizer was running for a long time without producing any output.
Query used:
SELECT
A.ORDER,
B.ORDER1
FROM
ORDER A
INNER JOIN DUORDER B ON A.ORDER=B.ORDER1 AND A.TYPE ='50'
(The result set contain only 400 records)
To analyze why, we need more info. Please please open Tools->Debug Window in DbVisualizer and enable debugging (just for DbVisualizer, not JDBC). Execute the query again, stopping it after some time (say a few minutes). Then submit a support request using Help->Contact Support, and make sure that Attach Logs is enabled. This will give us the info we need to see what may be wrong.
Best Regards,
Hans (DbVisualizer team)
When I execute a query for the first time in DBeaver it can take up to 10-15 seconds to display the result. In SQLDeveloper those queries only take a fraction of that time.
For example:
Simple "select column1 from table1" statement
DBeaver: 2006ms,
SQLDeveloper: 306ms
Example 2 (other way around; so theres no server-side caching):
Simple "select column1 from table2" statement
SQLDeveloper: 252ms,
DBeaver: 1933ms
DBeavers status box says:
Fetch resultset
Discover attribute column1
Find attribute column1
Late bind attribute colummn1
2, 3 and 4 use most of the query execution time.
I'm using oracle 11g, SQLDeveloper 4.1.1.19 and DBeaver 3.5.8.
See http://dbeaver.jkiss.org/forum/viewtopic.php?f=2&t=1870
What could be the cause?
DBeaver looks up some metadata related to objects in your query.
On an Oracle DB, it queries catalog tables such as
SYS.ALL_ALL_TABLES / SYS.ALL_OBJECTS - only once after connection, for the first query you execute
SYS.ALL_TAB_COLS / SYS.ALL_INDEXES / SYS.ALL_CONSTRAINTS / ... - I believe each time you query a table not used before.
Version 3.6.10 introduced an option to enable/disable a hint used in those queries. Disabling the hint made a huge difference for me. The option is in the Oracle Properties tab of the connection edit dialog. Have a look at issue 360 on dbeaver's github for more info.
The best way to get insight is to perfom the database trace
Perform few time the query to eliminate the caching effect.
Than repeat in both IDEs following steps
activate the trace
ALTER SESSION SET tracefile_identifier = test_IDE_xxxx;
alter session set events '10046 trace name context forever, level 12'; /* binds + waits */
Provide the xxxx to identify the test. You will see this string as a part of the trace file name.
Use level 12 to see the wait events and bind variables.
run the query
close the conenction
This is important to not trace other things.
Examine the two trace files to see:
what statements were performed
what number of rows was fetched
what time was elapsed in DB
for the rest of the time the client (IDE) is responsible
This should provide you enough evidence to claim if one IDE behaves different than other or if simple the DB statements issued are different.
When using vsql, I would like to see how long a query took to run once it completes. For example when i run:
select count(distinct key) from schema.table;
I would like to see an output like:
5678
(1 row)
total query time: 55 seconds.
If this is not possible, is there another way to measure query time?
In vsql type:
\timing
and then hit Enter. You'll like what you'll see :-)
Repeating that will turn it off.
Regarding the other part of your question:
is there another way to measure query time?
Vertica can log a history of all queries executed on the cluster which is another source of query time. Before 6.0 the relevant system table was QUERY_REPO, starting with 6.0 it is QUERY_REQUESTS.
Assuming you're on 6.0 or higher, QUERY_REQUESTS.REQUEST_DURATION_MS will give you the query duration in milliseconds.
Example of how you might use QUERY_REQUESTS:
select *
from query_requests
where request_type = 'QUERY'
and user_name = 'dbadmin'
and start_timestamp >= CURRENT_DATE
and request ilike 'select%from%schema.table%'
order by start_timestamp;
The QUERY_PROFILES.QUERY_DURATION_US and RESOURCE_ACQUISITIONS.DURATION_MS columns may also be of interest to you. Here are the short descriptions of those tables in case you're not already familiar:
RESOURCE_ACQUISITIONS - Retains information about resources (memory, open file handles, threads) acquired by each running request for each resource pool in the system.
QUERY_PROFILES - Provides information about queries that have run.
I'm not sure how to enable that in vsql or if that's possible. But you could get that information from a script.
Here's the psuedocode (I used to use perl):
print time
system("vsql -c 'select * from table'");
print time
Or put time into a variable and do some subtraction.
The other option is to use some tool like Toad to connect to Vertica instead of using vsql.