I use an open source application (Maarch) and after a migration from older version to newer, I have a postgresql problem.
There is two actions, involved pgsql, that use 100% of the CPU and block the application.
I have no apache2 error, no pgsql error neither. I'm currently block for now, cause I don't know where I can search the source of the problem.
The server is under debian 8, with PHP 7.0 and POSTGRESQL 9.4
Any idea ?
Please consider:
enabling logging of all statements eg http://cheng.logdown.com/posts/2016/04/08/enable-logging-in-postgresql
Please note that the above mentioned will slow down the application and may eat a lot of hdd for logs. Please browse / grep the logs.
executing a few times select * from pg_stat_activity this will let you know what queries are running under the hood.
execute select * from pg_stat_user_tables in search for last_vacuum or autovac dates - if correlates to dates of your slowdowns you gov the issue solved.
execute SELECT now() - query_start, usename, datname, waiting, state, query FROM pg_stat_activity WHERE now() - query_start > '5 seconds'::interval ORDER BY 1 DESC; to determine long lasting queries (above 5 secs.
If the above mentioned will not help please analyze if queries that are being executed are complex and if fields in joins / where sections of the running queries are indexed.
alternatively running a vacuum full with analyze may help a bit (long lasting operation).
Related
I have a sql file running many queries. I want to see the accumualted sum of all queries. I know that if I turn on timing, or call
\timing
query 1;
query 2;
query 3;
...
query n;
at the beginning of the script, it will start to show time it takes for each query to run. However, I need to have the accumulate results of all queries, without having to manually add them.
Is there a systematic way? If not, how can I fetch the interim times to throw them in a variable.
The pg_stat_statements is a good module that provides a means for tracking execution statistics.
First, add pg_stat_statements to shared_preload_libraries in the
postgresql.conf file. To know where this .conf file exists in your
filesystem, run show config_file;
shared_preload_libraries = 'pg_stat_statements'
Restart Postgres database
Create the extension
CREATE EXTENSION pg_stat_statements;
Now, the module provides a View, pg_stat_statements, which helps you to analyze various query execution metrics.
Reset the contents of stat collected before running queries.
SELECT pg_stat_statements_reset();
Now, execute your script file containing queries.
\i script_file.sql
You may get all the timing statistics of all the queries executed. To get the total time taken, simply run
select sum(total_time) from pg_stat_statements
where query !~* 'pg_stat_statements';
The time you get is in milliseconds, which may be converted to desired format using various timestamp related Postgres functions
If you want to time the whole script, on linux or mac you can use the time utility to launch the script.
The measurement in this case is a bit more than the sum of the raw query times, because it includes some overhead of starting and running the psql command. On my system this overhead is around 20ms.
$ time psql < script.sql
…
real 0m0.117s
user 0m0.008s
sys 0m0.007s
The real value is the time it took to execute the whole script, including the aforementioned overhead.
The approach in this answer is a crude, simple client side way to measure the runtime of the overall script. It is not useful to measure milli-second precision server side execution times. It still might be sufficient for many use-cases.
The solution of Kaushik Nayak is a way more precise method to time executions directly on the server. It also provides much more insight into the execution (eg. query level times).
I need to measure the execution time in a query in the ALTIBASE DBMS.
For example, I need to now in ms the time it took a SELECT * FROM example;
I tried to use the methods that have some traditional DBMS like MySQL or Oracle and doesn't work.
There are some method. The first one you can use the log from altibase to analyze. And the other method is to coding some program that recording time from using the SELECT to get result. Now, I also learn to use altibase - We can together.
I found the solution
After entering to the client with next command isql -u sys -p manager -sysdba you need to execute the next query
SET TIMING ON;
Next the following queries will have the execution time.
Hi i connected Hive using DB visualizer and fired a simple join query to fetch two columns according to the filter applied. But the query was running for more than an hour with the status "Executing". I fired the same query in Hive logging through Putty and got the result in less than 20 seconds.
Can anyone help me to understand why the query in DB visualizer was running for a long time without producing any output.
Query used:
SELECT
A.ORDER,
B.ORDER1
FROM
ORDER A
INNER JOIN DUORDER B ON A.ORDER=B.ORDER1 AND A.TYPE ='50'
(The result set contain only 400 records)
To analyze why, we need more info. Please please open Tools->Debug Window in DbVisualizer and enable debugging (just for DbVisualizer, not JDBC). Execute the query again, stopping it after some time (say a few minutes). Then submit a support request using Help->Contact Support, and make sure that Attach Logs is enabled. This will give us the info we need to see what may be wrong.
Best Regards,
Hans (DbVisualizer team)
When I execute a query for the first time in DBeaver it can take up to 10-15 seconds to display the result. In SQLDeveloper those queries only take a fraction of that time.
For example:
Simple "select column1 from table1" statement
DBeaver: 2006ms,
SQLDeveloper: 306ms
Example 2 (other way around; so theres no server-side caching):
Simple "select column1 from table2" statement
SQLDeveloper: 252ms,
DBeaver: 1933ms
DBeavers status box says:
Fetch resultset
Discover attribute column1
Find attribute column1
Late bind attribute colummn1
2, 3 and 4 use most of the query execution time.
I'm using oracle 11g, SQLDeveloper 4.1.1.19 and DBeaver 3.5.8.
See http://dbeaver.jkiss.org/forum/viewtopic.php?f=2&t=1870
What could be the cause?
DBeaver looks up some metadata related to objects in your query.
On an Oracle DB, it queries catalog tables such as
SYS.ALL_ALL_TABLES / SYS.ALL_OBJECTS - only once after connection, for the first query you execute
SYS.ALL_TAB_COLS / SYS.ALL_INDEXES / SYS.ALL_CONSTRAINTS / ... - I believe each time you query a table not used before.
Version 3.6.10 introduced an option to enable/disable a hint used in those queries. Disabling the hint made a huge difference for me. The option is in the Oracle Properties tab of the connection edit dialog. Have a look at issue 360 on dbeaver's github for more info.
The best way to get insight is to perfom the database trace
Perform few time the query to eliminate the caching effect.
Than repeat in both IDEs following steps
activate the trace
ALTER SESSION SET tracefile_identifier = test_IDE_xxxx;
alter session set events '10046 trace name context forever, level 12'; /* binds + waits */
Provide the xxxx to identify the test. You will see this string as a part of the trace file name.
Use level 12 to see the wait events and bind variables.
run the query
close the conenction
This is important to not trace other things.
Examine the two trace files to see:
what statements were performed
what number of rows was fetched
what time was elapsed in DB
for the rest of the time the client (IDE) is responsible
This should provide you enough evidence to claim if one IDE behaves different than other or if simple the DB statements issued are different.
When using vsql, I would like to see how long a query took to run once it completes. For example when i run:
select count(distinct key) from schema.table;
I would like to see an output like:
5678
(1 row)
total query time: 55 seconds.
If this is not possible, is there another way to measure query time?
In vsql type:
\timing
and then hit Enter. You'll like what you'll see :-)
Repeating that will turn it off.
Regarding the other part of your question:
is there another way to measure query time?
Vertica can log a history of all queries executed on the cluster which is another source of query time. Before 6.0 the relevant system table was QUERY_REPO, starting with 6.0 it is QUERY_REQUESTS.
Assuming you're on 6.0 or higher, QUERY_REQUESTS.REQUEST_DURATION_MS will give you the query duration in milliseconds.
Example of how you might use QUERY_REQUESTS:
select *
from query_requests
where request_type = 'QUERY'
and user_name = 'dbadmin'
and start_timestamp >= CURRENT_DATE
and request ilike 'select%from%schema.table%'
order by start_timestamp;
The QUERY_PROFILES.QUERY_DURATION_US and RESOURCE_ACQUISITIONS.DURATION_MS columns may also be of interest to you. Here are the short descriptions of those tables in case you're not already familiar:
RESOURCE_ACQUISITIONS - Retains information about resources (memory, open file handles, threads) acquired by each running request for each resource pool in the system.
QUERY_PROFILES - Provides information about queries that have run.
I'm not sure how to enable that in vsql or if that's possible. But you could get that information from a script.
Here's the psuedocode (I used to use perl):
print time
system("vsql -c 'select * from table'");
print time
Or put time into a variable and do some subtraction.
The other option is to use some tool like Toad to connect to Vertica instead of using vsql.