Equivalent of MySQL's `show processlist` in SparkSQL - apache-spark-sql

Is there an equivalent of MySQL's command show processlist; in Apache SparkSQL?

I am providing the answer myself here (got some tip from my colleagues).
There may be no direct equivalent commands in SparkSQL, but one can use the WebUI for Spark to achieve similar effects.
c.f. Monitoring and Instrumentation (Official Doc of Apache Spark)

Related

Command Strategy Class Not found in Spark 1.3

I am using spark 1.3 and am able to create spark context . when i try to access a Cassandra DB using CassandraSQL context. I get the below error.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/sql/execution/SparkStrategies$CommandStrategy
at org.apache.spark.sql.cassandra.CassandraSQLContext$$anon$1.(CassandraSQLContext.scala:67)
at org.apache.spark.sql.cassandra.CassandraSQLContext.(CassandraSQLContext.scala:64)
at SparkSample$.main(SparkSample.scala:28)
at SparkSample.main(SparkSample.scala)
I use the spark-cassandra connector from Datastax. Also that I am not able to see any documentation related to SQL Execution
https://spark.apache.org/docs/1.2.0/api/java/index.html?org.apache.spark.sql.execution in spark 1.3 version. Any thoughts ?
First, if you look at the Cassandra readme you will see that they do not support 1.3 yet. I'm sure they would accept PR's though :)
Now, to the crux of the matter; they are using package private pieces so they are prone to these types of breaking changes. If you look at SparkStrategies in the 1.2 branch, you will see the CommandStrategy at the bottom. However, in SparkStrategies 1.3, the last object has become DDLStrategy, which does not even look the same at a glance. So, they might have removed this altogether. Your best bet is to report this to the Cassandra connector project and wait for official support of 1.3

Is it possible to write a hibernate dialect for Apache Phoenix?

i would like to write an hibernate dialect for Apache Phoenix. The huge problem is that the keyword of an insert statement is "Upsert" instead of "Insert". This keyword is declared at org.hibernate.sql.Insert. Is there a possibility to write an dialect for Apache Phoenix ?
Source http://phoenix.apache.org/language/index.html#upsert_values
Kind regards
I found a theoretical solution. The easiest way is to override the jdbc created by apache phoenix. You have to override exceuteQuery() and updateQuery() functions.There you have to replace "insert" and "update" with "upsert".
What do you think? I didn't test it yet.
this seems to work pretty well if you are ok with using the "fat" phoenix client: https://github.com/jruesga/phoenix-hibernate-dialect

Equivalent to dbms_output in Soapui JDBC step?

I got a question nobody seems to care about because I have not found anything on the the web.
I am using the free version of SOAPUI. I use some JDBC steps (Oracle driver) in my testCases.
In my SQL code, it would be very usefull for me to output some traces for debbuging.
But the dbms_output.put_lines does not seem to trace anything. I look at all the 'log' windows in soapUI, but I found nothing.
Can you help me ?
Do I look at the wrong window ?
Is there a special command-line to use in my SQL code to print my output ?
Thankx in advance

Oracle: Is there a way to encode a value using the unix crypt method?

Is there a way to encode a value using the unix crypt method in oracle 10g?
(Other DBMS like MySQL support this)
The DBMS_CRYPTO PL/SQL package has encryption, but none using the native weakness of the Unix crypt(1). It's much more analogous to crypt(3).
If you need the exact crypt(1) implementation, you're going to have to code it yourself. Here's the C code for it.
Provided you are one Oracle8i or higher this is possible.
The solution is to write a Java Stored Procedure which can execute OS commands, and use that to run Unix crypt. Tim Hall's Oracle-Base site has an example of how to code one of these things. Find it here.

Is there a good alternative to SQL*PLUS for Oracle?

I am not a fan of using SQL*PLUS as an interface to Oracle. I usually use yasql, but it hasn't been updated since 2005 and can do with some improvements. A quick Google search shows yasql and SQLPal. I am using linux, so SQLPal is not an option.
Are there any alternatives out there, or am I stuck with an interface that I do not like or one that is no longer maintained?
I presume that you want a low-overhead method of knocking out queries, but want more functions than SQL*Plus provides? Why not use Oracle's SQL Developer? It's free.
Install, make a new connection to your database, then just start typing a script. Press F5 to run it (or just the part of the script that you've highlighted).
Take a look at gqlplus. It wraps sql*plus on linux and makes it more user-friendly by adding things like command history, table name completion and so on.
Emacs can provide so much more powerful text editing features and functionality beyond the default SQL*Plus command-line interface.
Here are a few links on how to use Emacs as a wrapper for SQL*Plus:
Emacs and Oracle
EmacsWiki:SqlPlus
TOAD is pretty expensive, but you can download a 90-day trial from the Quest site to see if it's got the feature set you want (don't be fooled by the "freeware" title - it's only free for 90 days, and then it expires, which definitely makes it shareware):
http://www.toadworld.com/Freeware/ToadforOracleFreeware/tabid/558/Default.aspx
Another options is a tool I've seen on CodeProject:
http://www.codeproject.com/KB/database/OQuery.aspx
It's in .NET, so you'd have to see if it compiled on Mono, but it might be worth a shot. I haven't used either tool (Toad or this one), since I'm a SQL Server guy, but I've heard good things about Toad.
If it's command-line you want, I'd recommend rlwrap to go with sqlplus; it gives you line-editing capabilities and command history, making sqlplus a somewhat usable tool.
You could try PL/SQL developer from allroundautomations, there is a trial available and the price is much lower than TOAD.
Regards
K
Take a look at Senora. This tool is written in Perl and therefore is cross platform. Also Senora is free, extensible and intends to be your primary Oracle shell. You can extend Senora easily by providing you own plugins. Senora attempts to provide a friendlier output formatting than sqlplus. Columns tend to be only as wide a really needed.
Another interesting alternative is SQLcl. It provides in-line editing, statement completion, command recall, DBA stuff (e.g. startup, shutdown) and also supporting your previously written SQL*Plus scripts.
It depends what you are looking for.
If it is a GUI query tool, then Oracle have their free SQL Developer product (though it has a hefty footprint). There's a few free cross-database ones too. I like SQUirrel SQL client myself. There's also DBVisualiser and a few others.
JEdit is an editor that has a DBConsole plugin for running database queries and DML/DDL.
They are all java based so run most places.
If you like a command line, check out sqlpython (the developer has identified a couple of others too)
I like SQL Developer. It's free, has an intuitive UI, and runs on Windows, Mac, and Linux. It also supports many sql*plus commands and supports version control
Apparently Oracle itself has phased out sql*plus and replaced it with SQLcl, which supports more modern features such as history, formatting, etc: https://www.oracle.com/database/technologies/appdev/sqlcl.html.
open source version of TOAD is TORA:
tora.sourceforge.net
If you're the VIM type kind of guy then I'd look into Vorax. It is basically a VIM wrapper around SQL*plus.
Have used both Toad & SQL Navigator, and I love the stability SQL Navigator has.
i like sqlsh
alias sqr='sqlsh -d DBI:Oracle:MYSERVER.COM -u USER -p PASSWORD'
toad from quest software if you can pay for a license
sql squirrel if you can't.
I used my own tool ocimlsh in conjunction with rlwrap.
I just use socat to add readline support to sqlplus. History and a working backspace key actually turn sqlplus into a pretty decent tool.
In my .bashrc:
function sqlplus {
socat READLINE,history=$HOME/.sqlplus_history EXEC:"$ORACLE_HOME/bin/sqlplus $(echo $# | sed 's/\([\:]\)/\\\1/g')",pty,setsid,ctty
status=$?
}
You might see alternatives that alias sqlplus to socat, but you will quickly discover that doing so prevents you from invoking sqlplus with its various command line options.
CAVEAT: Be sure to set $HOME/.sqlplus_history permissions to 0600. Passwords that you type end up in the history file. You might also consider adding cat /dev/null > $HOME/.sqlplus_history to your .bash_logout.