Syntax to express column names of VALUES-clause? - apache-spark-sql

Question

This is running,
val r = sql("SELECT T.* FROM ( VALUES ('xx','xxx','2019-01-01'), ('xxxx','yyyy','2019-01-02') ) T")
but r have "no name" columns, showed as col1| col2| col3. In standard SQL I can express names as table parameters, something as T(a,b,c) instead T... But this,
val r = sql("SELECT T.* FROM ( VALUES ('xx','xxx','2019-01-01'), ('xxxx','yyyy','2019-01-02') ) T(a,b,c)")
is not working, it is an ugly error message, say nothing about correct Spark syntax for it...
The question is "How to express column names?", and need an example that I can run in spark-shell v2.2.
Notes
The ugly message:
org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input '(' expecting {<EOF>, ',', 'WHERE', 'GROUP', 'ORDER', 'HAVING', 'LIMIT', 'JOIN', 'CROSS', 'INNER', 'LEFT', 'RIGHT', 'FULL', 'NATURAL', 'LATERAL', 'WINDOW', 'UNION', 'EXCEPT', 'MINUS', 'INTERSECT', 'SORT', 'CLUSTER', 'DISTRIBUTE', 'ANTI'}(line 1, pos 73)
== SQL ==
SELECT T.*, 'aaa' as chk FROM ( VALUES ('xx','xxx','2019-01-01') ) T (a,b,c)
----------------------------------------------------------------------^^^
at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:217)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:114)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:48)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:68)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:637)
... 50 elided

Answer 1

I suspect that the dataset (VALUES (),(),...) has a predefined column naming (col1,col2, ...) so the only way to override it, is to use aliases
Since you need columns a,b,c in the val r, it should look like this:
SELECT T.col1 as a, T.col2 as b, T.col3 as c FROM ( VALUES ('xx','xxx','2019-01-01'), ('xxxx','yyyy','2019-01-02') ) T

Answer 2

You can cast this to a dataframe and specify your column names:
val r = sql("SELECT T.* FROM ( VALUES ('xx','xxx','2019-01-01'), ('xxxx','yyyy','2019-01-02') ) T").toDF("foo","bar","baz")
Or you can use a a different approach to creating your dataframe:
val r = Seq(("xx","xxx","2019-01-01"),("xxxx","yyyy","2019-01-02")).toDF("foo","bar","baz")
Either way, you will get:
r.show
+----+----+----------+
| foo| bar| baz|
+----+----+----------+
| xx| xxx|2019-01-01|
|xxxx|yyyy|2019-01-02|
+----+----+----------+
Not necessarily any prettier/better than #mangusta's answer, just an alternative approach.