How to connect databricks SQL notebook to a parquet file via filepath - ssms

Question

I'm trying to connect to a list of parquet files that contain our data tables, I need to retrieve them to create a new table within a databricks notebook that will have the following fields:
Field Name
Data Type
Table Name
I just need to know the syntax for connecting to these parquet files via sql in a databricks notebook and any help with setting up these fields to display data as if pulling from information_schema in SSMS. Thanks.

Answer 1

The below syntax will help you to create table using given parquet file path:
%sql
CREATE TABLE <Table_Name>
USING parquet
OPTIONS (path "</path/to/Parquet>")
Change the <Table_Name> and </path/to/Parquet> with your values.
You can read the data by using SELECT statement:
SELECT * FROM <Table_Name>
Apache Spark also enables us to easily read and write Parquet files to Azure SQL Database.
df.write
.mode("overwrite")
.format("jdbc")
.option("url", f"jdbc:sqlserver://{servername}.database.windows.net;databaseName={databasename};")
.option("dbtable", "{tablename}")
.option("user", {localusername})
.option("password", {localpassword})
.option("driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver")
.option("customSchema", "sqlschemadetails")
.save()
We have to specify the JDBC connection string including SQL user name and password along with the schema name.
Useful link: https://www.c-sharpcorner.com/article/ingest-data-to-azure-sql-database-using-azure-databricks/
You can open Azure SQL Server database on SSMS simply by using servername, user and password as shown below.