How to get a list of column names on Sqlite3 database? - sql

I want to migrate my iPhone app to a new database version. Since I don't have some version saved, I need to check if certain column names exist.
This Stackoverflow entry suggests doing the select
SELECT sql FROM sqlite_master
WHERE tbl_name = 'table_name' AND type = 'table'
and parse the result.
Is that the common way? Alternatives?

PRAGMA table_info(table_name);
will get you a list of all the column names.

If you do
.headers ON
you will get the desired result.

If you have the sqlite database, use the sqlite3 command line program and these commands:
To list all the tables in the database:
.tables
To show the schema for a given tablename:
.schema tablename

Just for super noobs like me wondering how or what people meant by
PRAGMA table_info('table_name')
You want to use use that as your prepare statement as shown below. Doing so selects a table that looks like this except is populated with values pertaining to your table.
cid name type notnull dflt_value pk
---------- ---------- ---------- ---------- ---------- ----------
0 id integer 99 1
1 name 0 0
Where id and name are the actual names of your columns. So to get that value you need to select column name by using:
//returns the name
sqlite3_column_text(stmt, 1);
//returns the type
sqlite3_column_text(stmt, 2);
Which will return the current row's column's name. To grab them all or find the one you want you need to iterate through all the rows. Simplest way to do so would be in the manner below.
//where rc is an int variable if wondering :/
rc = sqlite3_prepare_v2(dbPointer, "pragma table_info ('your table name goes here')", -1, &stmt, NULL);
if (rc==SQLITE_OK)
{
//will continue to go down the rows (columns in your table) till there are no more
while(sqlite3_step(stmt) == SQLITE_ROW)
{
sprintf(colName, "%s", sqlite3_column_text(stmt, 1));
//do something with colName because it contains the column's name
}
}

If you want the output of your queries to include columns names and be correctly aligned as columns, use these commands in sqlite3:
.headers on
.mode column
You will get output like:
sqlite> .headers on
sqlite> .mode column
sqlite> select * from mytable;
id foo bar
---------- ---------- ----------
1 val1 val2
2 val3 val4

An alternative way to get a list of column names not mentioned here that is cross platform and does not rely on the sqlite3.exe shell is to select from the PRAGMA_TABLE_INFO() table value function.
SELECT name FROM PRAGMA_TABLE_INFO('your_table');
name
tbl_name
rootpage
sql
You can check if a certain column exists by querying:
SELECT 1 FROM PRAGMA_TABLE_INFO('your_table') WHERE name='column1';
1
This is what you use if you don't want to parse the result of select sql from sqlite_master or pragma table_info.
Note this feature is experimental and was added in SQLite version 3.16.0 (2017-01-02).
Reference:
https://www.sqlite.org/pragma.html#pragfunc

To get a list of columns you can simply use:
.schema tablename

I know it is an old thread, but recently I needed the same and found a neat way:
SELECT c.name FROM pragma_table_info('your_table_name') c;

When you run the sqlite3 cli, typing in:
sqlite3 -header
will also give the desired result

.schema table_name
This will list down the column names of the table from the database.
Hope this will help!!!

you can use Like statement if you are searching for any particular column
ex:
SELECT * FROM sqlite_master where sql like('%LAST%')

This command below sets column names:
.header on
Then, this is how it looks like below:
sqlite> select * from user;
id|first_name|last_name|age
1|Steve|Jobs|56
2|Bill|Gates|66
3|Mark|Zuckerberg|38
And this command below unsets column names:
.header off
Then, this is how it looks like below:
sqlite> select * from user;
1|Steve|Jobs|56
2|Bill|Gates|66
3|Mark|Zuckerberg|38
And these commands show the details of the command ".header":
.help .header
Or:
.help header
Then, this is how it looks like below:
sqlite> .help .header
.headers on|off Turn display of headers on or off
In addition, this command below sets the output mode "box":
.mode box
Then, this is how it looks like below:
sqlite> select * from user;
┌────┬────────────┬────────────┬─────┐
│ id │ first_name │ last_name │ age │
├────┼────────────┼────────────┼─────┤
│ 1 │ Steve │ Jobs │ 56 │
│ 2 │ Bill │ Gates │ 66 │
│ 3 │ Mark │ Zuckerberg │ 38 │
└────┴────────────┴────────────┴─────┘
And, this command below sets the output mode "table":
.mode table
Then, this is how it looks like below:
sqlite> select * from user;
+----+------------+------------+-----+
| id | first_name | last_name | age |
+----+------------+------------+-----+
| 1 | Steve | Jobs | 56 |
| 2 | Bill | Gates | 66 |
| 3 | Mark | Zuckerberg | 38 |
+----+------------+------------+-----+
And these commands show the details of the command ".mode":
.help .mode
Or:
.help mode
Then, this is how it looks like below:
sqlite> .help .mode
.import FILE TABLE Import data from FILE into TABLE
Options:
--ascii Use \037 and \036 as column and row separators
--csv Use , and \n as column and row separators
--skip N Skip the first N rows of input
--schema S Target table to be S.TABLE
-v "Verbose" - increase auxiliary output
Notes:
* If TABLE does not exist, it is created. The first row of input
determines the column names.
* If neither --csv or --ascii are used, the input mode is derived
from the ".mode" output mode
* If FILE begins with "|" then it is a command that generates the
input text.
.mode MODE ?OPTIONS? Set output mode
MODE is one of:
ascii Columns/rows delimited by 0x1F and 0x1E
box Tables using unicode box-drawing characters
csv Comma-separated values
column Output in columns. (See .width)
html HTML <table> code
insert SQL insert statements for TABLE
json Results in a JSON array
line One value per line
list Values delimited by "|"
markdown Markdown table format
qbox Shorthand for "box --width 60 --quote"
quote Escape answers as for SQL
table ASCII-art table
tabs Tab-separated values
tcl TCL list elements
OPTIONS: (for columnar modes or insert mode):
--wrap N Wrap output lines to no longer than N characters
--wordwrap B Wrap or not at word boundaries per B (on/off)
--ww Shorthand for "--wordwrap 1"
--quote Quote output text as SQL literals
--noquote Do not quote output text
TABLE The name of SQL table used for "insert" mode

In order to get the column information you can use the following snippet:
String sql = "select * from "+oTablename+" LIMIT 0";
Statement statement = connection.createStatement();
ResultSet rs = statement.executeQuery(sql);
ResultSetMetaData mrs = rs.getMetaData();
for(int i = 1; i <= mrs.getColumnCount(); i++)
{
Object row[] = new Object[3];
row[0] = mrs.getColumnLabel(i);
row[1] = mrs.getColumnTypeName(i);
row[2] = mrs.getPrecision(i);
}

//JUST little bit modified the answer of giuseppe which returns array of table columns
+(NSMutableArray*)tableInfo:(NSString *)table{
sqlite3_stmt *sqlStatement;
NSMutableArray *result = [NSMutableArray array];
const char *sql = [[NSString stringWithFormat:#"PRAGMA table_info('%#')",table] UTF8String];
if(sqlite3_prepare(md.database, sql, -1, &sqlStatement, NULL) != SQLITE_OK)
{
NSLog(#"Problem with prepare statement tableInfo %#",
[NSString stringWithUTF8String:(const char *)sqlite3_errmsg(md.database)]);
}
while (sqlite3_step(sqlStatement)==SQLITE_ROW)
{
[result addObject:
[NSString stringWithUTF8String:(char*)sqlite3_column_text(sqlStatement, 1)]];
}
return result;
}

.schema
in sqlite console when you have you're inside the table
it looks something like this for me ...
sqlite>.schema
CREATE TABLE players(
id integer primary key,
Name varchar(255),
Number INT,
Team varchar(255)

This is an old question, but here is an alternative answer that retrieves all the columns in the SQLite database, with the name of the associated table for each column :
WITH tables AS (SELECT name tableName, sql
FROM sqlite_master WHERE type = 'table' AND tableName NOT LIKE 'sqlite_%')
SELECT fields.name, fields.type, tableName
FROM tables CROSS JOIN pragma_table_info(tables.tableName) fields
This returns this type of result:
{
"name": "id",
"type": "integer",
"tableName": "examples"
}, {
"name": "content",
"type": "text",
"tableName": "examples"
}
For a simple table containing an identifier and a string content.

function getDetails(){
var data = [];
dBase.executeSql("PRAGMA table_info('table_name') ", [], function(rsp){
if(rsp.rows.length > 0){
for(var i=0; i<rsp.rows.length; i++){
var o = {
name: rsp.rows.item(i).name,
type: rsp.rows.item(i).type
}
data.push(o);
}
}
alert(rsp.rows.item(0).name);
},function(error){
alert(JSON.stringify(error));
});
}

-(NSMutableDictionary*)tableInfo:(NSString *)table
{
sqlite3_stmt *sqlStatement;
NSMutableDictionary *result = [[NSMutableDictionary alloc] init];
const char *sql = [[NSString stringWithFormat:#"pragma table_info('%s')",[table UTF8String]] UTF8String];
if(sqlite3_prepare(db, sql, -1, &sqlStatement, NULL) != SQLITE_OK)
{
NSLog(#"Problem with prepare statement tableInfo %#",[NSString stringWithUTF8String:(const char *)sqlite3_errmsg(db)]);
}
while (sqlite3_step(sqlStatement)==SQLITE_ROW)
{
[result setObject:#"" forKey:[NSString stringWithUTF8String:(char*)sqlite3_column_text(sqlStatement, 1)]];
}
return result;
}

I know it's too late but this will help other.
To find the column name of the table, you should execute select * from tbl_name and you will get the result in sqlite3_stmt *. and check the column iterate over the total fetched column. Please refer following code for the same.
// sqlite3_stmt *statement ;
int totalColumn = sqlite3_column_count(statement);
for (int iterator = 0; iterator<totalColumn; iterator++) {
NSLog(#"%s", sqlite3_column_name(statement, iterator));
}
This will print all the column names of the result set.

I was able to retrieve table names with corresponding columns by using one sql query, but columns output is comma separated. I hope it helps somebody
SELECT tbl_name, (SELECT GROUP_CONCAT(name, ',') FROM PRAGMA_TABLE_INFO(tbl_name)) as columns FROM sqlite_schema WHERE type = 'table';

Get a list of tables and columns as a view:
CREATE VIEW Table_Columns AS
SELECT m.tbl_name AS TableView_Name, m.type AS TableView, cid+1 AS Column, p.*
FROM sqlite_master m, Pragma_Table_Info(m.tbl_name) p
WHERE m.type IN ('table', 'view') AND
( m.tbl_name = 'mypeople' OR m.tbl_name LIKE 'US_%') -- filter tables
ORDER BY m.tbl_name;

//Called when application is started. It works on Droidscript, it is tested
function OnStart()
{
//Create a layout with objects vertically centered.
lay = app.CreateLayout( "linear", "VCenter,FillXY" );
//Create a text label and add it to layout.
txt = app.CreateText( "", 0.9, 0.4, "multiline" )
lay.AddChild( txt );
app.AddLayout(lay);
db = app.OpenDatabase( "MyData" )
//Create a table (if it does not exist already).
db.ExecuteSql( "drop table if exists test_table" )
db.ExecuteSql( "CREATE TABLE IF NOT EXISTS test_table " +
"(id integer primary key, data text, num integer)",[],null, OnError )
db.ExecuteSql( "insert into test_table values (1,'data10',100),
(2,'data20',200),(3,'data30',300)")
//Get all the table rows.
DisplayAllRows("SELECT * FROM test_table");
DisplayAllRows("select *, id+100 as idplus, 'hahaha' as blabla from
test_table order by id desc;")
}
//function to display all records
function DisplayAllRows(sqlstring) // <-- can you use for any table not need to
// know column names, just use a *
// example:
{
//Use all rows what is in ExecuteSql (try any, it will works fine)
db.ExecuteSql( sqlstring, [], OnResult, OnError )
}
//Callback to show query results in debug.
function OnResult( res )
{
var len = res.rows.length;
var s = txt.GetText();
// ***********************************************************************
// This is the answer how to read column names from table:
for(var ColumnNames in res.rows.item(0)) s += " [ "+ ColumnNames +" ] "; // "[" & "]" optional, i use only in this demo
// ***********************************************************************
//app.Alert("Here is all Column names what Select from your table:\n"+s);
s+="\n";
for(var i = 0; i < len; i++ )
{
var rows = res.rows.item(i)
for (var item in rows)
{
s += " " + rows[item] + " ";
}
s+="\n\n";
}
//app.Alert(s);
txt.SetText( s )
}
//Callback to show errors.
function OnError( msg )
{
app.Alert( "Error: " + msg )
}

If you're using the SQLite3, INFORMATION_SCHEMA is not supported. Use PRAGMA table_info instead. This will return 6 rows of information about the table. To fetch the column name (row2), use a for loop like the following
cur.execute("PRAGMA table_info(table_name)") # fetches the 6 rows of data
records = cur.fetchall()
print(records)
for row in records:
print("Columns: ", row[1])

For use in Python with sqlite3
Top answer PRAGMA table_info() returns a list of tuples, which might not be suitable for further processing, e.g.:
[(0, 'id', 'INTEGER', 0, None, 0),
(1, 'name', 'TEXT', 0, None, 0),
(2, 'age', 'INTEGER', 0, None, 0),
(3, 'profession', 'TEXT', 0, None, 0)]
When using sqlite3 in Python, simply add a list comprehension in the end to filter out unwanted information.
import sqlite3 as sq
def col_names(t_name):
with sq.connect('file:{}.sqlite?mode=ro'.format(t_name),uri=True) as conn:
cursor = conn.cursor()
cursor.execute("PRAGMA table_info({}) ".format(t_name))
data = cursor.fetchall()
return [i[1] for i in data]
col_names("your_table_name")
Result
["id","name","age","profession"]
DISCLAIMER: Do not use in production as this snippet is subject to possible SQL injection!

Related

QlikView Convert 1753-01-01 00:00:00.000 to NULL

I am trying to convert data from SQL as 1753-01-01 00:00:00.000 to be shown as NULL values in QlikView.
I do the following in the QlikView Load statements -
SET NullTimeStamp = if ($1 = '1753-01-01 00:00:00', null(), $1);
Then use it when LOAD:
LOAD
$(NullTimeStamp(YourDateField1)) AS YOURDATEFIELD1,
$(NullTimeStamp(YourDateField2)) AS YOURDATEFIELD2,
$(NullTimeStamp(YourDateField3)) AS YOURDATEFIELD3
However, I have many fields with Time and Dates in my tables so I was wondering if there is a more elegant way of solving this issue?
Ive done something similar in the past. The idea is to generate part of the load script in a variable and then use this variable as part of the next load script
DummyData:
Load * Inline [
Something1 , Something2, Something3, Something4, Something5
1753-01-01 00:00:00.000, 2, 3, 4, 1753-01-01 00:00:00.000
];
SET NullTimeStamp = if ($1 = '1753-01-01 00:00:00', null(), $1);
// Define a temp table
// that holds list of fields that have to be checked with NullTimeStamp
Fields:
Load * Inline [
FieldNames
Something1
Something2
Something3
Something4
Something5
];
let FieldsConcatenation = '';
// loop through the NullTimeStamp-ed fields
for a = 1 to NoOfRows('Fields')
let f = FieldValue('FieldNames', a);
// concatenate each iteration to form part of the RealLoad table script
let FieldsConcatenation = '$(FieldsConcatenation)' & '$(NullTimeStamp(' & '$(f)' & ')) as ' & Upper('$(f)') & ',' & chr(13);
next
// remove the last comma
let FieldsConcatenation = left('$(FieldsConcatenation)', Index('$(FieldsConcatenation)', ',' , -1) -1);
// we dont need this anymore
Drop Table Fields;
// add FieldsConcatenation variable as part of the load script
RealLoad:
Load
$(FieldsConcatenation),
'a' as LoadTheRestHere
Resident
DummyData;
// we dont need this anymore
Drop Table DummyData;
FieldsConcatenation variable will have the following content:
The original table:
And the final table:

gocql - using Cassandra 3 timeuuid functions

In cqlsh I can execute something like:
SELECT id FROM names WHERE name = 'dave' AND id > maxTimeuuid('2015-04-05 00:05') AND id < minTimeuuid('2019-01-01 00:05');
... this returns a list as expected.
id
--------------------------------------
632f4960-375b-11e8-90b1-02420a000203
91f03523-3761-11e8-a777-02420a000a05
32dbffc4-3762-11e8-a778-02420a000a05
9ffb1ab8-3762-11e8-a779-02420a000a05
c28a0372-3764-11e8-a77b-02420a000a05
However I am struggling to get this to work in gocql:
var found bool = false
m := map[string]interface{}{}
qString := "SELECT id FROM names WHERE name=? AND id > maxTimeuuid(?) AND id < minTimeuuid(?);"
query := Cassandra.Sessionds.Query(qString, name, toFrom.From, toFrom.To).Consistency(gocql.One)
fmt.Println("sending IsRunning query: " + query.String())
iterable := query.Iter()
for iterable.MapScan(m) {
found = true
lookupTable = HostLookup {
Id: m["id"].(gocql.UUID),
}
}
Even though the debug line looks ok,
sending IsRunning query: [query statement="SELECT id FROM names WHERE name=? AND id > maxTimeuuid(?) AND id < minTimeuuid(?);" values=[dave 2015-04-05 00:05 2019-01-01 00:05] consistency=ONE]
... I always end up with an empty iterable. I am using very similar code elsewhere successfully, the difference here are the timeuuid functions. I've tried various arrangements with literal quotes to no avail, any advice is appreciated.
When I looked at the iterable in more detail I found :
"can not marshal string into timestamp"
... this led me to the answer - GoCQL : Marshal string into timestamp

Removing column gives syntax error [duplicate]

I have a problem: I need to delete a column from my SQLite database. I wrote this query
alter table table_name drop column column_name
but it does not work. Please help me.
Update: SQLite 2021-03-12 (3.35.0) now supports DROP COLUMN. The FAQ on the website is still outdated.
From: http://www.sqlite.org/faq.html:
(11) How do I add or delete columns from an existing table in SQLite.
SQLite has limited ALTER TABLE support that you can use to add a
column to the end of a table or to change the name of a table. If you
want to make more complex changes in the structure of a table, you
will have to recreate the table. You can save existing data to a
temporary table, drop the old table, create the new table, then copy
the data back in from the temporary table.
For example, suppose you have a table named "t1" with columns names
"a", "b", and "c" and that you want to delete column "c" from this
table. The following steps illustrate how this could be done:
BEGIN TRANSACTION;
CREATE TEMPORARY TABLE t1_backup(a,b);
INSERT INTO t1_backup SELECT a,b FROM t1;
DROP TABLE t1;
CREATE TABLE t1(a,b);
INSERT INTO t1 SELECT a,b FROM t1_backup;
DROP TABLE t1_backup;
COMMIT;
Instead of dropping the backup table, just rename it...
BEGIN TRANSACTION;
CREATE TABLE t1_backup(a,b);
INSERT INTO t1_backup SELECT a,b FROM t1;
DROP TABLE t1;
ALTER TABLE t1_backup RENAME TO t1;
COMMIT;
For simplicity, why not create the backup table from the select statement?
CREATE TABLE t1_backup AS SELECT a, b FROM t1;
DROP TABLE t1;
ALTER TABLE t1_backup RENAME TO t1;
This option works only if you can open the DB in a DB Browser like DB Browser for SQLite.
In DB Browser for SQLite:
Go to the tab, "Database Structure"
Select you table Select Modify table (just under the tabs)
Select the column you want to delete
Click on Remove field and click OK
=>Create a new table directly with the following query:
CREATE TABLE table_name (Column_1 TEXT,Column_2 TEXT);
=>Now insert the data into table_name from existing_table with the following query:
INSERT INTO table_name (Column_1,Column_2) FROM existing_table;
=>Now drop the existing_table by following query:
DROP TABLE existing_table;
PRAGMA foreign_keys=off;
BEGIN TRANSACTION;
ALTER TABLE table1 RENAME TO _table1_old;
CREATE TABLE table1 (
( column1 datatype [ NULL | NOT NULL ],
column2 datatype [ NULL | NOT NULL ],
...
);
INSERT INTO table1 (column1, column2, ... column_n)
SELECT column1, column2, ... column_n
FROM _table1_old;
COMMIT;
PRAGMA foreign_keys=on;
For more info:
https://www.techonthenet.com/sqlite/tables/alter_table.php
I've made a Python function where you enter the table and column to remove as arguments:
def removeColumn(table, column):
columns = []
for row in c.execute('PRAGMA table_info(' + table + ')'):
columns.append(row[1])
columns.remove(column)
columns = str(columns)
columns = columns.replace("[", "(")
columns = columns.replace("]", ")")
for i in ["\'", "(", ")"]:
columns = columns.replace(i, "")
c.execute('CREATE TABLE temptable AS SELECT ' + columns + ' FROM ' + table)
c.execute('DROP TABLE ' + table)
c.execute('ALTER TABLE temptable RENAME TO ' + table)
conn.commit()
As per the info on Duda's and MeBigFatGuy's answers this won't work if there is a foreign key on the table, but this can be fixed with 2 lines of code (creating a new table and not just renaming the temporary table)
For SQLite3 c++ :
void GetTableColNames( tstring sTableName , std::vector<tstring> *pvsCols )
{
UASSERT(pvsCols);
CppSQLite3Table table1;
tstring sDML = StringOps::std_sprintf(_T("SELECT * FROM %s") , sTableName.c_str() );
table1 = getTable( StringOps::tstringToUTF8string(sDML).c_str() );
for ( int nCol = 0 ; nCol < table1.numFields() ; nCol++ )
{
const char* pch1 = table1.fieldName(nCol);
pvsCols->push_back( StringOps::UTF8charTo_tstring(pch1));
}
}
bool ColExists( tstring sColName )
{
bool bColExists = true;
try
{
tstring sQuery = StringOps::std_sprintf(_T("SELECT %s FROM MyOriginalTable LIMIT 1;") , sColName.c_str() );
ShowVerbalMessages(false);
CppSQLite3Query q = execQuery( StringOps::tstringTo_stdString(sQuery).c_str() );
ShowVerbalMessages(true);
}
catch (CppSQLite3Exception& e)
{
bColExists = false;
}
return bColExists;
}
void DeleteColumns( std::vector<tstring> *pvsColsToDelete )
{
UASSERT(pvsColsToDelete);
execDML( StringOps::tstringTo_stdString(_T("begin transaction;")).c_str() );
std::vector<tstring> vsCols;
GetTableColNames( _T("MyOriginalTable") , &vsCols );
CreateFields( _T("TempTable1") , false );
tstring sFieldNamesSeperatedByCommas;
for ( int nCol = 0 ; nCol < vsCols.size() ; nCol++ )
{
tstring sColNameCurr = vsCols.at(nCol);
bool bUseCol = true;
for ( int nColsToDelete = 0; nColsToDelete < pvsColsToDelete->size() ; nColsToDelete++ )
{
if ( pvsColsToDelete->at(nColsToDelete) == sColNameCurr )
{
bUseCol = false;
break;
}
}
if ( bUseCol )
sFieldNamesSeperatedByCommas+= (sColNameCurr + _T(","));
}
if ( sFieldNamesSeperatedByCommas.at( int(sFieldNamesSeperatedByCommas.size()) - 1) == _T(','))
sFieldNamesSeperatedByCommas.erase( int(sFieldNamesSeperatedByCommas.size()) - 1 );
tstring sDML;
sDML = StringOps::std_sprintf(_T("insert into TempTable1 SELECT %s FROM MyOriginalTable;\n") , sFieldNamesSeperatedByCommas.c_str() );
execDML( StringOps::tstringTo_stdString(sDML).c_str() );
sDML = StringOps::std_sprintf(_T("ALTER TABLE MyOriginalTable RENAME TO MyOriginalTable_old\n") );
execDML( StringOps::tstringTo_stdString(sDML).c_str() );
sDML = StringOps::std_sprintf(_T("ALTER TABLE TempTable1 RENAME TO MyOriginalTable\n") );
execDML( StringOps::tstringTo_stdString(sDML).c_str() );
sDML = ( _T("DROP TABLE MyOriginalTable_old;") );
execDML( StringOps::tstringTo_stdString(sDML).c_str() );
execDML( StringOps::tstringTo_stdString(_T("commit transaction;")).c_str() );
}
In case anyone needs a (nearly) ready-to-use PHP function, the following is based on this answer:
/**
* Remove a column from a table.
*
* #param string $tableName The table to remove the column from.
* #param string $columnName The column to remove from the table.
*/
public function DropTableColumn($tableName, $columnName)
{
// --
// Determine all columns except the one to remove.
$columnNames = array();
$statement = $pdo->prepare("PRAGMA table_info($tableName);");
$statement->execute(array());
$rows = $statement->fetchAll(PDO::FETCH_OBJ);
$hasColumn = false;
foreach ($rows as $row)
{
if(strtolower($row->name) !== strtolower($columnName))
{
array_push($columnNames, $row->name);
}
else
{
$hasColumn = true;
}
}
// Column does not exist in table, no need to do anything.
if ( !$hasColumn ) return;
// --
// Actually execute the SQL.
$columns = implode('`,`', $columnNames);
$statement = $pdo->exec(
"CREATE TABLE `t1_backup` AS SELECT `$columns` FROM `$tableName`;
DROP TABLE `$tableName`;
ALTER TABLE `t1_backup` RENAME TO `$tableName`;");
}
In contrast to other answers, the SQL used in this approach seems to preserve the data types of the columns, whereas something like the accepted answer seems to result in all columns to be of type TEXT.
Update 1:
The SQL used has the drawback that autoincrement columns are not preserved.
Just in case if it could help someone like me.
Based on the Official website and the Accepted answer, I made a code using C# that uses System.Data.SQLite NuGet package.
This code also preserves the Primary key and Foreign key.
CODE in C#:
void RemoveColumnFromSqlite (string tableName, string columnToRemove) {
try {
var mSqliteDbConnection = new SQLiteConnection ("Data Source=db_folder\\MySqliteBasedApp.db;Version=3;Page Size=1024;");
mSqliteDbConnection.Open ();
// Reads all columns definitions from table
List<string> columnDefinition = new List<string> ();
var mSql = $"SELECT type, sql FROM sqlite_master WHERE tbl_name='{tableName}'";
var mSqliteCommand = new SQLiteCommand (mSql, mSqliteDbConnection);
string sqlScript = "";
using (mSqliteReader = mSqliteCommand.ExecuteReader ()) {
while (mSqliteReader.Read ()) {
sqlScript = mSqliteReader["sql"].ToString ();
break;
}
}
if (!string.IsNullOrEmpty (sqlScript)) {
// Gets string within first '(' and last ')' characters
int firstIndex = sqlScript.IndexOf ("(");
int lastIndex = sqlScript.LastIndexOf (")");
if (firstIndex >= 0 && lastIndex <= sqlScript.Length - 1) {
sqlScript = sqlScript.Substring (firstIndex, lastIndex - firstIndex + 1);
}
string[] scriptParts = sqlScript.Split (new string[] { "," }, StringSplitOptions.RemoveEmptyEntries);
foreach (string s in scriptParts) {
if (!s.Contains (columnToRemove)) {
columnDefinition.Add (s);
}
}
}
string columnDefinitionString = string.Join (",", columnDefinition);
// Reads all columns from table
List<string> columns = new List<string> ();
mSql = $"PRAGMA table_info({tableName})";
mSqliteCommand = new SQLiteCommand (mSql, mSqliteDbConnection);
using (mSqliteReader = mSqliteCommand.ExecuteReader ()) {
while (mSqliteReader.Read ()) columns.Add (mSqliteReader["name"].ToString ());
}
columns.Remove (columnToRemove);
string columnString = string.Join (",", columns);
mSql = "PRAGMA foreign_keys=OFF";
mSqliteCommand = new SQLiteCommand (mSql, mSqliteDbConnection);
int n = mSqliteCommand.ExecuteNonQuery ();
// Removes a column from the table
using (SQLiteTransaction tr = mSqliteDbConnection.BeginTransaction ()) {
using (SQLiteCommand cmd = mSqliteDbConnection.CreateCommand ()) {
cmd.Transaction = tr;
string query = $"CREATE TEMPORARY TABLE {tableName}_backup {columnDefinitionString}";
cmd.CommandText = query;
cmd.ExecuteNonQuery ();
cmd.CommandText = $"INSERT INTO {tableName}_backup SELECT {columnString} FROM {tableName}";
cmd.ExecuteNonQuery ();
cmd.CommandText = $"DROP TABLE {tableName}";
cmd.ExecuteNonQuery ();
cmd.CommandText = $"CREATE TABLE {tableName} {columnDefinitionString}";
cmd.ExecuteNonQuery ();
cmd.CommandText = $"INSERT INTO {tableName} SELECT {columnString} FROM {tableName}_backup;";
cmd.ExecuteNonQuery ();
cmd.CommandText = $"DROP TABLE {tableName}_backup";
cmd.ExecuteNonQuery ();
}
tr.Commit ();
}
mSql = "PRAGMA foreign_keys=ON";
mSqliteCommand = new SQLiteCommand (mSql, mSqliteDbConnection);
n = mSqliteCommand.ExecuteNonQuery ();
} catch (Exception ex) {
HandleExceptions (ex);
}
}
In Python 3.8...
Preserves primary key and column types.
Takes 3 inputs:
a sqlite cursor: db_cur,
table name: t and,
list of columns to junk: columns_to_junk
def removeColumns(db_cur, t, columns_to_junk):
# Obtain column information
sql = "PRAGMA table_info(" + t + ")"
record = query(db_cur, sql)
# Initialize two strings: one for column names + column types and one just
# for column names
cols_w_types = "("
cols = ""
# Build the strings, filtering for the column to throw out
for r in record:
if r[1] not in columns_to_junk:
if r[5] == 0:
cols_w_types += r[1] + " " + r[2] + ","
if r[5] == 1:
cols_w_types += r[1] + " " + r[2] + " PRIMARY KEY,"
cols += r[1] + ","
# Cut potentially trailing commas
if cols_w_types[-1] == ",":
cols_w_types = cols_w_types[:-1]
else:
pass
if cols[-1] == ",":
cols = cols[:-1]
else:
pass
# Execute SQL
sql = "CREATE TEMPORARY TABLE xfer " + cols_w_types + ")"
db_cur.execute(sql)
sql = "INSERT INTO xfer SELECT " + cols + " FROM " + t
db_cur.execute(sql)
sql = "DROP TABLE " + t
db_cur.execute(sql)
sql = "CREATE TABLE " + t + cols_w_types + ")"
db_cur.execute(sql)
sql = "INSERT INTO " + t + " SELECT " + cols + " FROM xfer"
db_cur.execute(sql)
You'll find a reference to a query() function. Just a helper...
Takes two inputs:
sqlite cursor db_cur and,
the query string: query
def query(db_cur, query):
r = db_cur.execute(query).fetchall()
return r
Don't forget to include a "commit()"!

Lua get index name of table as table

Is there any way to get every index value of a table?
Example:
local mytbl = {
["Hello"] = 123,
["world"] = 321
}
I want to get this:
{"Hello", "world"}
local t = {}
for k, v in pairs(mytbl) do
table.insert(t, k) -- or t[#t + 1] = k
end
Note that the order of how pairs iterates a table is not specified. If you want to make sure the elements in the result are in a certain order, use:
table.sort(t)

Hive combine column values based upon condition

I was wondering if it is possible to combine column values based upon a condition. Let me explain...
Let say my data looks like this
Id name offset
1 Jan 100
2 Janssen 104
3 Klaas 150
4 Jan 160
5 Janssen 164
An my output should be this
Id fullname offsets
1 Jan Janssen [ 100, 160 ]
I would like to combine the name values from two rows where the offset of the two rows are no more apart then 1 character.
My question is if this type of data manipulation is possible with and if it is could someone share some code and explaination?
Please be gentle but this little piece of code return some what what I want...
ArrayList<String> persons = new ArrayList<String>();
// write your code here
String _previous = "";
//Sample output form entities.txt
//USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Berkowitz,PERSON,9,10660
//USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,10685
File file = new File("entities.txt");
try {
//
// Create a new Scanner object which will read the data
// from the file passed in. To check if there are more
// line to read from it we check by calling the
// scanner.hasNextLine() method. We then read line one
// by one till all line is read.
//
Scanner scanner = new Scanner(file);
while (scanner.hasNextLine()) {
if(_previous == "" || _previous == null)
_previous = scanner.nextLine();
String _current = scanner.nextLine();
//Compare the lines, if there offset is = 1
int x = Integer.parseInt(_previous.split(",")[3]) + Integer.parseInt(_previous.split(",")[4]);
int y = Integer.parseInt(_current.split(",")[4]);
if(y-x == 1){
persons.add(_previous.split(",")[1] + " " + _current.split(",")[1]);
if(scanner.hasNextLine()){
_current = scanner.nextLine();
}
}else{
persons.add(_previous.split(",")[1]);
}
_previous = _current;
}
} catch (Exception e) {
e.printStackTrace();
}
for(String person : persons){
System.out.println(person);
}
Working of this piece sample data
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Richard,PERSON,7,2732
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,2740
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,2756
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,3093
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,3195
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Berkowitz,PERSON,9,3220
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Berkowitz,PERSON,9,10660
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,10685
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Lea,PERSON,3,10858
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Lea,PERSON,3,11063
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Ken,PERSON,3,11186
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Marottoli,PERSON,9,11234
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Berkowitz,PERSON,9,17073
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Lea,PERSON,3,17095
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Stephanie,PERSON,9,17330
USER.A-GovDocs-f83c6ca3-9585-4c66-b9b0-f4c3bd57ccf4,Putt,PERSON,4,17340
Which produces this output
Richard Marottoli
Marottoli
Marottoli
Marottoli
Berkowitz
Berkowitz
Marottoli
Lea
Lea
Ken
Marottoli
Berkowitz
Lea
Stephanie Putt
Kind regards
Load the table using below create table
drop table if exists default.stack;
create external table default.stack
(junk string,
name string,
cat string,
len int,
off int
)
ROW FORMAT DELIMITED
FIELDS terminated by ','
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
location 'hdfs://nameservice1/....';
Use below query to get your desired output.
select max(name), off from (
select CASE when b.name is not null then
concat(b.name," ",a.name)
else
a.name
end as name
,Case WHEN b.off1 is not null
then b.off1
else a.off
end as off
from default.stack a
left outer join (select name
,len+off+ 1 as off
,off as off1
from default.stack) b
on a.off = b.off ) a
group by off
order by off;
I have tested this it generates your desired result.