Using UDT:
Note: cqlsh: show version
[cqlsh 5.0.1 | Cassandra 2.2.1 | CQL spec 3.3.0 | Native protocol v4]
Tables
CREATE TYPE fullname (
firstname text,
lastname text
);
CREATE TABLE sample (
id int PRIMARY KEY,
name frozen <fullname>
);
Code:
type Frozen struct {
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
}
myobj := &Frozen{
Firstname: "pony",
Lastname: "jim",
}
if err := session.Query(`INSERT INTO sample (id, name) VALUES (?,?)`,
2, myobj).Exec(); err != nil {
fmt.Println(err)
}
I get:
UDT are not available on protocols less than 3, please update config
Update config?
I updated Cassandra to 3.3 and it worked.
The error message tricked me into thinking that having "Native protocol v4" was ok running on Cassandra 2.2.
Upgrading to 3.3 I also had to add the following, just after defining keyspace when creating a session:
ClusterName.ProtoVersion = 4
Related
I've below class definition for data transfer object. I use spring-data-aerospike to persistence.
import org.springframework.data.annotation.Id;
public class User implements Serializable {
#Id
String uid;
Map<String, Object> ext = new HashMap<String, Object>();
// getters & setters
}
Sample data in the database is like -
select ext from department.User;
+-------+-------------------------+----------------------------------+
| PK | ext || #_class |
+-------+-------------------------+----------------------------------+
| "123" | MAP('{"idfa": "xyz"}') | "com.tut.dto.User" |
| "234" | MAP('{}') | "com.tut.dto.User" |
+-------+-------------------------+----------------------------------+
I need to query the database now that it should only return the records which have "idfa" key string in the ext field column.
I tried the following. But it didn't work
1.
select * from test.UserRecord where ext.idfa is not null;
Unsupported command format with token - '.'
Make sure string values are enclosed in quotes.
Type " aql --help " from console or simply "help" from within the aql-prompt.
```
2.
select * from test.UserRecord where ext contains 'idfa';
Unsupported command format with token - ''idfa''
Make sure string values are enclosed in quotes.
Type " aql --help " from console or simply "help" from within the aql-prompt.
How can I make it work?
Try executing:
select * from department.user in mapkeys where ext = "idfa"
Based on the following structure:
SELECT <bins> FROM <ns>[.<set>] IN <indextype> WHERE <bin> = <value>
Assuming you've created an index with collection type of "MAPKEYS", you can create it using #Indexed annotation on the ext field in your User class.
If your using Spring Data Aerospike latest version it should look something like that:
#Indexed(name = "indexName", type = IndexType.STRING, collectionType = IndexCollectionType.MAPKEYS)
Map<String, Object> ext = new HashMap<String, Object>();
If you're interested in interacting with Aerospike List/Map bins through code (instead of AQL) you should read about CDT (Collection Data Type) operations.
CDT documentation:
https://docs.aerospike.com/docs/guide/cdt.html
Map Operation class (including methods documentation):
https://github.com/aerospike/aerospike-client-java/blob/master/client/src/com/aerospike/client/cdt/MapOperation.java
And some examples in the Aerospike Java Client:
https://github.com/aerospike/aerospike-client-java/blob/master/test/src/com/aerospike/test/sync/basic/TestOperateMap.java
I had exported a bunch of tables (>30) as CSV files from MySQL database using phpMyAdmin. These CSV file contains NULL values like:
"id","sourceType","name","website","location"
"1","non-commercial","John Doe",NULL,"California"
I imported many such csv to a PostgreSQL database with TablePlus. However, the NULL values in the columns are actually appearing as text rather than null.
When my application fetches the data from these columns it actually retrieves the text 'NULL' rather than a null value.
Also SQL command with IS NULL does not retrieve these rows probably because they are identified as text rather than null values.
Is there a SQL command I can do to convert all text NULL values in all the tables to actual NULL values? This would be the easiest way to avoid re-importing all the tables.
PostgreSQL's COPY command has the NULL 'some_string' option that allows to specify any string as NULL value: https://www.postgresql.org/docs/current/sql-copy.html
This would of course require re-importing all your tables.
Example with your data:
The CSV:
"id","sourceType","name","website","location"
"1","non-commercial","John Doe",NULL,"California"
"2","non-commercial","John Doe",NULL,"California"
The table:
CREATE TABLE import_with_null (id integer, source_type varchar(50), name varchar(50), website varchar(50), location varchar(50));
The COPY statement:
COPY import_with_null (id, source_type, name, website, location) from '/tmp/import_with_NULL.csv' WITH (FORMAT CSV, NULL 'NULL', HEADER);
Test of the correct import of NULL strings as SQL NULL:
SELECT * FROM import_with_null WHERE website IS NULL;
id | source_type | name | website | location
----+----------------+----------+---------+------------
1 | non-commercial | John Doe | | California
2 | non-commercial | John Doe | | California
(2 rows)
The important part that transforms NULL strings into SQL NULL values is NULL 'NULL' and could be any other value NULL 'whatever string'.
UPDATE For whoever comes here looking for a solution
See answers for two potential solutions
One of the solutions provides a SQL COPY method which must be performed before the import itself. The solution is provided by Michal T and marked as accepted answer is the better way to prevent this from happening in the first place.
My solution below uses a script in my application (Built in Laravel/PHP) which can be done after the import is already done.
Note- See the comments in the code and you could potentially figure out a similar solution in other languages/frameworks.
Thanks to #BjarniRagnarsson suggestion in the comments above, I came up with a short PHP Laravel script to perform update queries on all columns (which are of type 'string' or 'text') to replace the 'NULL' text with NULL values.
public function convertNULLStringToNULL()
{
$tables = DB::connection()->getDoctrineSchemaManager()->listTableNames(); //Get list of all tables
$results = []; // an array to store the output results
foreach ($tables as $table) { // Loop through each table
$columnNames = DB::getSchemaBuilder()->getColumnListing($table); //Get list of all columns
$columnResults = []; // array to store the results per column
foreach ($columnNames as $column) { Loop through each column
$columnType = DB::getSchemaBuilder()->getColumnType($table, $column); // Get the column type
if (
$columnType == 'string' || //check if column type is string or text
$columnType == 'text'
) {
$query = "update " . $table . " set \"" . $column . "\"=NULL where \"" . $column . "\"='NULL'"; //Build the update query as mentioned in comments above
$r = DB::update($query); //perform the update query
array_push($columnResults, [
$column => $r
]); //Push the column Results
}
}
array_push($results, [
$table => $columnResults
]); // push the table results
}
dd($results); //Output the results
}
Note I was using Laravel 8 for this.
This seems like a common problem and may be posted somewhere already, but I can't find any threads talking about it, so here is the problem:
I have a Postgres table storeing a column of type BYTEA.
CREATE TABLE foo (
id VARCHAR PRIMARY KEY,
json_data BYTEA
)
The column json_data is really just JSON stored as BYTEA (It's not ideal I know). It is unstructured, but guaranteed to be of string -> string JSON.
When I query this table, I need to scan the query SELECT * FROM foo WHERE id = $1 into the following struct:
type JSONData map[string]string
type Foo struct {
ID string `db:"id"`
Data JSONData `db:"json_data"`
}
I'm using sqlx's Get method. When I execute a query I'm getting the error message sql: Scan error on column index 1, name "json_data": unsupported Scan, storing driver.Value type []uint8 into type *foo.JSONData.
Obviously, the scanner is having trouble scanning the JSON BYTEA into a map. I can implement my own scanner and call my custom scanner on the json_data column, but I'm wondering if there are better ways to do this. Could my JSONData type implement an existing interface to do this automatically?
As suggested by #iLoveReflection, implementing the Scanner interface on *JSONData worked. Here is the actual implementation:
func (j *JSONData) Scan(src interface{}) error {
b, ok := src.([]byte)
if !ok {
return errors.New("invalid data type")
}
return json.Unmarshal(b, j)
}
I am trying to Query a single row from a PostgreSQL database table.
func getPrefix(serverID int64, db *sql.DB) string {
var prefix string
err := db.QueryRow("SELECT prefix FROM servers WHERE serverid = 1234").Scan(&prefix)
if err != nil {
fmt.Println(err.Error())
}
spew.Dump(prefix)
fmt.Println("Prefix is " + prefix)
return prefix
}
Apparently, the variable prefix is an empty String, but when I query it in the database, it's not empty
You are now connected to database "mewbot" as user "postgres".
mewbot=# select * from servers;
serverid | prefix
----------+--------
1234 | ;
(1 row)
mewbot=#
My question is, why is it returning an Empty String when it should be ;
All checks taken; I've made sure I'm connected to the same database et al
Apparently it was not working because SSL mode was disabled on my server but I was trying to connect without specifying it was disabled.
Changing postgres://postgres:7890#localhost:5432/mewbot to postgres://postgres:7890#localhost:5432/mewbot?sslmode=disable solved my issue.
I'm learning to use Sequelize to use with a PostgreSQL database. All of the following is happening on a dev. environment. This happened while manually trying to insert data into my tables to check if things are setup correctly through Sequelize, check on failing unit tests, etc.
I've made two tables with Sequelize models: User and Publication. Both these tables are generating UUIDv4. I've associated the User hasMany Publications, and Publication belongsTo User (you may reference the extra info).
On my psql shell, I've inserted the following record to my User table (rest of the data cut out for brevity):
| id | firstName | lastName | ..|
|----------------------------------------|------------|-----------|---|
| 8c878e6f-ee13-4a37-a208-7510c2638944 | Aiz | .... |...|
Now I'm trying to insert a record into my Publication table while referencing my newly created user above. Here's what I entered into the shell:
INSERT INTO "Publications"("title", "fileLocation", ..., "userId")VALUES('How to Pasta', 'www.pasta.com', ..., 8c878e6f-ee13-4a37-a208-7510c2638944);
It fails and I receive the following error:
ERROR: syntax error at or near "c878e6f"
LINE 1: ...8c878e6f-ee...
(it points to the second character on the terminal in LINE 1 reference - the 'c').
What's wrong here? Are we supposed to enter UUIDs another way if we want to do it manually in psql? Do we paste the referenced UUID as a string? Is there a correct way I'm missing from my own research?
Some extra info if it helps:
From my models:
Publication.associate = function(models) {
// associations can be defined here
Publication.belongsTo(models.User, {
foreignKey: "userId"
});
};
and
User.associate = function(models) {
// associations can be defined here
User.hasMany(models.Publication, {
foreignKey: "userId",
as: "publications"
});
};
Here's how I've defined userId in Publication:
userId: {
type: DataTypes.UUID,
references: {
model: "User",
key: "id",
as: "userId"
}
}
If it's worth anything, my (primaryKey) id on both models are type: DataTypes.UUID, defaultValue: DataTypes.UUIDV4 (I don't know if this is an issue).
surround your uuid in apostrophes (write it as a string) and pg will convert it to a uuid
Starting and ending your string with {} is optional
Eg
INSERT INTO "Publications"("title", "fileLocation", ..., "userId")VALUES('How to Pasta', 'www.pasta.com', ..., '8c878e6f-ee13-4a37-a208-7510c2638944');
Or
INSERT INTO "Publications"("title", "fileLocation", ..., "userId")VALUES('How to Pasta', 'www.pasta.com', ..., '{8c878e6f-ee13-4a37-a208-7510c2638944}');
Source (I don't do pgsql much so I casted around for another person who wrote some working pgsql. If this doesn't work out for you let me know and I'll remove the answer): PostgreSQL 9.3: How to insert upper case UUID into table