I'm discovering Express by creating a simple CRUD without ORM.
Issue is, I'm not able to find any record through the Model.findBy() function
model User {
static async findBy(payload) {
try {
let attr = Object.keys(payload)[0]
let value = Object.values(payload)[0]
let user = await pool.query(
`SELECT * from users WHERE $1::text = $2::text LIMIT 1;`,
[attr, value]
);
return user.rows; // empty :-(
} catch (err) {
throw err
}
}
}
User.findBy({ email: 'foo#bar.baz' }).then(console.log);
User.findBy({ name: 'Foo' }).then(console.log);
I've no issue using psql if I surround $2::text by single quote ' like:
SELECT * FROM users WHERE email = 'foo#bar.baz' LIMIT 1;
Though that's not possible inside parametrized queries. I've tried stuff like '($2::text)' (and escaped variations), but that looks far from what the documentation recommends.
I must be missing something. Is the emptiness of user.rows related to the way I fetch attr & value ? Or maybe, is some kind of escape required when passing string parameters ?
"Answer":
As stated in the comment section, issue isn't related to string escape, but to dynamic column names.
Column names are not identifiers, and therefore cannot be dynamically set using a query parameter.
See: https://stackoverflow.com/a/50813577/11509906
Related
Let’s say you have a basic API (GET/POST/PATCH/DELETE) backed by an SQL database.
The PATCH call should only update the fields in the JSON payload that the user sends, without touching any of the other fields.
Imagine the table (let's call it sample) has id, string_a and string_b columns, and the struct which corresponds to it looks like:
type Sample struct {
ID int `json:"id"`
StringA string `json:"stringA"`
StringB string `json:"stringB"`
}
Let's say the user passes in { "stringA": "patched value" } as payload. The json will be unmarshalled to something that looks like:
&Sample{
ID: 0,
StringA: "patched value",
StringB: "",
}
For a project using database/sql, you’d write the query to patch the row something like:
// `id` is from the URL params
query := `UPDATE sample SET string_a=$1, string_b=$2 WHERE id=$3`
row := db.QueryRow(query, sample.StringA, sample.StringB, id)
...
That query would update the string_a column as expected, but it’d also update the string_b column to "", which is undesired behavior in this case. In essence, I’ve just created a PUT instead of a PATCH.
My immediate thought was - OK, that’s fine, let’s use strings.Builder to build out the query and only add a SET statement for those that have a non-nil/empty value.
However, in that case, if a user wanted to make string_a empty, how would they accomplish that?
Eg. the user makes a PATCH call with { "stringA": "" } as payload. That would get unmarshalled to something like:
&Sample{
ID: 0,
StringA: "",
StringB: "",
}
The “query builder” I was theorizing about would look at that and say “ok, those are all nil/empty values, don’t add them to the query” and no columns would be updated, which again, is undesired behavior.
I’m not sure how to write my API and the SQL queries it runs in a way that satisfies both cases. Any thoughts?
I think reasonable solution for smaller queries is to build UPDATE query and list of bound parameters dynamically while processing payload with logic that recognizes what was updated and what was left empty.
From my own experience this is clear and readable (if repetitive you can always iterate over struct members that share same logic or employ reflection and look at struct tags hints, etc.). Every (my) attempt to write universal solution for this ended up as very convoluted overkill supporting all sorts of corner-cases and behavioral differences between endpoints.
func patchSample(s Sample) {
var query strings.Builder
params := make([]interface{}, 0, 2)
// TODO Check if patch makes sense (e.g. id is non-zero, at least one patched value provided, etc.
query.WriteString("UPDATE sample SET")
if s.StringA != "" {
query.WriteString(" stringA = ?")
params = append(params, s.StringA)
}
if s.StringB != "" {
query.WriteString(" stringB = ?")
params = append(params, s.StringB)
}
query.WriteString(" WHERE id = ?")
params = append(params, s.ID)
fmt.Println(query.String(), params)
//_, err := db.Exec(query.String(), params...)
}
func main() {
patchSample(Sample{1, "Foo", ""})
patchSample(Sample{2, "", "Bar"})
patchSample(Sample{3, "Foo", "Bar"})
}
EDIT: In case "" is valid value for patching then it needs to be distinguishable from the default empty value. One way how to solve that for string is to use pointer which will default to nil if value is not present in payload:
type Sample struct {
ID int `json:"id"`
StringA *string `json:"stringA"`
StringB *string `json:"stringB"`
}
and then modify condition(s) to check if field was sent like this:
if s.StringA != nil {
query.WriteString(" stringA = ?")
params = append(params, *s.StringA)
}
See full example in playground: https://go.dev/play/p/RI7OsNEYrk6
For what it's worth, I solved the issue by:
Converting the request payload to a generic map[string]interface{}.
Implementing a query builder that loops through the map's keys to create a query.
Part of the reason I went this route is it fit all my requirements, and I didn't particularly like having *strings or *ints laying around.
Here is what the query builder looks like:
func patchQueryBuilder(id string, patch map[string]interface{}) (string, []interface{}, error) {
var query strings.Builder
params := make([]interface{}, 0)
query.WriteString("UPDATE some_table SET")
for k, v := range patch {
switch k {
case "someString":
if someString, ok := v.(string); ok {
query.WriteString(fmt.Sprintf(" some_string=$%d,", len(params)+1))
params = append(params, someString)
} else {
return "", []interface{}{}, fmt.Errorf("could not process some_string")
}
case "someBool":
if someBool, ok := v.(bool); ok {
query.WriteString(fmt.Sprintf(" some_bool=$%d,", len(params)+1))
params = append(params, someBool)
} else {
return "", []interface{}{}, fmt.Errorf("could not process some_bool")
}
}
}
if len(params) > 0 {
// Remove trailing comma to avoid syntax errors
queryString := fmt.Sprintf("%s WHERE id=$%d RETURNING *", strings.TrimSuffix(query.String(), ","), len(params)+1)
params = append(params, id)
return queryString, params, nil
} else {
return "", []interface{}{}, nil
}
}
Note that I'm using PostgreSQL, so I needed to provide numbered parameters to the query, eg $1, which is what params is used for. It's also returned from the function so that it can be used as follows:
// Build the patch query based on the payload
query, params, err := patchQueryBuilder(id, patch)
if err != nil {
return nil, err
}
// Use the query/params and get output
row := tx.QueryRowContext(ctx, query, params...)
Currently, I have a SQL query that returns information to me in a DBArrayList.
It returns data in this format : [{id=2kjhjlkerjlkdsf324523}]
For the next step, I need it to be in a List<Map> format without the id: [2kjhjlkerjlkdsf324523]
The Datatypes being used are DBArrayList, and List.
If it helps any, the next step is a function to collect the list and then to replace all single quotes if any [SQL-Injection prevention]. Using:
listMap = listMap.collect() { "'" + Util.removeSingleQuotes(it) + "'" }
public static String removeSingleQuotes(s) {
return s ? s.replaceAll(/'"/, '') : s
}
I spent this morning working on it, and I found out that I needed to actually collect the DBArrayList like this:
listMap = dbArrayList.collect { it.getAt('id')}
If you're in a bind like I was and restrained to a specific schema this might help, but #ou_ryperd has the correct answer!
While using a DBArrayList is not wrong, Groovy's idiom is to use the db result as a collection. I would suggest you use it that way directly from the db:
Map myMap = [:]
dbhandle.eachRow("select fieldSomeID, fieldSomeVal from yourTable;") { row ->
map[row.fieldSomeID] = row.fieldSomeVal.replaceAll(/'"/, '')
}
function mysql_insert($data_array){
$sql = "insert into `". $this->table_name. '`';
$array_keys = array_keys($data_array);
$array_keys_comma = implode(",\n", preg_replace('/^(.*?)$/', "`$1`", $array_keys));
for($a=0,$b=count($data_array); $a<$b; $a++){ $question_marks .="?,"; }
$array_values = array_values($data_array);
$array_values_comma = implode(",", $array_values);
$sql.= " ($array_keys_comma) ";
$sql.= " values(". substr($question_marks, 0,-1) .")";
$prepare = $this->connDB->prepare($sql);
$insert = $prepare->execute(array($array_values_comma));
}
I want to creat like this universal functions, $data_array-comes from $_POST
This function will work for all form. But i dont know what is my wrong :S
I don't know what is my wrong
That's quite easy to know: number of bound variables does not match number of tokens.
I want to creat like this universal functions, $data_array-comes from $_POST
Here you go: Insert/update helper function using PDO
$array_values_comma is a scalar after you implode() the array. So you always pass an array of one element to your execute() function. You should pass $array_values.
Here's how I'd write this function:
function mysql_insert($data_array){
$columns = array_keys($data_array);
$column_list_delimited = implode(",",
array_map(function ($name) { return "`$name`"; }, $columns));
$question_marks = implode(",", array_fill(1, count($data_array), "?"));
$sql = "insert into `{$this->table_name}` ($column_list_delimited)
values ($question_marks)";
// always check for these functions returning FALSE, which indicates an error
// or alternatively set the PDO attribute to use exceptions
$prepare = $this->connDB->prepare($sql);
if ($prepare === false) {
trigger_error(print_r($this->connDB->errorInfo(),true), E_USER_ERROR);
}
$insert = $prepare->execute(array_values($data_array));
if ($insert === false) {
trigger_error(print_r($prepare->errorInfo(),true), E_USER_ERROR);
}
}
A further improvement would be to do some validation of $this->table_name and the keys of $data_array so you know they match an existing table and its columns.
See my answer to escaping column name with PDO for an example of validating column names.
I've noticed that when ordering by a datetime column in CI with active record, it's treating the column as a string, or int.
Example:
$this->db->limit(12);
$this->db->where('subscribed',1);
$this->db->join('profiles','profiles.user_id=users.id');
$this->db->where('active',1);
$this->db->select('users.thumbUpload,users.vanity_url');
$this->db->select('users.created_on as time');
$this->db->order_by('time');
$query = $this->db->get('users');
This is where users.created_on is a datetime field. Firstly, is it because active record is rendering time escaped, or is it something else? And if it is, can I prevent the escaping on order_by somehow?
Also, stackoverflow, please stop autocorrecting 'datetime' to 'date time'. It's annoying.
Cheers!
When you set second argument as false, function wont check and escape string. Try this
$this->db->select('users.created_on as time', FALSE);
Or for you query use
$this->db->order_by('users.created_on', 'DESC'); //or ASC
And for complex queries
$this->db->query("query");
According to the signature of the method in core files of CI (currently 2.2), it does not have any option to allow to choose whether or not to escape.
// The original prototype of the order_by()
public function order_by($orderby, $direction = '') {
// Definition
}
As you see there is not argument as $escape = true in the argument list. One way to do so is to hack this core file (I normally do not suggest it, since if you upgrade CI to a newer version later, then these changes will be lost, but if you do not intend to do so, it is OK to use it).
To do so, first change the prototype as:
public function order_by($orderby, $direction = '', $escape = true) {
// Definition
}
And then check the conditions in the following parts of definition:
// Line 842
if($escape){
$part = $this->_protect_identifiers(trim($part));
}else {
$part = trim($part);
}
// Line 856
if($escape){
$orderby = $this->_protect_identifiers($orderby);
}
When you call it, to prevent the escaping:
$this->db->order_by($ORDERBY_CLAUSE, null, false);
I'm trying to build a query builder, where the sObject result can contain an indeterminate number of fields. I'm using the result to build a dynamic table, but I can't figure out a way to read the sObject for a list of fields that were in the query.
I know how to get a list of ALL fields using the getDescribe information, but the query might not contain all of those fields.
Is there a way to do this?
Presumably you're building the query up as a string, since it's dynamic, so couldn't you just loop through the fields in the describe information, and then use .contains() on the query string to see if it was requested? Not crazy elegant, but seems like the simplest solution here.
Taking this further, maybe you have the list of fields selected in a list of strings or similar, and you could just use that list?
Not sure if this is exactly what you were after but something like this?
public list<sObject> Querylist {get; set;}
Define Search String
string QueryString = 'select field1__c, field2__c from Object where';
Add as many of these as you need to build the search if the user searches on these fields
if(searchParameter.field1__c != null && searchParameter.field1__c != '')
{
QueryString += ' field1__c like \'' + searchParameter.field1__c + '%\' and ';
}
if(searchParameter.field2__c != null && searchParameter.field2__c != '')
{
QueryString += ' field2__c like \'' + searchParameter.field2__c + '%\' and ';
}
Remove the last and
QueryString = QueryString.substring(0, (QueryString.length()-4));
QueryString += ' limit 200';
add query to the list
for(Object sObject : database.query(QueryString))
{
Querylist.add(sObject);
}
To get the list of fields in an sObject, you could use a method such as:
public Set<String> getFields(sObject sobj) {
Set<String> fieldSet = new Set<String>();
for (String field : sobj.getSobjectType().getDescribe().fields.getMap().keySet()) {
try {
a.get(field);
fieldSet.add(field);
} catch (Exception e) {
}
}
return fieldSet;
}
You should refactor to bulkily this approach for your context, but it works. Just pass in an sObject and it'll give you back a set of the field names.
I suggest using a list of fields for creating both the query and the table. You can put the list of fields in the result so that it's accesible for anyone using it. Then you can construct the table by using result.getFields() and retrieve the data by using result.getRows().
for (sObject obj : result.getRows()) {
for (String fieldName : result.getFields()) {
table.addCell(obj.get(fieldName));
}
}
If your trying to work with a query that's out of your control, you would have to parse the query to get the list of fields. But I wouldn't suggest trying that. It complicates code in ways that are hard to follow.