I need make a query for a postgresdb without identify accents (á, í,ö, etc).
I'm already use Knex.js as query builder, and postgresql have a unaccent extension that works fine in sql querys directly to db, but in my code i use knex and unaccent function throws error in querys.
Can anyone help me, ¿is possible make querys with knex.js that use unaccent function of postgresql?
My solution is to process the string before submitting the query using the following code:
const normalize = (str) => str.normalize('NFD').replace(/[\u0300-\u036f]/g, '');
console.log(normalize('Ấ Á Ắ Ạ Ê')) -> 'A A A A A'.
Or if you use postgresql version 13 or later it already supports that functionality.
select normalize('hồ, phố, ầ', NFC) → 'ho, pho, a' -- NFC (the default), NFD, NFKC, or NFKD.
Document: https://www.postgresql.org/docs/13/functions-string.html
Related
I have to remove the accents from the person's name, but I cannot apply the function in Talend while it works in SQL oracle.
this query works in my tDBInput component :
"SELECT '"+((String)globalMap.get("copyOfSORTIE.NOM"))+"' as nom_nom_compl,
'"+((String)globalMap.get("copyOfSORTIE.ENTETE"))+"' entete
FROM DUAL"
However, when I want to add the convert function, it doesn't work
this query does not work :
"SELECT '"+((String)globalMap.get(CONVERT("copyOfSORTIE.NOM",'US7ASCII')))+"' as nom_nom_compl,
'"+((String)globalMap.get("copyOfSORTIE.ENTETE"))+"' entete
FROM DUAL"
In my talend :
I am getting this error
What is the syntax for it to work?
Thank you!
Two things there :
I don't know the CONVERT method, but I can see that you are applying it to the key of your globalMap variable , and not the value (as if you wanted to convert "myKey" and not "myValue" which is attached to the key). Are you sure this is what you want to achieve ? if not, the syntax should be something similar to "SELECT CONVERT('"+((String)globalMap.get("copyOfSORTIE.NOM"))+"','US7ASCII') "
A useful java method implemented in talend is TalendString.removeAccents("") that you can apply directly on your talend variable, thus not using a SQL method.
I don't have extensive experience in sql but I know basic.
I use online sql executor to receive the json file which will be prodiced from this sql query.
SET group_concat_max_len=18446744073709547520;
SELECT
CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT(
'id',messages.message_id,
'mailing_list_url',messages.mailing_list_url,
'type_of_recipient',messages_people.type_of_recipient,
'email_address',messages_people.email_address,
'message_body',messages.message_body,
'is_response_of',messages.is_response_of
) SEPARATOR ',\r'),
']') AS list
FROM messages
LEFT JOIN messages_people
ON messages.message_id = messages_people.message_id
AND messages.mailing_list_url = messages_people.mailing_list_url
WHERE email_address IN ('dr#gorsvet.kz', 'mathews#uk2.net', 'd.jentsch#fu-berlin.de', 'atul.soman#microfocus.com', 'kkrugler_lists#transpac.com', 'hcorg#minions.org.pl', 'bruno.ronchetti#mac.com', 'christophe.thiebaud#sap.com', 'kc.baltz#copart.com', 'havanki4j#gmail.com', 'joseph.obernberger#ngc.com', 'goran#roseen.se', 'hboutemy#apache.org', 'johann#gyger.name', 'fredrik#jonson.org', 'jose-marcio.martins#mines-paristech.fr', 'dmytro.kostiuchenko#gmail.com', 'jochen.wiedmann#softwareag.com', 'guyskk#qq.com', 'alejandroscandroli#gmail.com', '1983-01-06#gmx.net', 'jstrayer#proofpoint.com', 'daniel#degu.cl', 'hontvari#flyordie.com', 'david#andl.org', 'gholmes#pinsightmedia.com', 'dennis.geurts#luminis.eu', 'carlspring#gmail.com', 'jozef.koval#protonmail.ch', 'gabriele.ctn#gmail.com', 'ashish.disawal#evivehealth.com', 'aarslan2#anadolu.edu.tr', 'kshukla#yahoo-inc.com', 'alex.huang#gmail.com', 'markus#openindex.io', 'marohn#sipgate.de', 'ea#apache.org', 'krzysztof.szalast#gmail.com', 'kfoskey#tpg.com.au', 'krauss#in.tum.de', 'kenhans#hotmail.co.uk', 'jokin.c#odeian.com', 'eitch#eitchnet.ch', 'm.kristian#web.de', 'bob.sandiford#sirsidynix.com', 'asandstrom2#eastlink.ca', 'kelly#apache.org', 'antony.blakey#linkuistics.com.au', 'ashawki#hotmail.com', 'h.klocker-mark#tsn.at');
However from the first line I receive error.
As instructions are not friendly for me please can you help me how could I take the final file?
I had no clue that there are such things as online sql executors. Basically SQL injections as a service, nice.
Anyways, the command:
SET group_concat_max_len=18446744073709547520;
is not part of the official SQL standard and specific to the database server you are using (here MYSQL). From the manual for group_concat_max_len:
The maximum permitted result length in bytes for the GROUP_CONCAT() function. The default is 1024.
For this very reason a "online SQL executor" (wow, I am still puzzled) won't be able to run this. Anyways if you need this, you probably won't get around setting up a MYSQL server yourself. Or you could just use a container.
I want to convert this SQL query to JOOQ DSL.
select "p".*, array_agg("pmu") as projectmemberusers
from "Projects" as "p"
join "ProjectMemberUsers" as "pmu" on "pmu"."projectId" = "p"."id"
group by "p"."id";
Currently i have tried doing something like this using JOOQ:
val p = PROJECTS.`as`("p")
val pmu = PROJECTMEMBERUSERS.`as`("pmu")
val query = db.select(p.asterisk(), DSL.arrayAgg(pmu))
.from(p.join(pmu).on(p.ID.eq(pmu.PROJECTID)))
.groupBy(p.ID)
This does not work because DSL.arrayAgg expects something of type Field<T> as input.
I am new to JOOQ and not an SQL professional. Detailed explanations and impovement suggestions are highly appreciated.
First of all, the syntax indeed works, checked this in SQL Fiddle: http://sqlfiddle.com/#!17/e45b7/3
But it's not documented in detail: https://www.postgresql.org/docs/9.5/static/functions-aggregate.html
https://www.postgresql.org/docs/current/static/rowtypes.html#ROWTYPES-USAGE
That's probably the reason jOOQ doesn't support this currently: https://github.com/jOOQ/jOOQ/blob/master/jOOQ/src/main/java/org/jooq/impl/DSL.java#L16856
The only syntax that will work currently is with a single field: DSL.arrayAgg(pmu.field(1))
What you're looking for is a way to express PostgreSQL's "anonymous" nested records through the jOOQ API, similar to what is requested in this feature request: https://github.com/jOOQ/jOOQ/issues/2360
This is currently not possible in the jOOQ API as of version 3.11, but it definitely will be in the future.
Workaround 1
You could try using the experimental DSL.rowField() methods on a Row[N]<...> representation of your table type. This may or may not work yet, as the feature is currently not supported.
Workaround 2
A workaround is to create a type:
create type my_type as (...) -- Same row type as your table
And a view:
create view x as
select "p".*, array_agg("pmu"::my_type) as projectmemberusers
from "Projects" as "p"
join "ProjectMemberUsers" as "pmu" on "pmu"."projectId" = "p"."id"
group by "p"."id";
And then use the code generator to pick up the resulting type.
If I were retrieving the data I wanted from a plain sql query, the following would suffice:
select * from stvterm where stvterm_code > TT_STUDENT.STU_GENERAL.F_Get_Current_term()
I have a grails domain set up correctly for this table, and I can run the following code successfully:
def a = SaturnStvterm.findAll("from SaturnStvterm as s where id > 201797") as JSON
a.render(response)
return false
In other words, I can hardcode in the results from the Oracle function and have the HQL run correctly, but it chokes any way that I can figure to try it with the function. I have read through some of the documentation on Hibernate about using procs and functions, but I'm having trouble making much sense of it. Can anyone give me a hint as to the proper way to handle this?
Also, since I think it is probably relevant, there aren't any synonyms in place that would allow the function to be called without qualifying it as schema.package.function(). I'm sure that'll make things more difficult. This is all for Grails 1.3.7, though I could use a later version if needed.
To call a function in HQL, the SQL dialect must be aware of it. You can add your function at runtime in BootStrap.groovy like this:
import org.hibernate.dialect.function.SQLFunctionTemplate
import org.hibernate.Hibernate
def dialect = applicationContext.sessionFactory.dialect
def getCurrentTerm = new SQLFunctionTemplate(Hibernate.INTEGER, "TT_STUDENT.STU_GENERAL.F_Get_Current_term()")
dialect.registerFunction('F_Get_Current_term', getCurrentTerm)
Once registered, you should be able to call the function in your queries:
def a = SaturnStvterm.findAll("from SaturnStvterm as s where id > TT_STUDENT.STU_GENERAL.F_Get_Current_term()")
I have the following statement in Rails 3 using an SQLite3 database:
word = 'Hello'
word_entry = Word.where("name REGEXP :word", {:word => "[[:<:]]#{word}[[:>:]]"})
However, when running this under SQLite3, I keep getting:
SQLite3::SQLException: no such function: REGEXP
I read in the SQLite3 documentation that it does indeed support the REGEXP function.
In my gemfile, I have the line
gem 'sqlite3'
And my database config file looks like this:
development:
adapter: sqlite3
database: db/development.sqlite3
pool: 5
timeout: 5000
Any ideas what's going on?
RESOLUTION:
I ended up finding this solution. Unfortunately, it doesn't work for Rails 3.
So to use regular expressions I ended up switching to MYSQL instead of SQLite3.
I ran into the same issue. I took the code used in the resolution, ported it to work with Rails 3+ and made a gem for easier use. I hope this helps.
https://github.com/sei-mi/sqlite3_ar_regexp
From the fine manual:
The REGEXP operator is a special syntax for the regexp() user function. No regexp() user function is defined by default and so use of the REGEXP operator will normally result in an error message. If a application-defined SQL function named "regexp" is added at run-time, that function will be called in order to implement the REGEXP operator.
So the grammar supports REGEXP but the default SQLite library does not provide an implementation for it. You'll have to hook up your own implementation through some C wrangling if you want or need such a thing.
Presumably the rationale is that the SQLite people want to keep SQLite as small and tight as possible but including a whole regular expression library would add weight that most people don't want. Also, they would have to choose a regular expression library and include it with the SQLite source or they'd have to put up with the vagaries of everyone's regular expression support in libc. I'm not one of the SQLite developers so this is pure speculation.
I'm guessing that you'll probably have to make do with LIKE and GLOB. Using LIKE will provide a more portable solution.
You may be intested in the sqlite3-pcre package, which implements REGEXP for SQLite.
See this comment on a similar issue.
I had a similar question, and found a Gem named wherex that is well documented and worked out of the box.
Your expression from above
Word.where("name REGEXP :word", {:word => "[[:<:]]#{word}[[:>:]]"})
would there be
Word.where(:name => Regexp.new("[[:<:]]#{word}[[:>:]]"))
Works like a charm for me :-)
From source of sqlite3_ar_regexp project, I extract this:
db = SQLite3::Database.open( database_name )
db.create_function('regexp', 2) do |func, pattern, expression|
func.result = expression.to_s.match(
Regexp.new(pattern.to_s, Regexp::IGNORECASE)) ? 1 : 0
end
From source of sqlite3_ar_regexp project, I extract this:
db = ActiveRecord::Base.connection.raw_connection
db.create_function('regexp', 2) do |func, pattern, expression|
func.result = expression.to_s.match(
Regexp.new(pattern.to_s, Regexp::IGNORECASE)) ? 1 : 0
end
Improved upon a previous answer with ActiveRecord::Base.connection.raw_connection so that db name isn't needed