I am not very experienced with SQL outside of the basics. I'm identifying places in our code that need refactoring and I am trying to optimize some code to limit the number of calls to the DB. I've found some SQL updates that are all updating the same table and am curious how I could go about this...
Here are the update statements:
Host Variables:
[u.contentTitle, u.contentDescription, "title and dscr update", u.urlId, u.statusCode, u.urlId, u.statusCode]
[u.contentTitle, u.contentDescription, "title and dscr update", u.urlId, u.statusCode, u.urlId, u.statusCode]
[null, null, 'title and dscr update', u.urlId, u.statusCode]
UPDATE URL
SET CONTENT_TITLE = ?,
CONTENT_DSCR = ?,
SYSTEM_UPDATE_REASON = ?,
UPDATED_ON = SYSDATE
WHERE ID = ?
AND CONTENT_TITLE IS NULL
AND ? NOT BETWEEN 400 AND 599 OR ID = ?
AND CONTENT_TITLE = ''
AND ? NOT BETWEEN 400 AND 599
UPDATE URL
SET CONTENT_TITLE = ?,
CONTENT_DSCR = ?,
SYSTEM_UPDATE_REASON = ?,
UPDATED_ON = SYSDATE
WHERE ID = ?
AND CONTENT_DSCR IS NULL
AND ? NOT BETWEEN 400 AND 599 OR ID = ?
AND CONTENT_DSCR = ''
AND ? NOT BETWEEN 400 AND 599
UPDATE URL
SET CONTENT_TITLE = ?,
CONTENT_DSCR = ?,
SYSTEM_UPDATE_REASON = ?,
UPDATED_ON = SYSDATE
WHERE ID = ?
AND CONTENT_TITLE IS NOT NULL
AND ? BETWEEN 400 AND 599
All 3 statements are called during our scraping process... so you can imagine it's triple the calls which is not something we want. Can this be done in one single update?
Related
I have a MariaDB database in conjuction with an Express.js backend and therefore use the MariaDB provided Node.js Connector. I Initialized a Database with a Table that looks like this:
CREATE TABLE IF NOT EXISTS `Threads` (
`id` INTEGER UNSIGNED NOT NULL AUTO_INCREMENT,
`title` TINYTEXT NOT NULL,
`post` TEXT NOT NULL,
`created_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL,
`updated_at` DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP NOT NULL,
PRIMARY KEY (`id`)
);
Using the Node.js-Connector in my Backend, I want to insert some data to this table and use the default values for "created_at" and "updated_at".
I was thinking of something like this:
const insertThreadQuery = 'INSERT INTO Threads VALUES (?, ?, ?, ?, ?)';
con = await pool.getConnection();
const result = await con.query(insertThreadQuery, [null, "title", "post", null, null]);
Which obivously throws an error that tells me, that I cannot insert null for these Values ((conn=4, no: 1048, SQLState: 23000) Column 'updated_at' cannot be null sql: INSERT INTO Threads VALUES (?, ?, ?, ?, ?) - parameters:[null,'title','post',null,null])
My question is: How can I insert an entry like I was showing before, but instead of inserting "null" insert something else so my columns created_at and updated_at use the default value?
If you want to use DEFAULT values, then pass DEFAULT instead of NULL.
const insertThreadQuery = 'INSERT INTO Threads VALUES (NULL, ?, ?, DEFAULT, DEFAULT)';
con = await pool.getConnection();
const result = await con.query(insertThreadQuery, ["title", "post"]);
When I test REST API endpoints using POSTMAN, It works fine. This is the console message.
path : -1
****** doFilter!!!!!!!!!!!!!!!!!!!
2018-11-15 10:52:56,949 DEBUG [com.isu.ifm.wrapper.RequestWrapper::<init>:85] URI : /ifm/Board.do
2018-11-15 10:52:56,949 DEBUG [com.isu.ifm.wrapper.RequestWrapper::<init>:86] METHOD : GET
2018-11-15 10:52:56,950 DEBUG [com.isu.ifm.wrapper.RequestWrapper::<init>:126] PARAM : comments : [a]
enterCd : [ISU_PS]
bbsCd : [10003]
bbsSeq : [1]
cmd : [saveCmt]
sabun : [91006]
preHandler ::::::::::::::http://localhost:8083/ifm/Board.do
ISU_PS
91006
preHandler ::::::::::::::/ifm/Board.do
[2018-11-15 10:52:56] [DEBUG]『org.apache.commons.dbcp.PoolingDataSource:getConnection(106)』 Elapsed Time [0:00:00.007]
select
1
from
dual
[2018-11-15 10:52:56] [DEBUG]『org.springframework.jdbc.core.JdbcTemplate:execute(644)』 Elapsed Time [0:00:00.023]
INSERT INTO TSYS730(ENTER_CD, SABUN, BBS_SEQ, COMMENTS_SEQ, COMMENTS, CHKDATE, CHKID, BBS_CD)
VALUES( 'ISU_PS' /**PARAM*/, '91006' /**PARAM*/, '1' /**PARAM*/, BOARD_COMTSEQ.nextval, 'a' /**PARAM*/, sysdate, '91006' /**PARAM*/, '10003' /**PARAM*/)
Query Result[INSERT]: 1row
2018-11-15 10:52:56,984 DEBUG [com.isu.ifm.wrapper.ResponseWrapper::toByteArray:95] RETURN VALUE : {"message":"","map":{"comments":"a","enterCd":"ISU_PS","bbsCd":"10003","bbsSeq":"1","cmd":"saveCmt","sabun":"91006","ssnEnterCd":"ISU_PS","ssnSabun":"91006"},"code":1}
But when I try to test same endpoints using jUnit with MockMVC, it gets parameters but cannot map in query.
ERROR [org.anyframe.query.QueryService::processException:1889] Query Service : Fail to [update [query id = 'saveCmt'],
INSERT INTO TSYS730(ENTER_CD, SABUN, BBS_SEQ, COMMENTS_SEQ, COMMENTS, CHKDATE, CHKID, BBS_CD)
VALUES(:ssnEnterCd, :ssnSabun, :bbsSeq, BOARD_COMTSEQ.nextval, :comments, sysdate, :ssnSabun, :bbsCd)
, PreparedStatementCallback; SQL [
INSERT INTO TSYS730(ENTER_CD, SABUN, BBS_SEQ, COMMENTS_SEQ, COMMENTS, CHKDATE, CHKID, BBS_CD)
VALUES(?, ?, ?, BOARD_COMTSEQ.nextval, ?, sysdate, ?, ?)
]; ORA-01400: cannot insert NULL in ("EHR_ISU4"."TSYS730"."ENTER_CD")
; nested exception is java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL in ("EHR_ISU4"."TSYS730"."ENTER_CD")
].
Query = [org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [
INSERT INTO TSYS730(ENTER_CD, SABUN, BBS_SEQ, COMMENTS_SEQ, COMMENTS, CHKDATE, CHKID, BBS_CD)
VALUES(?, ?, ?, BOARD_COMTSEQ.nextval, ?, sysdate, ?, ?)
]; ORA-01400: cannot insert NULL in ("EHR_ISU4"."TSYS730"."ENTER_CD")
; nested exception is java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL in ("EHR_ISU4"."TSYS730"."ENTER_CD")
]
Reason = [{}].
MockHttpServletRequest:
HTTP Method = GET
Request URI = /Board.do
Parameters = {cmd=[saveCmt], enterCd=[ISU_PS], sabun=[91006], bbsCd=[10001], bbsSeq=[1], comments=[a]}
Headers = {}
Handler:
Type = com.isu.ifm.controller.BoardController
Method = public org.springframework.web.servlet.ModelAndView com.isu.ifm.controller.BoardController.saveCmt(javax.servlet.http.HttpSession,javax.servlet.http.HttpServletRequest,java.util.Map<java.lang.String, java.lang.Object>) throws java.lang.Exception
Async:
Async started = false
Async result = null
Resolved Exception:
Type = null
ModelAndView:
View name = jsonView
View = null
Attribute = code
value = -1
Attribute = message
value = fail
Attribute = map
value = {cmd=saveCmt, enterCd=ISU_PS, sabun=91006, bbsCd=10001, bbsSeq=1, comments=a, ssnEnterCd=null, ssnSabun=null}
FlashMap:
MockHttpServletResponse:
Status = 200
Error message = null
Headers = {}
Content type = null
Body =
Forwarded URL = jsonView
Redirected URL = null
Cookies = []
2018-11-15 10:49:19,788 INFO [org.springframework.context.support.GenericApplicationContext::doClose:862] Closing org.springframework.context.support.GenericApplicationContext#9660f4e: startup date [Thu Nov 15 10:49:14 KST 2018]; root of context hierarchy
Here is my testcase code.
public MockMvc mockMvc;
public void testGetMethod_param(String url, UnaryOperator<MockHttpServletRequestBuilder> operator)
throws Exception {
mockMvc.perform(operator.apply(get(url))).andDo(print()).andExpect(status().isOk());
}
#ContextConfiguration("file:src/main/webapp/WEB-INF/**/*.xml")
public class BoardControllerTest extends HttpRequestTestMethod implements CoreParameters, AdapterServiceParameters{
#Autowired
public BoardController boardController;
#Before
public void setUp() throws Exception {
mockMvc = MockMvcBuilders.standaloneSetup(boardController).build();
}
#Test
public void savecmt() throws Exception {
testGetMethod_param("/Board.do", builder->builder
.param("cmd", "saveCmt")
.param("enterCd", enterCd)
.param("sabun", sabun)
.param("bbsCd", bbsCd)
.param("bbsSeq", bbsSeq)
.param("comments", comments));
}
If I deliberately set enterCd to NULL, I get this following message.
[2018-11-15 11:06:02] [ERROR]『oracle.jdbc.driver.T4CTTIoer:processError(439)』
java.sql.SQLIntegrityConstraintViolationException
INSERT INTO TSYS730(ENTER_CD, SABUN, BBS_SEQ, COMMENTS_SEQ, COMMENTS, CHKDATE, CHKID, BBS_CD)
VALUES( null /**PARAM*/, '91006' /**PARAM*/, '1' /**PARAM*/, BOARD_COMTSEQ.nextval, 'a' /**PARAM*/, sysdate, '91006' /**PARAM*/, '10003' /**PARAM*/)
ORA-01400: cannot insert Null in ("EHR_ISU4"."TSYS730"."ENTER_CD")
But as attached above, when using jUnit, all parameters are set to "?"
Query = [org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [
INSERT INTO TSYS730(ENTER_CD, SABUN, BBS_SEQ, COMMENTS_SEQ, COMMENTS, CHKDATE, CHKID, BBS_CD)
VALUES(?, ?, ?, BOARD_COMTSEQ.nextval, ?, sysdate, ?, ?)
]; ORA-01400: cannot insert Null in ("EHR_ISU4"."TSYS730"."ENTER_CD")
; nested exception is java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert Null in ("EHR_ISU4"."TSYS730"."ENTER_CD")
]
Can anyone tell me what is wrong here?
Any help would be greatly appreciated.
I have a table that imports 221 rows from a database table. Whenever I add a row dynamically inside the HTML Table page, I want it to be able to pull the MAX ID, so in this case 221, and add 1 to it, therefore bringing the MR_ID to 222. I need this because each ID must be unique. Whenever I add a row into the table and look at it in the database, it displays as NULL. I have a little something for that, but it doesn't seem to be working. If you need any more code than what I posted, let me know and I will provide it. Thank you!
<?php
$MR_ID = $_POST['MR_ID'];
$MR_Name = $_POST['MR_Name'];
$Buyer_ID = $_POST['Buyer_ID'];
$MR_POC_N = $_POST['MR_POC_N'];
$MR_POC_E = $_POST['MR_POC_E'];
$MR_POC_P = $_POST['MR_POC_P'];
$host="xxxxxxx";
$dbName="xxxx";
$dbUser="xxxxxxxxxxx";
$dbPass="xxxxxxxxx";
$pdo = new PDO("sqlsrv:server=".$host.";Database=".$dbName, $dbUser, $dbPass);
$MR_ID = "Select MAX(MR_ID) + 1 FROM Stage_Rebate_Master";
$sql = "INSERT INTO Stage_Rebate_Master (MR_ID, MR_Name, Buyer_ID, MR_POC_N, MR_POC_E, MR_POC_P) VALUES (?, ?, ?, ?, ?, ?)";
$stmt = $pdo->prepare($sql);
$result = $stmt->execute(array($MR_ID, $MR_Name, $Buyer_ID, $MR_POC_N, $MR_POC_E, $MR_POC_P));
echo json_encode($result);
?>
you can try like this
$sql = "INSERT INTO Stage_Rebate_Master (MR_ID, MR_Name, Buyer_ID, MR_POC_N, MR_POC_E, MR_POC_P) SELECT ISNULL(MAX(MR_ID)+1,1), ?, ?, ?, ?, ? FROM Stage_Rebate_Master";
$stmt = $pdo->prepare($sql);
$result = $stmt->execute(array( $MR_Name, $Buyer_ID, $MR_POC_N, $MR_POC_E, $MR_POC_P));
My MySQL query won't work because of quotes, or missing quotes, but I don't understand how to use it properly. I need some explanations about this:
Perl script writing to .csv file:
open(ECRIRE,">$ARGV[1]") || die ("Impossible de creer le fichier de sortie");
foreach my $key (sort keys %caisse)
{
print ECRIRE "insert into etablissement(code_etablissement,nom, contact_ce_nom, contact_ce_tel, contact_ce_mail) values ($key,$caisse{$key}[0];$caisse{$key}[1];$caisse{$key}[2];$caisse{$key}[3]) on duplicate key update contact_ce_nom=$caisse{$key}[1],contact_ce_tel=$caisse{$key}[2],contact_ce_mail=$caisse{$key}[3];\n";
}
close(ECRIRE);
Bash script executing the SQL request:
$mysql -f -h $db_address -P $db_port -u $db_user -p$db_passwd $db_name < $vtiger_temporary_file_etablissement_clean
Mysql is crying over almost every informations like this one
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'migrationrh2cepal#mail.com) on duplicate key update contact_ce_no' at line 1
I even tried quoting every variable with single quotes, with the same results...
EDIT : Using DBI for perl
The code now looks like :
foreach my $key (sort keys %caisse)
{
my $insert = $sql_connection->prepare('insert into etablissement values(?, ?, ?, ?, ?, ?) on duplicate key update');
$insert->execute($key, $caisse{$key}[0], $caisse{$key}[1], $caisse{$key}[2], $caisse{$key}[3],'');
}
I now have the "on duplicate key" issue. How can I add the "on duplicate key" statement in here ?
I tried adding it at the end, just like this :
my $insert = $sql_connection->prepare('insert into etablissement values(?, ?, ?, ?, ?, ?) on duplicate key update');
But it's not working
Instead of quoting the variables yourself, use the DBI module and placeholders:
$db = 'DBI'->connect(...);
my $insert = $db->prepare('insert into etablissement values(?, ?, ?)');
$insert->execute($key, $caisse{$key}[0], $caisse{$key}[1]);
For repeated values, numbered placeholders are usually used:
my $insert = $db->prepare(<<'__SQL__');
INSERT INTO etablissement
(code_etablissement, nom, contact_ce_nom, contact_ce_tel, contact_ce_mail)
VALUES (:1, :2, :3, :4, :5)
ON DUPLICATE KEY UPDATE contact_ce_nom = :3,
contact_ce_tel = :4,
contact_ce_mail = :5'
__SQL__
$insert->execute($key, #{ $caisse{$key} }[0 .. 3]);
If your driver doesn't support them (I don't see them mentioned in DBD::mysql), you can workaround it e.g.
my $insert = $db->prepare(<<'__SQL__');
INSERT INTO etablissement
(code_etablissement, nom, contact_ce_nom, contact_ce_tel, contact_ce_mail)
VALUES (?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE contact_ce_nom = ?,
contact_ce_tel = ?,
contact_ce_mail = ?
__SQL__
$insert->execute($key, #{ $caisse{$key} }[0 .. 3], #{ $caisse{$key} }[1 .. 3]);
or you can play with quote and omit placeholders totally.
Using partial answer from choroba :
The final question is "how to use on duplicate key syntax with perl DBI ?"
Like this
my $insert = $sql_connection->prepare('insert into etablissement values(?, ?, ?, ?, ?, ?) on duplicate key update contact_ce_nom=?,contact_ce_tel=?,contact_ce_mail=?');
$insert->execute($key, $caisse{$key}[0], $caisse{$key}[1], $caisse{$key}[2], $caisse{$key}[3],'',$caisse{$key}[1],$caisse{$key}[2],$caisse{$key}[3]);
The last 3 ARGS are used by the "on udplicate key" syntax...
I have a query :
$q->andWhere($q->getRootAlias().'.is_published = ?', 1);
$q->andWhere($q->getRootAlias().'.published_at >= ?', time());
$q->leftJoin($q->getRootAlias().'.EventLdapGroup l');
$q->andWhereIn('l.ldap_group_id', $permissions_id );
$q->orWhere('l.ldap_group_id IS NULL);
which outputs :
FROM
Event r LEFT JOIN r.EventLdapGroup l
WHERE
r.is_published = ? AND
r.published_at >= ? AND
l.ldap_group_id IN (?, ?) OR
l.ldap_group_id IS NULL
The only problem is that if ldap_group_id is null (the last condition) it'll remove the is_published and published_at conditions.
I want something like that [either the values are in ldap_group_id, or either it's null] :
FROM
Event r LEFT JOIN r.EventLdapGroup l
WHERE
r.is_published = ? AND
r.published_at >= ? AND
(l.ldap_group_id IN (?, ?) OR l.ldap_group_id IS NULL)
And i must say that i'm lost with the complex where condition.
How to achieve that ?
Thanks
Instead of your last two lines
$q->andWhereIn('l.ldap_group_id', $permissions_id );
$q->orWhere('l.ldap_group_id IS NULL);
Try this approach
$q->andWhere($qb->expr()->orx(
$qb->expr()->in( 'l.ldap_group_id', $permissions_id ),
$qb->expr()->isNull( 'l.ldap_group_id' );
Ok solution is here :
$q->andWhere('(l.ldap_group_id IN ( ' . implode(",", $permissions_id) . ') OR l.ldap_group_id IS NULL )' );