UTF8 changed to Latina 1 - Umlauts are not considered - sql

I am currently updating a table using the UPDATE command, whereby a text section is also read from another table using a substring. The command works fine so far. The only problem are the umlauts, which are not taken into account during the update. As I found out, for some reason the substring is rewritten into the format latina1, although the corresponding column of the table (action) has utf8 preset. Enclosed is the code for updating.
SQL:
update vms_vertrag_datei d
inner join vms_vertrag_verlauf v ON d.vertrag = v.vertrag
SET d.nutzer = v.nutzer, d.uploaddatum= v.timestamp
WHERE d.filename in (SELECT DISTINCT SUBSTRING(v.aktion,LOCATE('"',v.aktion)+1,(((LENGTH(v.aktion))-LOCATE('"', REVERSE(v.aktion))-1)-LOCATE('"',v.aktion)))FROM vms_vertrag_verlauf v)
AND v.aktion like 'Datei%hinzugefügt';
Does anyone know how I can now also consider text with umlauts? Am just after longer online research something at despair.

Related

Using JOIN in Query within MS Access 2016 for Fields in the Long Text Format

I have two queries which are almost identical. The only difference is the format of the fields being joined. One works, the other doesn't.
The query which JOINs two Integer fields works perfectly.
The query which JOINs two Long Text fields produces the following error:
"Cannot join on Memo, OLE, or Hyperlink Object (alarmlogwithstring2.[Tag_Value]=ECLString.[Tag_Value])."
Functional Query:
SELECT alarmlogwithdescs.TableIndex, alarmlogwithdescs.Date_Stamp, alarmlogwithdescs.Time_Stamp, alarmlogwithdescs.Tag_Name, alarmlogwithdescs.Tag_Value, ErrorCodeLookup.ErrorDescription
FROM ErrorCodeLookup INNER JOIN alarmlogwithdescs ON ErrorCodeLookup.[Tag_Value] = alarmlogwithdescs.[Tag_Value]
ORDER BY alarmlogwithdescs.TableIndex;
Nonfunctional Query:
SELECT alarmlogwithstring2.TableIndex, alarmlogwithstring2.Date_Stamp, alarmlogwithstring2.Time_Stamp, alarmlogwithstring2.Tag_Value, ECLString.ErrorDescription
FROM alarmlogwithstring2 INNER JOIN ECLString ON alarmlogwithstring2.[Tag_Value] = ECLString.[Tag_Value]
ORDER BY alarmlogwithstring2.TableIndex;
What I've Tried:
1.) I swapped the table following "FROM" to be ECLString with all necessary changes that should follow. (i.e. Then, after INNER JOIN I changed ECLString to be alarmlogwithstring2, etc...) This makes the two queries more identical, but shouldn't have an effect on the outcome. I did the same for the functional query just to be sure. The functional one still worked and the nonfunctional one still does not...
2.) I tried making my lookup table's Tag_Value field Short Text while keeping the actual data table's Tag_Value field Long Text. No effect.
3.) I tried changing the JOIN type when creating the relationship between the two tables. No effect.
4.) Changed alarmlogwithstring2.[Tag_Value]=ECLString.[Tag_Value]
to CAST(alarmlogwithstring2.[Tag_Value] AS varchar(max)) = CAST(ECLString.[Tag_Value] AS varchar(max)) and get the following error:
"Syntax error (missing operator) in query expression CAST(alarmlogwithstring2.[Tag_Value] AS varchar(max)) = CAST(ECLString.[Tag_Value] AS varchar(max))."
For whatever reason, after clicking "Ok" to close the error message the comma following SELECT alarmlogwithstring2.TableIndex, is highlighted, suggesting the missing operator is there. Okay?
Any help would be greatly appreciated. Thank you for your time!
Got it! Works for my situation, at least. Any other method for doing this would still be appreciated.
This works for me because my Tag_Value field contains text such as "Error0, Error1, Error2," etc...
So, I used the following code:
SELECT alarmlogwithstring2.TableIndex, alarmlogwithstring2.Date_Stamp, alarmlogwithstring2.Time_Stamp, alarmlogwithstring2.Tag_Value, ECLString.ErrorDescription
FROM alarmlogwithstring2 INNER JOIN ECLString ON Right( alarmlogwithstring2.[Tag_Value] , 1) = Right(ECLString.[Tag_Value], 1)
ORDER BY alarmlogwithstring2.TableIndex;
This works because of the integer on the end of my Tag_Value text. Using the Right(string,length) function causes only the integers within each value to be compared as they're all on the right-side of the value.
If your situation is similar to mine, then the code above is fine; however, if your number of error codes (or whatever) gets into the double digits, be sure to reflect this in the fields of both tables. (i.e. Make Error0 => Error00, make Error1 => Error01, etc...) within both tables and use Right(string,2) instead of Right(string,1). [Seems obvious, but may not be for everyone.]
However, this will NOT always be the case for me and everyone else. Someone may have pure text, for example. Thus, again, if you know of another, more general, solution, please, do let me know and I'll make your answer the answer for this question.
Thanks!

Got it. See below for general solution. It uses StrComp(string1,string2)=0 to match strings.
SELECT alarmlogwithstring2.TableIndex, alarmlogwithstring2.Date_Stamp, alarmlogwithstring2.Time_Stamp, alarmlogwithstring2.Tag_Name, alarmlogwithstring2.Tag_Value, ECLString.ErrorDescription
FROM alarmlogwithstring2 INNER JOIN ECLString ON StrComp(alarmlogwithstring2.[Tag_Value], ECLString.[Tag_Value]) = 0
ORDER BY alarmlogwithstring2.TableIndex;

Syntax error on WITH clause

I am working on a web app and there are some long winded stored procedures and just trying to figure something out, I have extracted this part of the stored proc, but cant get it to work. The guy who did this is creating alias after alias.. and I just want to get a section to work it out. Its complaining about the ending but all the curly brackets seem to match. Thanks in advance..
FInputs is another stored procedure.. the whole thing is referred to as BASE.. the result of this was being put in a temp table where its all referred to as U. I am trying to break it down into separate sections.
;WITH Base AS
(
SELECT
*
FROM F_Inputs(1,1,100021)
),
U AS
(
SELECT
ISNULL(q.CoverPK,r.CoverPK) AS CoverPK,
OneLine,
InputPK,
ISNULL(q.InputName,r.InputName) AS InputName,
InputOrdinal,
InputType,
ParentPK,
InputTriggerFK,
ISNULL(q.InputString,r.InputString) AS InputString,
PageNo,
r.RatePK,
RateName,
Rate,
Threshold,
ISNULL(q.Excess,r.Excess) AS Excess,
RateLabel,
RateTip,
Refer,
DivBy,
RateOrdinal,
RateBW,
ngRequired,
ISNULL(q.RateValue,r.RateValue) AS RateValue,
ngClass,
ngPattern,
UnitType,
TableChildren,
TableFirstColumn,
parentRatePK,
listRatePK,
NewParentBW,
NewChildBW,
ISNULL(q.SumInsured,0) AS SumInsured,
ISNULL(q.NoItems,0) AS NoItems,
DisplayBW,
ReturnBW,
StringBW,
r.lblSumInsured,
lblNumber,
SubRateHeading,
TrigSubHeadings,
ISNULL(q.RateTypeFK,r.RateTypeFK) AS RateTypeFK,
0 AS ListNo,
0 AS ListOrdinal,
InputSelectedPK,
InputVis,
CASE
WHEN ISNULL(NewChildBW,0) = 0
THEN 1
WHEN q.RatePK is NOT null
THEN 1
ELSE RateVis
END AS RateVis,
RateStatus,
DiscountFirstRate,
DiscountSubsequentRate,
CoverCalcFK,
TradeFilter,
ngDisabled,
RateGroup,
SectionNo
FROM BASE R
LEFT JOIN QuoteInputs Q
ON q.RatePK = r.RatePK
AND q.ListNo = 0
AND q.QuoteId = 100021 )

Well, I explained the issue in the comments section already. I'm doing it here again, so future readers find the answer more easily.
A WITH clause is part of a query. It creates a view on-the-fly, e.g.:
with toys as (select * from products where type = 'toys') select * from toys;
Without the query at the end, the statement is invalid (and would not make much sense anyhow; if one wanted a permanent view for later use, one would use CREATE VIEW instead).

PostgreSQL view won't work - column doesnt exist

Hi I am trying to migrate an access database into postgresql and everything was going well until i tried this view. I am wanting it to create a new column called 'CalculatedHours'. And as Im new to postgresql I am slightly confused. Heres the code that I keep putting into pgAdmin and getting the error...
SELECT "SessionsWithEnrolmentAndGroups"."SessionID",
"Assignments"."Staff",
"SessionsWithEnrolmentAndGroups"."groups",
"SessionsWithEnrolmentAndGroups"."SessionQty",
"SessionsWithEnrolmentAndGroups"."Hours",
"SessionsWithEnrolmentAndGroups"."Weeks",
"Assignments"."Percentage",
"Assignments"."AdditionalHours",
Round((coalesce(("groups"),1)*("SessionQty")*("Hours")*("Weeks")
*("Percentage"))) AS CalculatedHours,
(CalculatedHours)+coalesce(("AdditionalHours"),0) AS "TotalHours"
FROM "SessionsWithEnrolmentAndGroups"
INNER JOIN "Assignments"
ON "SessionsWithEnrolmentAndGroups"."SessionID" = "Assignments"."SessionID";

You cannot access column aliases in the same select where they are defined. I would suggest a subquery:
SELECT t.*,
(CalculatedHours)+coalesce(("AdditionalHours"), 0) AS "TotalHours"
FROM (SELECT eag."SessionID", a, eag."groups", eag."SessionQty",
eag."Hours", eag."Weeks", a."Percentage", a."AdditionalHours",
Round((coalesce(("groups"),1)*("SessionQty")*("Hours")*("Weeks")*("Percentage"))) AS CalculatedHours
FROM "SessionsWithEnrolmentAndGroups" eag INNER JOIN
"Assignments" a
ON eag."SessionID" = a."SessionID"
) t;
Your queries would also be much more readable using table aliases and getting rid of the escape characters (double quotes) unless they are really, really needed.

looking for db2 text function or method I can do a text contain rather than like

I'm looking for a db2 function that does a text contain search. At present I am running the following query against the data below....
SELECT distinct
s.search_id,
s.search_heading,
s.search_url
FROM repman.search s, repman.search_tags st
WHERE s.search_id = st.search_id
AND ( UPPER(s.search_heading) LIKE (cast('%REPORT%' AS VARGRAPHIC(32)))
OR (UPPER(st.search_tag) LIKE cast('%REPORT%' AS VARGRAPHIC(32)))
)
ORDER BY s.search_heading;
Which returns...
But if I change the search text to %REPORTS% rather than %REPORT% (which I need to do) the like search does not work and I get zero results.
I read a link that used a function named CONTAINS like below but when trying to use the function I get an error.
SELECT distinct
s.search_id,
s.search_heading,
s.search_url
FROM repman.search s, repman.search_tags st
WHERE s.search_id = st.search_id
AND CONTAINS(s.search_heading, 'REPORTS') = 1
Has anynoe got any suggestions? I'm on db2 version DB2/LINUXPPC 9.1.6.
Thanks

In order to look for a pattern in a string, you can use Regular Expressions. They are built-in DB2 with xQuery since DB2 v9. There are also other ways to do that. I wrote an article in my blog (in Spanish that you can translate) about Regular Expressions in DB2.
xmlcast(xmlquery('fn:matches(\$TEXT,''^[A-Za-z 0-9]*$'')')

What is the best way to run N independent column updates in PostgreSQL? What is the best way to do it in the SQL spec?

I'm looking for a more efficient way to run many columns updates on the same table like this:
UPDATE TABLE table
SET col = regexp_replace( col, 'foo', 'bar' )
WHERE regexp_match( col, 'foo' );
Such that foo, and bar, will be a combination of 40 different regex-replaces. I doubt even 25% of the dataset needs to be updated at all, but what I'm wanting to know is it is possible to cleanly achieve the following in SQL.
A single pass update
A single match of the regex, triggers a single replace
Not running all possible regexp_replaces if only one matches
Not updating all columns if only one needs the update
Not updating a row if no column has changed
I'm also curious, I know in MySQL (bear with me)
UPDATE foo SET bar = 'baz'
Has an implicit WHERE bar != 'baz' clause
However, in PostgreSQL I know this doesn't exist: I think I could at least answer one of my questions if I knew how to skip a single row's update if the target columns weren't updated.
Something like
UPDATE TABLE table
SET col = *temp_var* = regexp_replace( col, 'foo', 'bar' )
WHERE col != *temp_var*

Do it in code. Open up a cursor, then: grab a row, run it through the 40 regular expressions, and if it changed, save it back. Repeat until the cursor doesn't give you any more rows.
Whether you do it that way or come up with the magical SQL expression, it's still going to be a row scan of the entire table, but the code will be much simpler.
Experimental Results
In response to criticism, I ran an experiment. I inserted 10,000 lines from a documentation file into a table with a serial primary key and a varchar column. Then I tested two ways to do the update. Method 1:
in a transaction:
opened up a cursor (select for update)
while reading 100 rows from the cursor returns any rows:
for each row:
for each regular expression:
do the gsub on the text column
update the row
This takes 1.16 seconds with a locally connected database.
Then the "big replace," a single mega-regex update:
update foo set t =
regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(regexp_replace(t,
E'\bcommit\b', E'COMMIT'),
E'\b9acf10762b5f3d3b1b33ea07792a936a25e45010\b',
E'9ACF10762B5F3D3B1B33EA07792A936A25E45010'),
E'\bAuthor:\b', E'AUTHOR:'),
E'\bCarl\b', E'CARL'), E'\bWorth\b',
E'WORTH'), E'\b\b',
E''), E'\bDate:\b',
E'DATE:'), E'\bMon\b', E'MON'),
E'\bOct\b', E'OCT'), E'\b26\b',
E'26'), E'\b04:53:13\b', E'04:53:13'),
E'\b2009\b', E'2009'), E'\b-0700\b',
E'-0700'), E'\bUpdate\b', E'UPDATE'),
E'\bversion\b', E'VERSION'),
E'\bto\b', E'TO'), E'\b2.9.1\b',
E'2.9.1'), E'\bcommit\b', E'COMMIT'),
E'\b61c89e56f361fa860f18985137d6bf53f48c16ac\b',
E'61C89E56F361FA860F18985137D6BF53F48C16AC'),
E'\bAuthor:\b', E'AUTHOR:'),
E'\bCarl\b', E'CARL'), E'\bWorth\b',
E'WORTH'), E'\b\b',
E''), E'\bDate:\b',
E'DATE:'), E'\bMon\b', E'MON'),
E'\bOct\b', E'OCT'), E'\b26\b',
E'26'), E'\b04:51:58\b', E'04:51:58'),
E'\b2009\b', E'2009'), E'\b-0700\b',
E'-0700'), E'\bNEWS:\b', E'NEWS:'),
E'\bAdd\b', E'ADD'), E'\bnotes\b',
E'NOTES'), E'\bfor\b', E'FOR'),
E'\bthe\b', E'THE'), E'\b2.9.1\b',
E'2.9.1'), E'\brelease.\b',
E'RELEASE.'), E'\bThanks\b',
E'THANKS'), E'\bto\b', E'TO'),
E'\beveryone\b', E'EVERYONE'),
E'\bfor\b', E'FOR')
The mega-regex update takes 0.94 seconds to update.
At 0.94 seconds compared to 1.16, it's true that the mega-regex update is faster, running in 81% of the time of doing it in code. It is not, however a lot faster. And ye Gods, look at that update statement. Do you want to write that, or try to figure out what went wrong when Postgres complains that you dropped a parenthesis somewhere?
Code
The code used was:
def stupid_regex_replace
sql = Select.new
sql.select('id')
sql.select('t')
sql.for_update
sql.from(TABLE_NAME)
Cursor.new('foo', sql, {}, #db) do |cursor|
until (rows = cursor.fetch(100)).empty?
for row in rows
for regex, replacement in regexes
row['t'] = row['t'].gsub(regex, replacement)
end
end
sql = Update.new(TABLE_NAME, #db)
sql.set('t', row['t'])
sql.where(['id = %s', row['id']])
sql.exec
end
end
end
I generated the regular expressions dynamically by taking words from the file; for each word "foo", its regular expression was "\bfoo\b" and its replacement string was "FOO" (the word uppercased). I used words from the file to make sure that replacements did happen. I made the test program spit out the regex's so you can see them. Each pair is a regex and the corresponding replacement string:
[[/\bcommit\b/, "COMMIT"],
[/\b9acf10762b5f3d3b1b33ea07792a936a25e45010\b/,
"9ACF10762B5F3D3B1B33EA07792A936A25E45010"],
[/\bAuthor:\b/, "AUTHOR:"],
[/\bCarl\b/, "CARL"],
[/\bWorth\b/, "WORTH"],
[/\b<cworth#cworth.org>\b/, "<CWORTH#CWORTH.ORG>"],
[/\bDate:\b/, "DATE:"],
[/\bMon\b/, "MON"],
[/\bOct\b/, "OCT"],
[/\b26\b/, "26"],
[/\b04:53:13\b/, "04:53:13"],
[/\b2009\b/, "2009"],
[/\b-0700\b/, "-0700"],
[/\bUpdate\b/, "UPDATE"],
[/\bversion\b/, "VERSION"],
[/\bto\b/, "TO"],
[/\b2.9.1\b/, "2.9.1"],
[/\bcommit\b/, "COMMIT"],
[/\b61c89e56f361fa860f18985137d6bf53f48c16ac\b/,
"61C89E56F361FA860F18985137D6BF53F48C16AC"],
[/\bAuthor:\b/, "AUTHOR:"],
[/\bCarl\b/, "CARL"],
[/\bWorth\b/, "WORTH"],
[/\b<cworth#cworth.org>\b/, "<CWORTH#CWORTH.ORG>"],
[/\bDate:\b/, "DATE:"],
[/\bMon\b/, "MON"],
[/\bOct\b/, "OCT"],
[/\b26\b/, "26"],
[/\b04:51:58\b/, "04:51:58"],
[/\b2009\b/, "2009"],
[/\b-0700\b/, "-0700"],
[/\bNEWS:\b/, "NEWS:"],
[/\bAdd\b/, "ADD"],
[/\bnotes\b/, "NOTES"],
[/\bfor\b/, "FOR"],
[/\bthe\b/, "THE"],
[/\b2.9.1\b/, "2.9.1"],
[/\brelease.\b/, "RELEASE."],
[/\bThanks\b/, "THANKS"],
[/\bto\b/, "TO"],
[/\beveryone\b/, "EVERYONE"],
[/\bfor\b/, "FOR"]]
If this were a hand-generated list of regex's, and not automatically generated, my question is still appropriate: Which would you rather have to create or maintain?

For the skip update, look at suppress_redundant_updates - see http://www.postgresql.org/docs/8.4/static/functions-trigger.html.
This is not necessarily a win - but it might well be in your case.
Or perhaps you can just add that implicit check as an explicit one?