I have a column that I'm looking to extract but am having issues! The column is stored as type ntext and contains an RTF document so looks something like this:
{\rtf1\ansi\ansicpg1252\uc1\deff0{\fonttbl {\f0\fnil\fcharset0\fprq2 Arial;} {\f1\fswiss\fcharset0\fprq2 Arial;} {\f2\froman\fcharset2\fprq2 Symbol;}} {\colortbl;\red0\green0\blue0;\red255\green255\blue255;} {\stylesheet{\s0\itap0\nowidctlpar\f0\fs24 [Normal];}{\*\cs10\additive Default Paragraph Font;}} {\*\generator TX_RTF32 15.0.530.502;} \deftab1134\paperw11909\paperh16834\margl1138\margt1138\margr1138\margb1138\widowctrl\formshade\sectd \headery720\footery720\pgwsxn11909\pghsxn16834\marglsxn1134\margtsxn1134\margrsxn1134\margbsxn1134\pard\itap0\nowidctlpar\plain\f1\fs20 Stephan Bos 28/11/2011 11:19:55\par\par Sold in guy. He likes him, feedback this afternoon.\par Will send him the CV and also our terms.\par Made him aware of our fees.\par }
But I'm looking to extract this back into (rtf or txt I don't really mind)
I've tried using BCP which has had success in extracting the documents but they end up exactly the same as the column but with spaces between each character rather than as I'd expect (example above would end up reading something like:
Stephan Bos 28/11/2011 11:19:55
Sold in guy. He likes him, feedback this afternoon.
Will send him the CV and also our terms.
Made him aware of our fees.
The BCP extract that I'm using (which is extracting) is as follows:
set nocount on;
Declare #sql varchar(1000);
declare #noteid int;
declare xx1 cursor for select nic.NotebookItemId from NotebookItemContent nic
inner join NotebookLinks nl on nl.NotebookItemId = nic.NotebookItemId
inner join NotebookItems ni on ni.NotebookItemId = nic.NotebookItemId
where nl.clientid = 1235074
AND ni.NotebookTypeId = 56;
open xx1;
fetch xx1 into #noteid;
while (##fetch_status = 0)
begin
set #sql = 'BCP "SELECT memo FROM Monarch_Pronet_ITOIL.dbo.notebookitemcontent where notebookitemid=' + cast(#noteid as varchar) +
'" QUERYOUT \\bhamws475\docs\' + cast(#noteid as varchar) + '.rtf -T -f \\bhamws475\docs\bcp.fmt -S ' + ##SERVERNAME
EXEC master.dbo.xp_CmdShell #sql
fetch xx1 into #noteid;
end;
close xx1;
deallocate xx1;
Can anyone point me in the right direction?
I think I understood now - the problem is that the RTF saved by BCP is not recognized as an RTF file by Word - it is opened as a plain text file.
This is due to the fact that the exported file is in Unicode (you see that by the fact that each character is followed by an empty space in the screen-shot).
The solution is to tell bcp to save not in Unicode - that I think can be done either with the -c switch or specifying the desired character set in the format file.
Related
I am wondering if there is a faster way to handle the following code in SQL. Currently I am using SQL cursor to do a select and build a string of delimiter values, as a dynamic value of suggestion items?
Here is the snippet of SQL:
begin
set #cursor = cursor for
select top 5 Manufacturer,ManufacturerPartNumber,Description as ManufacturerDescription, CONVERT(money,Price) as Price,fms.Score
from Products_OurProducts_Products_View
open #cursor
fetch next from #cursor
into #CURSOR_Mfr,#CURSOR_Model,#CURSOR_Desc,#CURSOR_Price,#CURSOR_Score
while ##FETCH_STATUS = 0
begin
set #suggestionsStringBuilder += #CURSOR_Mfr + ',' + #CURSOR_Model + ',' + #CURSOR_Desc + ',' + convert(varchar(20),#CURSOR_Price) + ',' + convert(varchar(4),#CURSOR_Score) + '^'
fetch next from #SuggestionsListCursor
into #CURSOR_Mfr,#CURSOR_Model,#CURSOR_Desc,#CURSOR_Price,#CURSOR_Score
end
insert into BASE (Manufacturer, ManufacturerOrig, ManufacturerPartNumber,ManufacturerPArtNumberOrig,ManufacturerDescription, QWDescription, Serial,AssetID,Price,Score,ItemType,MfrFound,ModelFound,trained, SuggestionList,LineNumberIn)
values(#objectORIGMfr,#objectORIGMfr, #objectORIGModel, #objectORIGModel, #objectDescription, #objectDescription, '',#objectAssetID,'0.00',#topMaxScore,'NA','1','0',#trained,#suggestionsStringBuilder,#objectLineNumber)
close #cursor
deallocate #cursor
end
The code above is trying to build a dynamic column of delimiter values such as shown below:
Object Example:
Mfr,
Model,
Price,
Score,
Description,
Suggestions = 'Mfr,Model,Desc,Price^Mfr,Model,Description,Price^
A return model would truly be as follows:
BaseMfr:Fluke,
BaseModel:Tb1,
BaseDescription:'Multi meter item',
BasePrice:120.00,
Suggestions: "Fluke, Tc1, 'Desc', '120.00' ^ 'Fluke', 'T11', 'Desc', 220.00"
Can I do the string builder / cursor section without having to use a looping cursor? The idea behind this is we send in items to be priced. If the item is not found, we then build a list of suggestions to bring back to the user of what they may use in the system or so they can see if there is a typo in the data.
The suggestion list is just the rows found, separating the columns by a "," and separating entities by a "^".
Thanks very much in advance!
Thanks you all for the feedback and I appreciate the help even though I know I had a rough time explaining the question correctly. Thanks to the suggestion from Sean Lange, I was able to be directed in the correct direction and came up with this. Now I will test the performance of it to see if it is better or not. Here is the code:
select
SUBSTRING(
(select top 5
Manufacturer + ',' + ManufacturerPartNumber + ',' + Description +',' + CONVERT(VARCHAR,Price) +',' + CONVERT(varchar,fms.Score) +'^' as [text()]
from Products_OurProducts_Products_View
CROSS APPLY (
select
dbo.FuzzyControlMatch('Flooke', Manufacturer) AS score
) AS fms
order by fms.score desc
FOR XML PATH ('')
), 2, 1000) [Suggestions]
The above code produces the following string:
ARD BROOKE,WB808 10UNF,TORQUE SCREWDRIVER,70.00,50^WARD BROOKE,WB808 1146,TORQUE SCREWDRIVER,70.00,50^WARD BROOKE,WB808 1246,TORQUE SCREWDRIVER,70.00,50^WARD BROOKE,WB808 6UNC,TORQUE SCREWDRIVER,70.00,50^ROKEM TECHNOLOGIES,FIRESET,RC STANDARD,105.00,50^
Now I am not sure if I am handling this the best way, but this is what I was searching for. I will post a comment update to let the feed know if the performance is better or worse.
-Thanks-
We take daily backups of all DB's. There isn't enough space on the server so we copy it to a different server and also a different disk based on what day it is. i.e
Day 1 goes to \\SERVER2\E:\SqlBackups\Day1
Day 2 goes to \\SERVER2\E:\SqlBackups\Day2
Day 3 goes to \\SERVER2\E:\SqlBackups\Day3
Day 4 goes to \\SERVER2\H:\SqlBackups\Day4
Day 5 goes to \\SERVER2\H:\SqlBackups\Day5
etc
I want to use dynamic SQL and CMD but its not working, it either can't find the path or says the credentials are wrong even though the account has access to the shared drive
I've now decided to try getting the CMD to call a .bat file to do the copy but i'm not really familiar with how to do this.
I can obviously create a simple copy statement, but i do not know how to include the parameters etc
DECLARE #SQL NVARCHAR(MAX)
DECLARE #Location nvarchar(200)
DECLARE #Day int
SET #Day = DATEPART(dw, getdate())
IF #Day in (1,2,3) SET #Location =
'\\Server2\BackUps\SqlBackups\'
IF #Day in (4,5,6,7) SET #Location =
'\\Server\BackUps2\SqlBackups2\'
SET #Location = #Location + 'Day'+ cast(#Day as nvarchar(1)) + '\'
SET #Location = #Location + 'DB1.bak'
SET #SQL = 'master..xp_cmdshell ''copy /Y
F:\SqlBackups\LatestToCopyToServer2\DB1.bak ' + #location +
''''
EXEC #SQL
The System cannot find the path specified.
if i just do a select the string looks good to me ...
master..xp_cmdshell 'copy /Y F:\SqlBackups\LatestToCopyToServer2\DB1.bak
\Server2\BackUps\SqlBackups\Day5\DB1.bak'
To clarify i need assistance with the batch file and what to include :)
After reading several articles about SQLRPGLE and retrieving data and storing them in data structure arrays, I came up with dynamic sql statements.
This works fine as long as I am using these dynamic to-replace fields for my where condition. But as soo as I am using these ? parameter in the select part, or in general as replacement for database fields, the result is blank.
Here is the DDS definition and the program:
TESTPF
A**************************************************************************
A*
A*-------------------------------------------------------------------------
A*
A R TESTPFR
A
A FLD01 2S 0
A FLD02 20A
A
A**************************************************************************
I have already filled this file with some dummy data. Here is what's inside:
runqry () qtemp/testpf
FLD01 FLD02
000001 1 Text 01
000002 2 Text 02
000003 3 Text 03
000004 4 Text 04
000005 5 Text 05
000006 6 Text 06
000007 7 Text 07
000008 8 Text 08
000009 9 Text 09
000010 10 Text 10
And this is the program:
TST001I
D**********************************************************************************************
D* Standalone Fields
D*---------------------------------------------------------------------------------------------
D stm s 500a inz(*blanks)
D fieldName01 s 10a inz(*blanks)
D fieldName02 s 10a inz(*blanks)
D fieldName03 s 2a inz(*blanks)
D text s 20a inz(*blanks)
D
C**********************************************************************************************
C* M A I N P R O G R A M M
C**********************************************************************************************
stm = 'SELECT fld02 FROM testpf WHERE fld01 = 1';
exec sql prepare s1 from :stm;
exec sql declare c1 cursor for s1;
exec sql open c1;
exec sql fetch c1 into :text;
exec sql close c1;
dsply text; // Prints 'Text 01'
text = *blanks;
stm = 'SELECT fld02 FROM testpf WHERE fld01 = ?';
exec sql prepare s2 from :stm;
exec sql declare c2 cursor for s2;
fieldName03 = '2';
exec sql open c2 using :fieldName03;
exec sql fetch c2 into :text;
exec sql close c2;
dsply text; // Prints 'Text 02'
text = *blanks;
stm = 'SELECT ? FROM testpf WHERE fld01 = 3';
exec sql prepare s3 from :stm;
exec sql declare c3 cursor for s3;
fieldName01 = 'FLD02';
exec sql open c3 using :fieldName01;
exec sql fetch c3 into :text;
exec sql close c3;
dsply text; // Prints ' '
text = *blanks;
stm = 'SELECT ? FROM testpf WHERE ? = ?';
exec sql prepare s4 from :stm;
exec sql declare c4 cursor for s4;
fieldName01 = 'FLD02';
fieldName02 = 'FLD01';
fieldName03 = '4';
exec sql open c4 using :fieldName01, :fieldName02, :fieldName03;
exec sql fetch c4 into :text;
exec sql close c4;
dsply text; // Prints ' '
text = *blanks;
*inlr = *on;
C**********************************************************************************************
This is the output:
DSPLY Text 01
DSPLY Text 02
DSPLY
DSPLY
DSPLY
May someone help me and explain why this is the case?
When using a prepared statement, you can use ? as a parameter marker wherever you can use a host variable in a static statement. Of your four sample prepared statements, the first 3 should work, though the third one will not return what you seem to expect as it is equivalent to:
SELECT 'FLD02' FROM testpf WHERE fld01 = 3
I would expect to receive the value 'FLD02' as the result, not the value in column FLD02. This is because the ? is not a string replacement marker, but a parameter field marker. You can't use it to select a column, but you can use it to provide a value for comparisons, or a constant to be output.
The fourth sample is valid SQL, but it is equivalent to:
SELECT 'FLD02' FROM testpf WHERE 'FLD01' = '4'
This will return nothing since 'FLD01' does not equal '4'.
Another consequence of this is that the ? can be used to provide a numeric value to the prepared statement. So you can do this:
dcl-s seqno Packed(5:0);
exec sql declare c2 cursor for s2;
stm = 'SELECT fld02 FROM testpf WHERE fld01 = ?';
exec sql prepare s2 from :stm;
seqno = 2;
exec sql open c2 using :seqno;
Also notice that I removed the declaration of the cursor to somewhere outside the logic flow as the declaration is not an executable statement. I see programs where the declare is in a subroutine that is called before a separate subroutine containing the open for the cursor. This is semantically incorrect. The DECLARE CURSOR statement is more correctly equivalent to an RPGLE dcl- statement. But because the SQL precompiler processes the source linearly, largely without regard to subroutines or sub-procedures, the requirement is for the DECLARE CURSOR to be physically before the OPEN in the source.
Generally I like to put my SQL Declares at the head of the program right after the SET OPTION statement which must be the first SQL embedded in the program. This is where I put the declares when I am using prepared statements. I also declare the statement name as well though this isn't strictly necessary. There is a little gotcha for this though, and that exists when using static SQL with locally scoped host variables. To deal with this, I declare static cursors a bit differently when using sub-procedures. The SQL precompiler recognizes that sub-procedures use locally scoped variables, so if you are declaring a static cursor with locally scoped host variables, the host variables and the cursor declaration must be in the same scope. That means I must declare my static cursors in the same sub-procedure as the open. I still declare the cursor up near the RPGLE dcl- statements to keep the declarations together.
I was wondering if anyone could help.
I have this piece of code in my stored procedure, but the Merge is not working for field Address. Works for the other fields.
I have a source file with all address to be filled to the Source.
The Source contains a field with all addresses. When I run the query the Target Address column fills only one row with the correct address. I can see that there was a Location change in source, the Location with the Address was updated for that one row, but not for the rest of the properties in the table.
DECLARE #SQLMerge nvarchar(4000)
SET #SQLMerge=' MERGE '+#RPTblName+' AS target'+
' USING '+#CMTblName+' AS source
ON target.VersionEndDate IS NULL AND target.PtyId = source.PtyId
WHEN MATCHED AND NOT (target.Location =source.Location AND target.Price=source.Price
AND target.PtyCode=source.PtyCode
AND target.[FloorSpace] = source.[FloorSpace]
AND target.Address = source.Address)
THEN UPDATE SET VersionEndDate ='''+CONVERT(NVARCHAR(50),#ReportMonthEND,121 )+''';'
IF #debug=1 print #SQLMerge
EXECUTE sp_executesql #SQLMerge
Try this:
DECLARE #SQLMerge nvarchar(4000)
SET #SQLMerge=' MERGE '+#RPTblName+' AS target'+
' USING '+#CMTblName+' AS source
ON target.VersionEndDate IS NULL AND target.PtyId=source.PtyId
WHEN MATCHED AND NOT (target.Location =source.Location AND target.Price=source.Price
AND target.PtyCode=source.PtyCode
AND target.[FloorSpace] = source.[FloorSpace]
AND target.Address = source.Address)
THEN UPDATE SET target.VersionEndDate ='''+CONVERT(NVARCHAR(50),#ReportMonthEND,121 )+''';'
IF #debug=1 print #SQLMerge
EXECUTE sp_executesql #SQLMerge
I've been handed a MS SQL 2000 database which has been injected with malware.
The malware script is as follows:
<script src=http://www.someAddress.ru/aScript.js></script>
Now I want to remove this piece of code from the table rows.
As a test, I inputed < h1> Test < /h1> on a row, and successfully ran the following query:
UPDATE myTable
SET description = REPLACE (description, '<h1>','')
WHERE id = 2;
This removed the h1 tag.
But trying the same with the script tag does not work:
UPDATE myTable
set description = REPLACE (description, '<script src=http://www.someAddress.ru/aScript.js></script>','')
WHERE id = 2
Why does this not work?
UPDATE 2
WOHO! I found the solution!
I'm using the folloing code, which I found here: http://www.tek-tips.com/viewthread.cfm?qid=1563568&page=3
-- Look for open and close HTML tags making sure a letter or / follows < ensuring its an opening
-- HTML tag or closing HTML tag and not an unencoded < symbol
CREATE FUNCTION [dbo].[udf_StripHTML]
(#HTMLText VARCHAR(8000))
RETURNS VARCHAR(8000)
AS
BEGIN
DECLARE #Start INT
DECLARE #End INT
DECLARE #Length INT
SET #Start = CHARINDEX('<',#HTMLText)
SET #End = CHARINDEX('>',#HTMLText,CHARINDEX('<',#HTMLText))
SET #Length = (#End - #Start) + 1
WHILE #Start > 0
AND #End > 0
AND #Length > 0
BEGIN
SET #HTMLText = STUFF(#HTMLText,#Start,#Length,'')
SET #Start = CHARINDEX('<',#HTMLText)
SET #End = CHARINDEX('>',#HTMLText,CHARINDEX('<',#HTMLText))
SET #Length = (#End - #Start) + 1
END
RETURN Replace(LTRIM(RTRIM(#HTMLText)),' ',' ')
END
GO
To remove the HTML tags / scripts, I run the following query:
UPDATE mytable
SET description = [dbo].[udf_StripHTML](description)
//WHERE id = 35;
This works perfectly. Note that this script removes ALL html. So if I only want to remove < script> , I just replace '<' with '< script'.
Have you tried looking for just aScript.js, the entry could be url_encoded, or something similar, so it gives something like
%3Cscript+src%3Dhttp%3A%2F%2Fwww.someAddress.ru%2FaScript.js%3E%3C%2Fscript%3E
Reread Question
Do you mean that even when you have the script tag in a column with id=2 it doesn't work? Because if its not working are you sure that it exists in row with id=2? :p
Should work, unless there are other hidden characters in there you can't see, or there is some form of encoding going on. Can you SELECT a suspect row to look at more closely.
I would tend to completely DELETE FROM myTable WHERE description LIKE '%someAddress.ru%' where possible.
However, fixing the database isn't a real solution; the application must be fixed. It shouldn't ever be echoing text out of the database unencoded. If someone enters some data including the string <script> it should simply appear on the page as the literal string <script>, or in the source <script>.
Wouldn't the src attribute value be surrounded by quotes? If so, you would have to escape them to get a proper match on the replace.
Why not try:
UPDATE myTable
set description = REPLACE (description, 'www.someAddress.ru','localhost')
WHERE id = 2
That would eliminate the immediate hijacking problem, and would likely avoid line break / funky characters problems.
You could try the following to strip the code out of your field (I'm assuming you have information in the same field that you want to keep):
update myTable
set description = case when PATINDEX('%<script%', notes) > 0
then SUBSTRING(notes, 1, PATINDEX('%<script%', notes)-1) + SUBSTRING(notes, PATINDEX('%script>%', notes) + 7, LEN(notes))
else notes
end
where id=2
You could first run a select to see if the value returned by the CASE statement is correct before running the update. It should not affect fields without a script tag in them, though.
Hold on...
Is the database related to a financial system? Is the application under Sarbanes-Oxley? Has any fraud been committed?
Any of those things preclude you from making changes that would, "destroy evidence." Those little guys running around with "FBI" on their jackets don't take kindly to that. It would be a good thing to back it up now, and the logs (SQL and Web), and put that backup on a few DVDs. It would be better to remove the disk and put in another one (but that may not be an option).
Moving on to cleansing:
bobince's direction is the correct one. Don't look for the whole SCRIPT tag, or try to find variations. Instead, look for something in the script tag that isn't part of the normal dataset. That's what you key off. If it SELECTs okay, then turn it into a DELETE and save that query, because you will need it while you turn to fixing the application (guaranteed your database will get corrupted again).