Pl Sql Decode and Unzip - sql

I have the below Java script which works but runs into issues at times. I need to try and write it as Pl Sql which I can then work with inside the application. I can get to the point of base64_decode but I can not get UTL_COMPRESS.LZ_UNCOMPRESS to work. Any ideas, suggestions, please?
Bellow is the working(ish) java code i need to emulate in Pl SQL
import java.util.*;
import java.util.zip.*;
import java.io.*;
import java.util.Arrays;
import org.apache.commons.codec.binary.Base64;
//import org.apache.bob;
public class decompress {
public static void main(String[] args) {
//String zpl = "rVrbbhw5kn33V+hlgV1sWg7mjVk9wADMqmRnVkmyWlK37cGiAE2PVnb3WDbc02uMH/phsT8x
this is a very long string staored as a clob in the data base .
6kwF4z6vyGzQYdMKVlUUj1lsWrJJlABbPLdfYvz7xzZVbzZv/7DPwE=";
try{
String zpl = args[0];
if (zpl.length()<2) {
System.out.println("ERROR");
} else {
//System.out.println(zpl);
//byte[] res = decompress(zpl.getBytes("UTF-8"));
Base64 b64 = new Base64();
byte[] res=Base64.decodeBase64(zpl);
//System.out.println(new String(res, "UTF-8"));
byte[] decom=decompress(res);
System.out.println(new String(decom,"UTF-8"));
}
//System.out.println("THIS " + res.toString());
} catch (Exception e) {
//System.out.println(e.getMessage());
//e.printStackTrace();
System.out.println("ERROR");
}
}
public static byte[] decompress(byte[] data) throws IOException, DataFormatException {
Inflater inflater = new Inflater(true);
inflater.setInput(data);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
while (!inflater.finished()) {
int count = inflater.inflate(buffer);
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
inflater.end();
return output;
}
}
SQL Function attempt which seam to do teh base 64 bit but falls apart when trying to decompress.
CREATE OR REPLACE
FUNCTION decode_base64_x(
p_clob_in IN CLOB)
RETURN BLOB
IS
v_blob BLOB ;
v_result BLOB ;
v_offset INTEGER;
v_buffer_size binary_integer := 48;
v_buffer_varchar VARCHAR2(48);
v_buffer_raw raw(48);
l_uncompressed_blob BLOB ;
l_in_blob BLOB;
BEGIN
IF p_clob_in IS NULL THEN
RETURN NULL;
END IF;
dbms_lob.createtemporary(v_blob, true);
v_offset := 1;
FOR i IN 1 .. ceil(dbms_lob.getlength(p_clob_in) / v_buffer_size)
LOOP
dbms_lob.read(p_clob_in, v_buffer_size, v_offset, v_buffer_varchar);
v_buffer_raw := utl_raw.cast_to_raw(v_buffer_varchar);
v_buffer_raw := utl_encode.base64_decode(v_buffer_raw);
dbms_lob.writeappend(v_blob, utl_raw.length(v_buffer_raw), v_buffer_raw);
v_offset := v_offset + v_buffer_size;
END LOOP;
v_result := v_blob;
--
--
-- l_in_blob := (UTL_RAW.CAST_TO_RAW (v_blob) );
--
UTL_COMPRESS.LZ_UNCOMPRESS( src => v_blob, dst => l_uncompressed_blob);
--
dbms_lob.freetemporary(v_blob);
-- l_uncompressed_blob := UTL_COMPRESS.LZ_UNCOMPRESS( src => v_blob ) ;
-- UTL_COMPRESS.lz_uncompress (src => v_blob , dst => l_uncompressed_blob);
-- RETURN l_uncompressed_blob;
RETURN v_result;
END decode_base64_x;
i think that its just the UTL_COMPRESS.LZ_UNCOMPRESS bit i am getting wrong !

Related

SQL stored procedure to convert a blob field to XML

I have an xml file stored in the DB. this xml file is converted to a zipped byte[] and then stored in the db as shown below:
ByteArrayOutputStream byteOut = null;
ZipOutputStream out = null;
try {
byteOut = new ByteArrayOutputStream();
out = new ZipOutputStream(byteOut);
out.setLevel(level);
ZipInput entry = new ZipStringInput("string", xmlString));
out.putNextEntry(getZipEntry(encrypt, entry.getEntryName()));
InputStream in = entry.getInputStream();
byte[] buffer = new byte[BUFFER];
for (int length; (length = in.read(buffer)) != -1; ) {
out.write(buffer, 0, length);
}
out.finish();
saveToDB(byteOut.toByteArray()
In java I can easily fetch it like shown below (simplified wihtout catch, close etc.,):
byte[] xmlBytes = (byte[])blob;
ByteArrayOutputStream out = new ByteArrayOutputStream();
try (ByteArrayInputStream byteIn = new ByteArrayInputStream(xmlBytes );) {
try (ZipInputStream zipIn = new ZipInputStream(byteIn);) {
for (ZipEntry entry; (entry = zipIn.getNextEntry()) != null;) {
byte[] buffer = new byte[32768];
for (int length; (length = zipIn.read(buffer, 0, buffer.length)) != -1;) {
out.write(buffer, 0, length);
}
}
return out.toString("UTF-8"));
}
Oracle DB: Now I have to task to do the same in a SQL stored procedure. how do I do it. I have tried combinations of lz_uncompress, lob.substr etc., I can see the bytes, but not sure how to convert it to the XML file.
CREATE OR REPLACE PROCEDURE print_clob AS
v_clob CLOB;
v_varchar VARCHAR2(32767);
v_start PLS_INTEGER := 1;
v_buffer PLS_INTEGER := 32767;
blob_in BLOB;
x XMLTYPE;
BEGIN
dbms_lob.createtemporary(v_clob, true);
FOR i IN (
SELECT
blobdata AS blob_in
FROM
myTable
WHERE
id = 123
) LOOP
FOR j IN 1..ceil(dbms_lob.getlength(i.blob_in) / v_buffer) LOOP
v_varchar := dbms_lob.substr(i.blob_in, v_buffer, v_start);
dbms_lob.writeappend(v_clob, length(v_varchar), v_varchar);
v_start := v_start + v_buffer;
END LOOP;
dbms_output.put_line(v_clob ); prints some binary stirng looking like '504B0304140008000800E0843E4200000000000000000.....'
-- dbms_output.put_line(UTL_RAW.CAST_TO_NVARCHAR2(v_clob )); -> throws error
-- dbms_output.put_line(utl_compress.lz_uncompress(v_clob )); -> throws error
--x := xmltype.createxml(v_clob); -> Throws error that the input it not an XML <, but some randoms tring
-- dbms_output.put_line(x.getclobval());
END LOOP;
END;
You are trying to substring a BLOB and stick it into a CLOB without any conversion. If the BLOB is compressed, you'll need to uncompress it first with utl_compress.lz_uncompress. Then once uncompressed, you need to convert it to a CLOB using dbms_lob.converttoclob.

How to connect to the CDP WebSocket server in Golang?

I want to send CDP commands to WebSocket server exposed by chrome browser.
For example, I create a new session on selenium and get WebSocket url as -> ws://172.20.10.2:4444/session/335d5805e9d98f3c37af004fa91e0f6e/se/cdp
Now I want to send CDP commands to this WebSocket connection and get results.
Sample client code :
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/websocket"
"log"
"net/url"
)
type message1 struct {
Id int `json:"id"`
Result Result `json:"result"`
}
type Result struct {
TargetInfo []TargetInfo `json:"targetInfos"`
}
type TargetInfo struct {
TargetId string `json:"targetId"`
}
type message2 struct {
Id int `json:"id"`
Method string `json:"method"`
Params Parameter `json:"params"`
}
type Parameter struct {
TargetId string `json:"targetId"`
Flatten bool `json:"flatten"`
}
type message3 struct {
Id int `json:"id"`
Method string `json:"method"`
SessionId string `json:"sessionId"`
Parameters EmulationParameter `json:"params"`
}
type EmulationParameter struct {
Width int `json:"width"`
Height int `json:"height"`
DeviceScaleFactor int `json:"deviceScaleFactor"`
Mobile bool `json:"mobile"`
}
type Response2 struct {
Method string `json:"method"`
Parameter SessionResponse `json:"params"`
}
type SessionResponse struct {
SessionId string `json:"sessionId"`
}
type message4 struct {
Id int `json:"id"`
Method string `json:"method"`
SessionId string `json:"sessionId"`
Parameters ScreenshotParameter `json:"params"`
}
type ScreenshotParameter struct {
Width int `json:"width,omitempty"`
}
func main() {
hostURL := "172.20.10.2:4444"
pathURL := "/session/dd44b246de3261c405cb9fcb017ab59a/se/cdp"
u := url.URL{Scheme: "ws", Host: hostURL, Path: pathURL}
c, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
if err != nil {
log.Fatal("dial:", err)
}
defer c.Close()
messageChannel := make(chan string)
go func() {
for {
_, message, err := c.ReadMessage()
//err = c.ReadJSON(p)
if err != nil {
log.Println("read:", err)
return
}
messageChannel <- string(message)
fmt.Println(string(message))
}
}()
message := "{\"id\":1,\"method\":\"Target.getTargets\"}"
//err = c.WriteJSON(websocket.TextMessage, []byte(message))
err = c.WriteMessage(websocket.TextMessage, []byte(message))
if err != nil {
log.Println("write:", err)
return
}
response := <-messageChannel
fmt.Println(response)
decodedResponse := message1{}
json.Unmarshal([]byte(response), &decodedResponse)
fmt.Println(decodedResponse)
fmt.Println(decodedResponse.Result.TargetInfo[0].TargetId)
temp := message2{}
temp.Id = 2
temp.Method = "Target.attachToTarget"
temp.Params.Flatten = true
temp.Params.TargetId = decodedResponse.Result.TargetInfo[0].TargetId
event2, _ := json.Marshal(temp)
err = c.WriteMessage(websocket.TextMessage, event2)
if err != nil {
log.Println("write:", err)
return
}
response2 := <-messageChannel
fmt.Println(response2)
decodedResponse2 := Response2{}
json.Unmarshal([]byte(response2), &decodedResponse2)
fmt.Println("The session Id is")
fmt.Println(decodedResponse2.Parameter.SessionId)
width := 645
height := 9651
scale := 2
temp2 := message3{}
temp2.Id = 1
temp2.SessionId = decodedResponse2.Parameter.SessionId
temp2.Method = "Emulation.setDeviceMetricsOverride"
temp2.Parameters.DeviceScaleFactor = scale
temp2.Parameters.Height = height
temp2.Parameters.Width = width
temp2.Parameters.Mobile = false
event3, _ := json.Marshal(temp2)
err = c.WriteMessage(websocket.TextMessage, event3)
if err != nil {
log.Println("write:", err)
return
}
response3 := <-messageChannel
fmt.Println(response3)
temp3 := message4{}
temp3.Id = 1
temp3.SessionId = decodedResponse2.Parameter.SessionId
temp3.Method = "Page.captureScreenshot"
event4, _ := json.Marshal(temp3)
err = c.WriteMessage(websocket.TextMessage, event4)
if err != nil {
log.Println("write:", err)
return
}
response4 := <-messageChannel
fmt.Println(response4)
}
I wasn't able to reuse this part of the code to send CDP commands to the WebSocket. I want a simple example CDP command.

Indy 10 SSL root certificate

I am trying to verify server certificate. I use Indy 10 and OpenSSL. I specify Root RootCertFile and VerifyDepth to MaxInt. OnVerifyPeer works fine - AOk is true. I wonder how to load certificates from Windows Trusted Root Certification Authorities. There is my stripped code of a client:
uses
{Delphi}
IdSSLOpenSSL
, IdHTTP
, IdHeaderList
, System.Classes
{Project}
;
type
TUnicodeHTTPPoster = class
strict private
FidHTTP: TIdHTTP;
FLastError: string;
FCertPassword: string;
procedure OnGetPassword(var Password: string);
function OnVerifySSLPeer(Certificate: TIdX509;AOk: Boolean; ADepth, AError: Integer): Boolean;
public
constructor Create(const ASSLVersion: TIdSSLVersion; const AAccept: string = 'application/xml';
const ACharSet: string = 'utf-8'; const ACertFile: string = ''; const AKeyFile: string = '';
const ACertPassword: string = ''); reintroduce;
destructor Destroy; override;
function Post(const ACustomHeaders: TIdHeaderList; const ARawBody: TStream;
const AURL: string; out AResponse: string): integer;
end;
implementation
uses
{Delphi}
System.SysUtils
, IdURI
, IdGlobal
{Project}
;
constructor TUnicodeHTTPPoster.Create(const ASSLVersion: TIdSSLVersion; const AAccept: string = 'application/xml';
const ACharSet: string = 'utf-8'; const ACertFile: string = ''; const AKeyFile: string = '';
const ACertPassword: string = '');
var
_IdSSLIOHandlerSocketOpenSSL: TIdSSLIOHandlerSocketOpenSSL;
begin
inherited Create;
FidHTTP := TIdHTTP.Create(nil);
FidHTTP.Request.Accept := 'application/xml';
if AAccept <> '' then
FidHTTP.Request.Accept := AAccept;
FidHTTP.Request.Charset := 'utf-8';
if ACharSet <> '' then
FidHTTP.Request.Charset := ACharSet;
_IdSSLIOHandlerSocketOpenSSL := TIdSSLIOHandlerSocketOpenSSL.Create(FidHTTP);
if FileExists(ACertFile) then
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.CertFile := ACertFile;
if FileExists(AKeyFile) then
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.KeyFile := AKeyFile;
FCertPassword := ACertPassword;
FidHTTP.Request.BasicAuthentication := False;
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.Mode := sslmClient;
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.Method := ASSLVersion;
_IdSSLIOHandlerSocketOpenSSL.OnGetPassword := OnGetPassword;
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.VerifyMode := [sslvrfPeer];
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.VerifyDepth := MaxInt;
_IdSSLIOHandlerSocketOpenSSL.OnVerifyPeer := OnVerifySSLPeer;
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.RootCertFile := 'C:\Users\ekolesnikovics\Desktop\Projects\nDentity\ndentify\Build\dc_ofisas.nsoft.lt.pem';
FidHTTP.IOHandler := _IdSSLIOHandlerSocketOpenSSL;
end;
function TUnicodeHTTPPoster.OnVerifySSLPeer(Certificate: TIdX509;AOk: Boolean; ADepth, AError: Integer): Boolean;
begin
Result := AOk;
end;
procedure TUnicodeHTTPPoster.OnGetPassword(var Password: string);
begin
Password := FCertPassword;
end;
function TUnicodeHTTPPoster.Post(const ACustomHeaders: TIdHeaderList; const ARawBody: TStream;
const AURL: string; out AResponse: string): integer;
var
_URL: string;
_ResponseStream: TStringStream;
begin
Result := 500;
FLastError := '';
try
if Trim(AURL) = '' then
raise EArgumentException.Create('URL is not provided.');
_URL := TIdURI.URLEncode(AURL, IndyTextEncoding_UTF8);
_ResponseStream := TStringStream.Create('', TEncoding.UTF8);
try
if Assigned(FidHTTP.Request.CustomHeaders) then
FidHTTP.Request.CustomHeaders.Clear;
if Assigned(ACustomHeaders) then
FidHTTP.Request.CustomHeaders := ACustomHeaders;
FidHTTP.Post(_URL, ARawBody, _ResponseStream);
_ResponseStream.Position := 0;
AResponse := _ResponseStream.DataString;
finally
FreeAndNil(_ResponseStream);
end;
Result := 200;
except
on E: EIdHTTPProtocolException do
begin
Result := E.ErrorCode;
FLastError := E.ErrorMessage;
FidHTTP.Disconnect;
end;
on E: Exception do
begin
FLastError := E.Message;
FidHTTP.Disconnect;
end;
end;
end;
I ended up using free https://github.com/magicxor/WinCryptographyAPIs
procedure TUnicodeHTTPPoster.ExportWindowsCertificateStoreToFile(const ACertFile: string);
var
_hStore: HCERTSTORE;
_CertContext: PCertContext;
_pchString: Cardinal;
_szString: string;
_CertList: TStringList;
begin
_hStore := CertOpenSystemStore(0, PChar('ROOT'));
if (_hStore = nil) then
RaiseLastOSError;
_CertList := TStringList.Create;
try
_CertContext := CertEnumCertificatesInStore(_hStore, nil);
if (_CertContext = nil) then
RaiseLastOSError;
while _CertContext <> nil do
begin
_pchString := 0;
if not CryptBinaryToString(_CertContext.pbCertEncoded,
_CertContext.cbCertEncoded, CRYPT_STRING_BASE64, nil, _pchString) then
RaiseLastOSError;
SetLength(_szString, 0);
SetLength(_szString, _pchString - 1);
if not CryptBinaryToString(_CertContext.pbCertEncoded,
_CertContext.cbCertEncoded, CRYPT_STRING_BASE64, PWideChar(_szString),
_pchString) then
RaiseLastOSError;
_CertList.Add('-----BEGIN CERTIFICATE-----');
_CertList.Add(Trim(StrPas(PWideChar(_szString))));
_CertList.Add('-----END CERTIFICATE-----');
_CertContext := CertEnumCertificatesInStore(_hStore, _CertContext);
end;
_CertList.SaveToFile(ACertFile);
finally
FreeAndNil(_CertList);
CertCloseStore(_hStore, 0);
end;
end;
_RootCertFileName := TPath.Combine(ExtractFilePath(ParamStr(0)), 'windows_cert.pem');
ExportWindowsCertificateStoreToFile(_RootCertFileName);
_IdSSLIOHandlerSocketOpenSSL.SSLOptions.RootCertFile := _RootCertFileName;

Generate N-grams while preserving spaces in apache lucene

I am trying to generate N-grams using apache Lucene 5.5.4 for a given set input text. Following is my java code to do the same.
public static void main( String[] args )
{
Analyzer analyzer = createAnalyzer( 2 );
List<String> nGrams = generateNgrams( analyzer, "blah1 blah2 blah3" );
for ( String nGram : nGrams ) {
System.out.println( nGram );
}
}
public static Analyzer createAnalyzer( final int shingles )
{
return new Analyzer() {
#Override
protected TokenStreamComponents createComponents( #NotNull String field )
{
final Tokenizer source = new WhitespaceTokenizer();
final ShingleFilter shingleFilter = new ShingleFilter( new LowerCaseFilter( source ), shingles );
shingleFilter.setOutputUnigrams( true );
return new TokenStreamComponents( source, shingleFilter );
}
};
}
public static List<String> generateNgrams( Analyzer analyzer, String str )
{
List<String> result = new ArrayList<>();
try {
TokenStream stream = analyzer.tokenStream( null, new StringReader( str ) );
stream.reset();
while ( stream.incrementToken() ) {
String nGram = stream.getAttribute( CharTermAttribute.class ).toString();
result.add( nGram );
LOG.debug( "Generated N-gram = {}", nGram );
}
} catch ( IOException e ) {
LOG.error( "IO Exception occured! {}", e );
}
return result;
}
For my input blah1 blah2 blah3, the output is as follows and i am okay with it.
blah1
blah1 blah2
blah2
blah2 blah3
blah3
However, when the input is Foo bar Foo2, my requirement is to generate the following output:
Foo
Foo bar
bar
bar Foo2
Foo2
If you noticed, I have to preserve the spaces in between 2 words as it is in the input.(Foo bar and not Foo bar).
Can I make any tweaks and ask lucene to handle it internally?
May be its a minor tweak like adding a filter or something and since I am new to Lucene, I don't know where to start.
Thanks in Advance.
I had to write custom tokenizers and and trim filters to achieve this.
1) I created an abstract class DelimiterPreservingCharTokenizer by extending org.apache.lucene.analysis.Tokenizer class. Next, gave my implementation for incrementToken method. I would have extended org.apache.lucene.analysis.util.CharTokenizer if not the class was final. DelimiterPreservingCharTokenizer looks like below.
package lucene.tokenizers;
import java.io.IOException;
import org.apache.lucene.analysis.Tokenizer;
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
import org.apache.lucene.analysis.tokenattributes.OffsetAttribute;
import org.apache.lucene.analysis.util.CharTokenizer;
import org.apache.lucene.analysis.util.CharacterUtils;
import org.apache.lucene.analysis.util.CharacterUtils.CharacterBuffer;
import org.apache.lucene.util.AttributeFactory;
/**
*
* #author Arun Gowda.
* This class is exactly same as {#link CharTokenizer}. Except that, the stream will have leading delimiters. This is to support N-gram vicinity matches.
*
* We are creating a new class instead of extending CharTokenizer because, incrementToken method is final and we can not override it.
*
*/
public abstract class DelimiterPreservingCharTokenizer extends Tokenizer
{
/**
* Creates a new {#link DelimiterPreservingCharTokenizer} instance
*/
public DelimiterPreservingCharTokenizer()
{}
/**
* Creates a new {#link DelimiterPreservingCharTokenizer} instance
*
* #param factory
* the attribute factory to use for this {#link Tokenizer}
*/
public DelimiterPreservingCharTokenizer( AttributeFactory factory )
{
super( factory );
}
private int offset = 0, bufferIndex = 0, dataLen = 0, finalOffset = 0;
private static final int MAX_WORD_LEN = 255;
private static final int IO_BUFFER_SIZE = 4096;
private final CharTermAttribute termAtt = addAttribute( CharTermAttribute.class );
private final OffsetAttribute offsetAtt = addAttribute( OffsetAttribute.class );
private final CharacterUtils charUtils = CharacterUtils.getInstance();
private final CharacterBuffer ioBuffer = CharacterUtils.newCharacterBuffer( IO_BUFFER_SIZE );
/**
* Returns true iff a codepoint should be included in a token. This tokenizer
* generates as tokens adjacent sequences of codepoints which satisfy this
* predicate. Codepoints for which this is false are used to define token
* boundaries and are not included in tokens.
*/
protected abstract boolean isTokenChar( int c );
/**
* Called on each token character to normalize it before it is added to the
* token. The default implementation does nothing. Subclasses may use this to,
* e.g., lowercase tokens.
*/
protected int normalize( int c )
{
return c;
}
#Override
public final boolean incrementToken() throws IOException
{
clearAttributes();
int length = 0;
int start = -1; // this variable is always initialized
int end = -1;
char[] buffer = termAtt.buffer();
while ( true ) {
if ( bufferIndex >= dataLen ) {
offset += dataLen;
charUtils.fill( ioBuffer, input ); // read supplementary char aware with CharacterUtils
if ( ioBuffer.getLength() == 0 ) {
dataLen = 0; // so next offset += dataLen won't decrement offset
if ( length > 0 ) {
break;
} else {
finalOffset = correctOffset( offset );
return false;
}
}
dataLen = ioBuffer.getLength();
bufferIndex = 0;
}
// use CharacterUtils here to support < 3.1 UTF-16 code unit behavior if the char based methods are gone
final int c = charUtils.codePointAt( ioBuffer.getBuffer(), bufferIndex, ioBuffer.getLength() );
final int charCount = Character.charCount( c );
bufferIndex += charCount;
if ( isTokenChar( c ) ) { // if it's a token char
if ( length == 0 ) { // start of token
assert start == -1;
start = offset + bufferIndex - charCount;
end = start;
} else if ( length >= buffer.length - 1 ) { // check if a supplementary could run out of bounds
buffer = termAtt.resizeBuffer( 2 + length ); // make sure a supplementary fits in the buffer
}
end += charCount;
length += Character.toChars( normalize( c ), buffer, length ); // buffer it, normalized
if ( length >= MAX_WORD_LEN ) // buffer overflow! make sure to check for >= surrogate pair could break == test
break;
} else if ( length > 0 ) // at non-Letter w/ chars
break; // return 'em
}
if ( length > 0 && bufferIndex < ioBuffer.getLength() ) {//If at least one token is found,
//THIS IS THE PART WHICH IS DIFFERENT FROM LUCENE's CHARTOKENIZER
// use CharacterUtils here to support < 3.1 UTF-16 code unit behavior if the char based methods are gone
int c = charUtils.codePointAt( ioBuffer.getBuffer(), bufferIndex, ioBuffer.getLength() );
int charCount = Character.charCount( c );
bufferIndex += charCount;
while ( !isTokenChar( c ) && bufferIndex < ioBuffer.getLength() ) {// As long as we find delimiter(not token char), keep appending it to output stream.
if ( length >= buffer.length - 1 ) { // check if a supplementary could run out of bounds
buffer = termAtt.resizeBuffer( 2 + length ); // make sure a supplementary fits in the buffer
}
end += charCount;
length += Character.toChars( normalize( c ), buffer, length ); // buffer it, normalized
if ( length >= MAX_WORD_LEN ) {// buffer overflow! make sure to check for >= surrogate pair could break == test
break;
}
c = charUtils.codePointAt( ioBuffer.getBuffer(), bufferIndex, ioBuffer.getLength() );
charCount = Character.charCount( c );
bufferIndex += charCount;
}
//ShingleFilter will add a delimiter. Hence, the last iteration is skipped.
//That is, for input `abc def ghi`, this tokenizer will return `abc `(2 spaces only). Then, Shingle filter will by default add another delimiter making it `abc `(3 spaces as it is in the input).
//If there are N delimiters, this token will at max return N-1 delimiters
bufferIndex -= charCount;
}
termAtt.setLength( length );
assert start != -1;
offsetAtt.setOffset( correctOffset( start ), finalOffset = correctOffset( end ) );
return true;
}
#Override
public final void end() throws IOException
{
super.end();
// set final offset
offsetAtt.setOffset( finalOffset, finalOffset );
}
#Override
public void reset() throws IOException
{
super.reset();
bufferIndex = 0;
offset = 0;
dataLen = 0;
finalOffset = 0;
ioBuffer.reset(); // make sure to reset the IO buffer!!
}
}
2) A concrete class WhiteSpacePreservingTokenizer extending the above abstract class to provide delimiter
package spellcheck.lucene.tokenizers;
import org.apache.lucene.analysis.Tokenizer;
import org.apache.lucene.analysis.core.WhitespaceTokenizer;
import org.apache.lucene.util.AttributeFactory;
/**
*
* #author Arun Gowda
*
* This class is exactly same as {#link WhitespaceTokenizer} Only difference is, it extends DelimiterPreservingCharTokenizer instead of CharTokenizer
*/
public class WhiteSpacePreservingTokenizer extends DelimiterPreservingCharTokenizer
{
/**
* Construct a new WhitespaceTokenizer.
*/
public WhiteSpacePreservingTokenizer()
{}
/**
* Construct a new WhitespaceTokenizer using a given
* {#link org.apache.lucene.util.AttributeFactory}.
*
* #param factory
* the attribute factory to use for this {#link Tokenizer}
*/
public WhiteSpacePreservingTokenizer( AttributeFactory factory )
{
super( factory );
}
/** Collects only characters which do not satisfy
* {#link Character#isWhitespace(int)}.*/
#Override
protected boolean isTokenChar( int c )
{
return !Character.isWhitespace( c );
}
}
3) The tokenizer above will result in tailing spaces. (Ex: blah____) we need to add a filter to trim those spaces. So we need DelimiterTrimFilter as folows.(We can also just trim by using java's trim. but doing so will be very inefficient since it creates new string)
package spellcheck.lucene.filters;
import java.io.IOException;
import org.apache.lucene.analysis.TokenFilter;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
public class DelimiterTrimFilter extends TokenFilter
{
private final CharTermAttribute termAtt = addAttribute( CharTermAttribute.class );
private char delimiter;
/**
* Create a new {#link DelimiterTrimFilter}.
* #param in the stream to consume
* #param delimiterToTrim delimiter that should be trimmed
*/
public DelimiterTrimFilter( TokenStream in, char delimiterToTrim )
{
super( in );
this.delimiter = delimiterToTrim;
}
#Override
public boolean incrementToken() throws IOException
{
if ( !input.incrementToken() )
return false;
char[] termBuffer = termAtt.buffer();
int len = termAtt.length();
if ( len == 0 ) {
return true;
}
int start = 0;
int end = 0;
// eat the first characters
for ( start = 0; start < len && termBuffer[start] == delimiter; start++ ) {
}
// eat the end characters
for ( end = len; end >= start && termBuffer[end - 1] == delimiter; end-- ) {
}
if ( start > 0 || end < len ) {
if ( start < end ) {
termAtt.copyBuffer( termBuffer, start, ( end - start ) );
} else {
termAtt.setEmpty();
}
}
return true;
}
}
4) My createAnalyzer will look like below
public static Analyzer createAnalyzer( final int shingles )
{
return new Analyzer() {
#Override
protected TokenStreamComponents createComponents( #NotNull String field )
{
final Tokenizer source = new WhiteSpacePreservingTokenizer();
final TokenStream filter = new ShingleFilter( new LowerCaseFilter( source ), shingles );
filter = new DelimiterTrimFilter( filter, ' ' );
return new TokenStreamComponents( source, filter );
}
};
}
Rest of the code will remain the same

Oracle Stored Procedures using Spring JDBC

Hi I am trying to execute stored procedures using Spring JDBC.
Here's the SP class
class IncrementExtraBalanceStoredProcedure extends StoredProcedure {
/**
* #param jdbcTemplate
* #param procedureName
*/
public IncrementExtraBalanceStoredProcedure(JdbcTemplate jdbcTemplate, String procedureName) {
super(jdbcTemplate, procedureName);
declareParameter(new SqlOutParameter(O_RETURN_CODE, Types.INTEGER));
declareParameter(new SqlParameter(P_NUMEC, Types.INTEGER));
declareParameter(new SqlParameter(P_GBYTES, Types.INTEGER));
compile();
}
/**
* #param inputBean
* #return resultObjects
*/
public Map<String, Object> execute(RateLimitLogBean inputBean) {
Map<String,Object> sqlMap = new HashMap<String,Object>();
sqlMap.put(P_NUMEC, inputBean.getNumec());
sqlMap.put(P_GBYTES, inputBean.getGb());
return super.execute(sqlMap);
}
}
I am calling this class from this method.
public int incrementExtraBalance(RateLimitLogBean inputBean) {
IncrementExtraBalanceStoredProcedure procedure = new IncrementExtraBalanceStoredProcedure(this.jdbcTemplate, "RATELIMIT_OWN.increment_extra_balance");
Map<String, Object> resultMap = procedure.execute(inputBean);
if (!StringUtils.isEmpty(resultMap)) {
return ((Integer) resultMap.get(O_RETURN_CODE)).intValue();
}
return -1;
}
But I am getting null value as O_RETURN_CODE. It's supposed to return 0
The execution of this function from Toad - Oracle Db
var z number
exec RATELIMIT_OWN.unlimit_contract (0123,:z)
print z
I got 0 as output in Toad.
Why I am getting null value as return from Java code (no sql exceptions).
Is there anything wrong with code?
native calls returning proper output
public void unlimitContract(RateLimitLogBean inputBean, boolean load) throws SQLException {
String sql = "{call RATELIMIT_OWN.unlimit_contract (?,?)}";
CallableStatement callableStatement = this.dataSource.getConnection().prepareCall(sql);
callableStatement.setInt(1, 0123);
callableStatement.registerOutParameter(2, java.sql.Types.INTEGER);
callableStatement.executeUpdate();
int resultCode = callableStatement.getInt(2);
}
SQL SP
CREATE OR REPLACE PROCEDURE RATELIMIT_OWN.increment_extra_balance (p_numec IN NUMBER,
p_gbytes IN NUMBER,
o_return_code OUT NUMBER)
AS
message logs.errormsg%TYPE;
BEGIN
update balance set extrabalance=extrabalance+(p_gbytes*1073741824),limited=0 WHERE numec = p_numec;
IF SQL%ROWCOUNT = 0
THEN
o_return_code:=1;
ELSE
o_return_code:=0;
message := 'Cops added ' || p_gbytes || ' gb extra volume';
INSERT INTO logs (logid, eventid, origin, numec, VALUE, errormsg) VALUES (seq_log.NEXTVAL, 'NEXTRAROV', 'increment_extra_balance', p_numec, p_gbytes, message);
END IF;
commit;
EXCEPTION
WHEN OTHERS
THEN
o_return_code := SQLCODE;
ROLLBACK;
END;
/
The order of your parameters looks wrong. Try:
declareParameter(new SqlParameter(P_NUMEC, Types.INTEGER));
declareParameter(new SqlParameter(P_GBYTES, Types.INTEGER));
declareParameter(new SqlOutParameter(O_RETURN_CODE, Types.INTEGER));