I have used FastText to train sarcasm.txt file. The Text file looks like below (I have use Tokenization so it looks like this)
__label__1 [ ' beijing ' , ' fire ' , ' department ' , ' extinguishe ' , ' massive ' , ' fivealarm ' , ' burn ' , ' cloud ' , ' of ' , ' smog ' ]
__label__0 [ ' the ' , ' outfit ' , ' that ' , ' get ' , ' this ' , ' woman ' , ' kick ' , ' out ' , ' of ' , ' she ' , ' school ' , ' gym ' ]
__label__1 [ ' area ' , ' smoker ' , ' one ' , ' of ' , ' america ' , ' top ' , ' phlegmproducer ' ]
__label__1 [ ' ryan ' , ' seacr ' , ' nervous ' , ' about ' , ' how ' , ' audience ' , ' will ' , ' respond ' , ' to ' , ' slightly ' , ' short ' , ' haircut ' ]
__label__0 [ ' lot ' , ' of ' , ' parent ' , ' know ' , ' this ' , ' scenario ' ]
__label__1 [ ' great ' , ' daughter ' , ' measure ' , ' selfworth ' , ' against ' , ' some ' , ' 13yearold ' , ' name ' , ' skyla ' , ' now ' ]
__label__0 [ ' video ' , ' relaunche ' , ' investigation ' , ' into ' , ' death ' , ' of ' , ' man ' , ' hold ' , ' by ' , ' chicago ' , ' police ' ]
__label__0 [ ' trumpcare ' , ' score ' , ' so ' , ' badly ' , ' it ' , ' could ' , ' actually ' , ' help ' , ' the ' , ' senate ' ]
__label__1 [ ' break ' , ' flight ' , ' attendant ' , ' currently ' , ' attempt ' , ' to ' , ' pass ' , ' cup ' , ' of ' , ' cranberry ' , ' juice ' , ' over ' , ' your ' , ' laptop ' ]
__label__1 [ ' newly ' , ' swornin ' , ' north ' , ' korean ' , ' official ' , ' wonder ' , ' how ' , ' hell ' , ' eventually ' , ' be ' , ' execute ' ]
I used the code here to train this into FastText:
model_testing = fasttext.train_supervised('sarcasm_train.txt', minn=1, maxn=3)
I can access word embedding for a word using this code:
model_testing.get_word_vector('colorblind')
The vector looks like this:
array([ 0.02441351, -0.02762945, 0.0044672 , 0.02173712, -0.02845885,
-0.01788719, -0.01404536, -0.00040487, -0.02752208, -0.01858106,
0.00499649, -0.00495898, -0.02149334, -0.03080621, 0.01518185,
0.02701956, 0.00429512, -0.01376632, 0.00949627, 0.01379911,
0.01134215, -0.03492524, -0.02157653, 0.00827039, 0.00609733,
0.0153464 , 0.00993295, 0.00396577, 0.01779681, -0.01860862,
0.02510854, 0.0039301 , 0.01420423, 0.04053118, 0.05107417,
0.00303496, -0.00626393, 0.02165087, -0.00677479, 0.02452566,
-0.01753292, -0.0372146 , -0.0109925 , -0.01465018, 0.00677392,
-0.0062581 , -0.00041438, 0.01762512, 0.01957742, -0.02030487,
0.03129215, -0.02718819, 0.02537155, -0.02269336, 0.01991356,
0.01586418, 0.01099472, -0.00405847, -0.02595887, 0.0024101 ,
0.02718542, -0.01486973, -0.02627936, -0.03032344, 0.00531832,
0.00363665, 0.02218294, -0.01040652, -0.00741611, -0.02091474,
0.03373858, -0.01403952, 0.01352888, 0.01178332, 0.00370314,
0.01607108, -0.01730891, -0.0314983 , 0.00030702, 0.00751614,
0.0237149 , -0.0080571 , -0.02801514, 0.02703649, 0.00633383,
0.01455203, -0.00644819, 0.00479063, 0.02993772, 0.00366506,
-0.03687849, -0.01704783, -0.03367983, 0.03158782, -0.00666518,
0.02971195, -0.01610409, -0.02939436, 0.02555595, 0.0267504 ],
dtype=float32)
I have 3 Questions:
Does this vector look right? Should I skip Tokenization?
How can I access the vectors for all the words of all my words? Can I make a Numpy array consisting of the vectors?
Can I split those vectors based on Unigram, Bigram and Trigram? Each having the same 100 Features?
I have:
SELECT KEYWORDS = CAST(USRN AS VARCHAR(15)) + ' ' +
RTRIM(SD) + ' ' + RTRIM(NL.LOCALITY_NAME) + ' ' +
RTRIM(NT.TOWN_NAME) + ' ' + RTRIM(NA.AUTHORITY_NAME)
That gives me what looks like a column, but is not:
I want to have it so my code only selects the rows from KEYWORDS that match whatever the user is typing. Normally, if KEYWORDS was a column, I would write:
SELECT .... WHERE KEYWORDS = '%whateverTheUserIsTyping%'
but I cannot because keywords is not a real column and it is telling me that it does not exist.
How do I get around this? thanks
The column alias KEYWORDS isn't visible to the SQL Engine at the time that the WHERE clause is evaluated. You can just repeat your CAST statment, though.
SELECT
KEYWORDS = CAST(USRN AS VARCHAR(15)) + ' ' +
RTRIM(STREET_DESCRIPTOR) + ' ' + RTRIM(NSG_LOCALITY.LOCALITY_NAME) + ' ' +
RTRIM(NSG_TOWN.TOWN_NAME) + ' ' + RTRIM(NSG_AUTHORITY.AUTHORITY_NAME)
FROM yourTable
WHERE
CAST(USRN AS VARCHAR(15)) + ' ' +
RTRIM(STREET_DESCRIPTOR) + ' ' + RTRIM(NSG_LOCALITY.LOCALITY_NAME) + ' ' +
RTRIM(NSG_TOWN.TOWN_NAME) + ' ' + RTRIM(NSG_AUTHORITY.AUTHORITY_NAME)
LIKE '%whateverTheUserIsTyping%'
You can use derived table with an alias (here Q) and get the result from that by filtering in WHERE clause:
SELECT Q.KEYWORDS FROM (
SELECT KEYWORDS = CAST(USRN AS VARCHAR(15)) + ' ' +
RTRIM(STREET_DESCRIPTOR) + ' ' + RTRIM(NSG_LOCALITY.LOCALITY_NAME) + ' ' +
RTRIM(NSG_TOWN.TOWN_NAME) + ' ' + RTRIM(NSG_AUTHORITY.AUTHORITY_NAME)
) AS Q
WHERE Q.KEYWORDS LIKE '%whateverTheUserIsTyping%'
Because you can't use the column alias in the WHERE clause as you mentioned. Also instead of the KEYWORDS = '%whateverTheUserIsTyping%', you can use LIKE operator.
Here is the SQL Source Code for database selection that generates the following
error: "Query execution failed, Reason: SQL Error [920] [42000]:
ORA-00920: opérateur relationnel non valide.
Please Help!
WITH adr_siege as (
SELECT Nvl(adr.libadr, ' ') AS libadr, Nvl(adr.adress, ' ') AS adress,
Nvl(adr.codpos, ' ') AS codpos, Nvl(adr.locali, ' ') AS locali,
Nvl(adr.codpay, ' ') AS codpay
FROM gnx.tie
JOIN gnx.adr
ON adr.codsoc = tie.codsoc
AND adr.typtie = tie.typtie
AND adr.sigadr = tie.sigtie
AND typadr = 'COM'
WHERE tie.typtie = 'DEP'
AND tie.codsoc=1
AND tie.sigtie = '00000137'
)
SELECT Nvl(a.codpro, ' ') AS codpro, Nvl(a.nompro, ' ') AS nompro,
Nvl(ll.prix_achat, 0) AS prix_achat, Nvl(ll.qte, 0) AS qte,
Nvl(l.nom_fournisseur, ' ') AS nom_fournisseur, Nvl(m.raison, ' ') AS raison,
Nvl(concat(m.adresse1,' ' || m.adresse2), ' ') AS adresse_mag,
Nvl(m.cp, ' ') AS cp, Nvl(m.ville, ' ') AS ville,
Nvl(p.libelle, ' ') AS libelle,
Nvl(To_Char(c.date_validation, 'DD/MM/YYYY'), ' ') AS date_validation,
Nvl((
SELECT sum(fll.prix_achat * fll.qte)
FROM fourniture.frnt_livraison_ligne fll
WHERE fll.noliv
), 0) AS total_prix,
To_Char(SYSDATE, 'DD/MM/YYYY') AS date_edition,
Nvl(f.libadr, ' ') AS fou_lib_adr,
Nvl(f.tel, ' ') AS fou_tel,
Nvl(f.adress, ' ') AS fou_adr, Nvl(f.codpos, ' ') AS fou_cp,
Nvl(f.locali, ' ') AS fou_ville, Nvl(pf.libelle, ' ') AS fou_pays,
Nvl(to_char(p.date_livraison_s1, 'DD/MM/YYYY'), ' ') AS date_livraison_s1,
Nvl(to_char(p.date_livraison_s2, 'DD/MM/YYYY'), ' ') AS date_livraison_s2,
Nvl((
SELECT tbl.lib1
FROM gnx.tie
LEFT JOIN gnx.tbl
ON tbl.codtbl = 'mrg'
AND tbl.lib1 IS NOT NULL
AND tbl.codsoc=1
AND tbl.cletbl = tie.modrgl
WHERE tie.typtie = 'FOU'
AND tie.sigtie = l.code_fournisseur
AND tie.codsoc =1
), ' ') AS mode_paiement,
l.num_sous_periode,
adr_siege.*,
Nvl(m.nummag, ' ') AS nummag,
Nvl(m.tel1, ' ') AS tel1,
l.ref_gnx,
Nvl(tva.taux_tva, 0)/100 AS taux_tva,
Nvl(f.code, ' ') AS code_fournisseur,
(
CASE WHEN (m.lundi_ouverture != '0' OR m.mardi_ouverture != '0' OR m.mercredi_ouverture != '0'
OR m.jeudi_ouverture != '0' OR m.vendredi_ouverture != '0' OR m.samedi_ouverture != '0') THEN 1
ELSE 0
END
) AS AFFICH_HORRAIRE,
a.conditionnement,
a.refpro as REFPRO
FROM adr_siege, fourniture.frnt_livraison l
INNER JOIN fourniture.frnt_livraison_ligne ll ON ll.noliv = l.noliv
INNER JOIN fourniture.frnt_article a ON a.codpro = ll.codpro
INNER JOIN fourniture.frnt_commande c ON c.nocde = l.nocde
INNER JOIN polymag.magasin m ON m.nummag = c.nummag
INNER JOIN polymag.pays p ON p.code = m.codepays
LEFT JOIN fourniture.frnt_fournisseur f ON f.code =l.code_fournisseur
INNER JOIN polymag.pays pf ON pf.code = f.codpay
INNER JOIN fourniture.frnt_periode p ON p.noperiode = c.noperiode
LEFT JOIN fourniture.frnt_tva tva ON tva.code_pays_fou = f.codpay AND tva.code_pays_mag = m.codepays
WHERE l.type_livraison = 'FOU'
(we are selecting in schema named 'fournisseur')
Thank you in advance for your help.
problem is at line 25 because of WHERE fll.noliv that's without any relational operator like <,>,!,= after this statement.
TOTAL_PRIX seems to be invalid; look at its WHERE clause:
(
SELECT SUM(fll.prix_achat * fll.qte)
FROM fourniture.frnt_livraison_ligne fll
WHERE fll.noliv ), 0) AS total_prix,
I am attempting to generate lines for an SDF (Space Delimited File). I am creating these lines from a DBASE IV DBF file using an OLEDB adapter with extended properties DBASEIV to get at the data. My data column output is 425 characters long after padding, I am placing this into a datagridview in VB.NET to display it.
However when I run my query, while it seems to execute correctly the resultant field is restricted to 256 characters. The longest individual field I am reading is 35 characters and I am returning a dataset with 2 fields, the barcode and the SDF line. As I understand it OLEDB Jet 4.0 tries to guess the type based on the first 8 rows, however as all rows are equal length for the data column (425 chars) I don't get why it is choosing the smaller field type. I assume it is because my field is a generated one using string concatenation. I have included the horrible SQL at the bottom of this question. So my question is how can I get the full 425 character output? Or is there a way I can specify the datatype for my own field as memo?
SELECT scan,
RIGHT('0000000000000' + trim(cstr(scan)), 13) +
LEFT(trim(cstr(name)) + ' ', 35) +
LEFT(trim(cstr(name)) + ' ', 16) +
' ' +
' ' +
' ' +
'1 ' +
'0.00 ' +
'0.00 ' +
'1' +
'0.00 ' +
'0.01 ' +
'0.00 ' +
'F' +
'2' +
'0.00 ' +
'0.00 ' +
' ' +
' ' +
' ' +
'SALS' +
' ' +
' ' +
LEFT(trim(cstr(plof)) + ' ', 13) +
' ' +
' ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
'F' +
'T' +
'001' +
' ' +
'T' +
'01' +
' ' +
' ' +
' ' +
' ' +
'F' +
'F' +
' ' +
' ' +
'0 ' +
'0.00 ' +
' ' +
'0 ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
'1 ' +
'1 ' +
'1 ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
'0.00 ' +
' ' +
' ' +
' '
as STTEMPLINE
from salus where cstr(scan) in (select distinct cstr(scan) from nonscan)
Thanks in advance for any help.
This is a known issue where data types are guessed based on data in the first few rows.
See this post...
Load Excel sheet into DataTable but only the first 256 chars per cell are imported,
Also, see this post for an explanation of how OLEDB deals with mixed data types...
http://social.msdn.microsoft.com/Forums/pl-PL/csharplanguage/thread/0404d003-5bfb-44f9-8e6b-aebdfce24875
I have a dynamic query like this :
SET #str_Query = 'SELECT SIM.Item_ID,
SIM.Item_Description,
SU.Short_Description AS Unit,
SIM.Std_Lead_Time,'+
'' ''+' AS Last_Purchase_Rate
FROM FKMS_Item_Master AS SIM
INNER JOIN FKMS_STP_Units SU
ON SIM.Item_Purchase_Unit=SU.Unit_Id' +
' WHERE ' + #str_Condition +
' AND SIM.Location_Id =' + CAST(#aint_Location_Id AS VARCHAR(10)) +
' AND SIM.Item_Deleted =0
AND SIM.Approved_On IS NOT NULL'
+' ORDER BY SIM.Item_Description'
I want to retrieve space as Last_Purchase_Rate
It is showing syntax error in the portion of '' ''+' AS Last_Purchase_Rate
when I execute this query.
If I print this dynamic query, query seems correct. It shows as AS Last_Purchase_Rate with space before AS. Please help.
I would write
...SIM.Std_Lead_Time, '' '' AS Last_Purchase_Rate...
instead of
...SIM.Std_Lead_Time,'+'' ''+' AS Last_Purchase_Rate...
Why not use NULL instead of space and then handle the result in your app?
I.e.,
SET #str_Query = 'SELECT SIM.Item_ID,
SIM.Item_Description,
SU.Short_Description AS Unit,
SIM.Std_Lead_Time,
NULL AS Last_Purchase_Rate, -- and so on.
You could also use CHAR(32):
SET #str_Query = 'SELECT SIM.Item_ID,
SIM.Item_Description,
SU.Short_Description AS Unit,
SIM.Std_Lead_Time,
CHAR(32) AS Last_Purchase_Rate, -- and so on.
You did not escape all quotes.
A working version of your statement would be
SET #str_Query = 'SELECT SIM.Item_ID,
SIM.Item_Description,
SU.Short_Description AS Unit,
SIM.Std_Lead_Time,'
+ ''' '''
+ ' AS Last_Purchase_Rate
FROM FKMS_Item_Master AS SIM
INNER JOIN FKMS_STP_Units SU
ON SIM.Item_Purchase_Unit=SU.Unit_Id' +
' WHERE ' + #str_Condition +
' AND SIM.Location_Id =' + CAST(#aint_Location_Id AS VARCHAR(10)) +
' AND SIM.Item_Deleted =0
AND SIM.Approved_On IS NOT NULL'
+' ORDER BY SIM.Item_Description'
but I find that with a little reformatting, the error is easier to spot
SET #str_Query =
'SELECT SIM.Item_ID '
+ ', SIM.Item_Description '
+ ', SU.Short_Description AS Unit '
+ ', SIM.Std_Lead_Time '
+ ', '' ''' + ' AS Last_Purchase_Rate '
+ 'FROM FKMS_Item_Master AS SIM '
+ ' INNER JOIN FKMS_STP_Units SU '
+ ' ON SIM.Item_Purchase_Unit=SU.Unit_Id '
+ ' WHERE ' + #str_Condition
+ ' AND SIM.Location_Id = ' + CAST(#aint_Location_Id AS VARCHAR(10))
+ ' AND SIM.Item_Deleted =0 '
+ ' AND SIM.Approved_On IS NOT NULL '
+ ' ORDER BY SIM.Item_Description '
Try using tsql function SPACE(1)