REST API (JSON) that updates SQL Table using Windows Console Application and Scheduled Tasks - sql

I am a newbie at JSON programming. Most of my experience is in C# and some in XML and Javascript. So I am a bit lost. I will attempt to be as specific as possible.
I have written a windows console application that runs via the task scheduler. Basically the windows application is supposed to take the API from a site that is managed by an outside company but the information is owned by my company and put the information within a SQL table. The API is pretty standard and written in JSON.
I am successful in parsing the JSON language and (for example) displaying it in a command prompt but I need to be able to parse the language and place it into an SQL table. I have read up on SQL injection attacks and I feel fairly confident that we have covered our bases here. So the problem lies in the fact that it does not update the table when the application is run via the scheduler or without the scheduler.
I have included a little bit of the JSON language below along with the language for my console application.
{"date":"2015-09-24","data":[{"cid":"17","rank":1},{"cid":"26","rank":1},{"cid":"80","rank":1},{"cid":"30","rank":1},{"cid":"90","rank":1},{"cid":"62","rank":1},{"cid":"147","rank":1},{"cid":"28","rank":1}"s":1,"e":null}
using System;
using System.Collections;
using System.IO;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Net.Http.Formatting;
using System.Threading.Tasks;
using System.Net;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
namespace JsonApiClient
{
class Program
{
static void Main(string[] args)
{
ExecuteRiskSearch();
Console.ReadLine();
}
static void ExecuteRiskSearch()
{
string url = "https://localhost/api/getWatchList/";
string json = new WebClient().DownloadString(url);
JObject results = JObject.Parse(json);
foreach (var result in results)
{
string cid = (string)results["CID"];
JToken rank = results["rank"];
string risk = "";
if (rank is JValue)
{
risk = (string)rank;
}
else if (rank is JArray)
{
risk = (string)((JArray)rank).First;
}
else
{
SqlConnection connection = null;
SqlCommand command = null;
try
{
connection = new SqlConnection("Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=apiData;Data Source=serverName;");
command = new SqlCommand("UPDATE apiData.dbo.API SET [Category] WHERE CID=CID", connection);
connection.Open();
int numrows = command.ExecuteNonQuery();
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex.Message);
}
finally
{
command.Dispose();
connection.Dispose();
}
}
}
}
}
}
What am I missing to make the JSON data update my SQL table? I have scoured Google search results and I haven't found much information. Any help would be so greatly appreciated.

For the need to foreach with the correct part of the JSON object, what I mean is very simply that your variable results includes the entire JSON object, from the "date" through the "e". You need to start with the "data" object and iterate through its array or your string cid will error out on assignment, as it will be attempting to assign an array to a single value. The same goes for your JToken rank. I believe it should be this:
foreach(datum in results["data"])
{
string cid = datum["cid"];
JToken rank = datum["rank"];
/* ... */
}
In addition, your set command isn't doing anything. You need to use SET columName = " + newValue + " WHERE CID == " + cid to actually affect a change, where columnName is the column you wish to alter and newValue is your C# variable carrying the desired replacement.
It's also a best practice to include a change to an updated date field when updating via an automated process, if there is one present. Generally the convention is to have a created date and an updated date for each row in a table.
I hope this at least points you in the right direction.
-C§

As an alternative, you can send entire text to Sql Server and load it there.
Sql Server 2016 will enable you to store JSON using single command - OPENJSON. In the older versions you can use existing CLR/JSON libraries such as Json4Sql or JsonSelect.

Related

Dbcontext with using-clause in Asp.Net Json-Call

I ran into a problem when switching from the Local Development Server to the Local IIS Server (ASP.NET MVC4) using the following Controller-Method:
public JsonResult GetJsonData(string Context, string Language)
{
using (KeyValueDBContext db = new KeyValueDBContext())
{
var entries = from u in db.KeyValues
where ((u.Context == Context) && (u.Language == Language))
select u;
return Json(entries, JsonRequestBehavior.AllowGet);
}
}
Using the local server, I received data when calling the method from Javascript without a problem. The method retrieves a collection of basically Key-Value pairs from a database repository and sends them to the client). After switching to IIS I got an Exception telling me that the dbcontext had already been disposed of (although the using clause ends after the return-statement). (NB: Visual Studio also was not able to find JSONSerializer.cs for some reason, but only when the error occurred). Switching to the following version solved the problem completely:
public JsonResult GetJsonData(string Context, string Language)
{
KeyValueDBContext db = new KeyValueDBContext();
var entries = from u in db.KeyValues
where ((u.Context == Context) && (u.Language == Language))
select u;
return Json(entries, JsonRequestBehavior.AllowGet);
}
In both cases, this was the using-block:
using System;
using System.Collections.Generic;
using System.Data;
using System.Data.Entity;
using System.Linq;
using System.Web;
using System.Web.Mvc;
using beepov4.Models; // project-models
My question: Is this an acceptable way to use dbcontext for the purpose of JSON-calls (and dropping the using-clause) or is there a particular downside or hidden problem I should be aware of?
Try reading the entries into memory with entries.ToList() before passing them to Json():
using (KeyValueDBContext db = new KeyValueDBContext())
{
var entries = from u in db.KeyValues
where ((u.Context == Context) && (u.Language == Language))
select u;
return Json(entries.ToList(), JsonRequestBehavior.AllowGet);
}

Dapper not adding parameters

I am trying to use Dapper for our complex queries to remove any lost overhead that was previously existing with NH.
I have the following query (note this has been considerably shrunk):
SELECT DISTINCT *
FROM tasks t
WHERE t.initials = #UserInits
Which is called via our repository as so:
taskRepo.RawExec<TaskListItemDTO>(Query,new {UserInits = "SAS"})
Our implementation of DapperExec consist as follows:
public IEnumerable<T> RawExec<T>(string SQL, object param)
{
return _session.Connection.Query<T>(SQL,param);
}
But Dapper doesn't appear to be adding the parameters to the query, and as a result, we are getting syntax errors.
Incase it helps, we are connecting over ODBC to Informix.
Thanks
Update Code Sample:
Sorry it took so long, been very busy with work! Below is a sample for MS SQL (2008) Server that should simple query the sys.all_objects (systables?) with a param value of 1 or 0 - but in this sample, as ODBC does not use named params, this won't work.
using Dapper;
using DapperSQL;
using System.Collections.Generic;
using System.Data;
using System.Data.Odbc;
namespace DapperTests
{
public class SQLEx
{
private OdbcConnection GetConnection()
{
var cnn = new OdbcConnection("DSN=ODBCSOURCE");
cnn.Open();
// wrap the connection with a profiling connection that tracks timings
return cnn;
}
public IEnumerable<object> DapperTest()
{
using (OdbcConnection conn = GetConnection())
{
return conn.Query("SELECT * FROM sys.all_objects where is_ms_shipped = ?", new { is_ms_shipped = 1 });
}
}
}
I know this is old post, just use SP instead of query, please check this link Dapper using ODBC store procedure Input parm, this using sybase odbc Sp, all odbc use same technique, I wish it works in Informix.

Using SMO Library in SSIS 2008

Good afternoon,
Im having some trouble writing a simple script using SMO objects in C# to validate if an object exists and then create it. This code is within a Script Task Component in SSIS. The code executes successfully, however the new database does not show up on my local instance. Any help would be greatly appreciated.
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Collections;
using System.Data.SqlClient;
using Microsoft.SqlServer.Management.Common;
using Microsoft.SqlServer.Management.Smo;
public void Main()
{
//String DBName = Dts.Variables["TmpViewDBName"].Value.ToString();
//String Instance = Dts.Variables["TmpViewDBInstance"].Value.ToString();
String DBName = "localhost";
String Instance = "TmpViewDB";
Server TmpViewServer = new Server(Instance);
//Windows Auth
TmpViewServer.ConnectionContext.LoginSecure = true;
TmpViewServer.ConnectionContext.Connect();
if (TmpViewServer.Databases[DBName] != null)
{
TmpViewServer.Databases[DBName].Drop();
}
Database TmpViewDB = new Database(TmpViewServer, DBName);
if (TmpViewServer.ConnectionContext.IsOpen)
TmpViewServer.ConnectionContext.Disconnect();
Dts.TaskResult = (int)ScriptResults.Success;
}
I believe you need to add a line to actually create the object. As it stands now, you've only instantiated the object but never actually made the call to the database to create the remote object.
Database TmpViewDB = new Database(TmpViewServer, DBName);
TmpViewDB.Create();

Informix with NHibernate

I am trying to get Informix working with NHibernate on windows 7. I have a connection string that works fine with informix now, it is this,
Database=db;Server=server:port;uid=username;password=password;pooling=false
I am using the IBM.Data.Informix .NET provider version 9.0.0.2.
We have a number of different applications that work fine using this provider with the Informix servers that we are running.
My nhibernate application is connecting to the informix server now, but the problem is the form of the SQL that it is producing.
If my nhibernate code looks like this,
using (ISession session = Config.SessionFactory.OpenSession())
{
return session
.CreateCriteria<DBTable>()
.Add(Restrictions.Eq("FieldValue", true))
.List<DBTable>();
}
I am new to Informix, but if I am not wrong the correct SQL would be this,
select * from DBTable where fieldValue = 'T'
But instead the SQL is it producing is,
select * from DBTable where fieldValue = True
Which is not working. I tried adding stuff like this to the nhibernate config file,
<property name="query.substitutions">True=T,False=F</property>
<property name="query.substitutions">True 'T',False 'F'</property>
<property name="query.substitutions">True='T',False='F'</property>
<property name="query.substitutions">True T,False F</property>
but that just doesn't seem to work. I couldn't find consistent documentation as to how to use the query.substitutions, and it seemed to differ depending on what database type you are using.
What version of NHibernate are you using?
What is the property type of FieldValue?
I'm using NHibernate with Informix and queries with boolean restrictions work fine. These are the relevant configuration values:
<property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
<property name="connection.driver_class">NHibernate.Driver.IfxDriver</property>
<property name="dialect">NHibernate.Dialect.InformixDialect1000</property>
Commentary on original version of question
To answer this is going to require some more information - and more information requests than fits into a comment.
First, I assume from the discussion that you are using IBM Informix Dynamic Server (IDS) on some variant of Windows (but which one?). Which version of IDS are you using, please? Also, which version of the ClientSDK are you using?
Next, I assume that you can connect to the IDS database using some program - the problem is related to access via NHibernate, rather than access at all. If your problem is that you have not yet been able to access IDS at all, then the debugging steps are rather different from 'not via an alternative access method'.
When you connect via the other method, (a) what is that other method, and (b) what is the connection string you use? I would like to see the 'structural details' of string values. For example, if you specify the database as 'sensitive#secret', I'd like to see the notation 'xxxxxxxxx#yyyyyy' because although we don't necessarily need to know that the names are sensitive and secret, we do need to know roughly what the names look like. Similarly for any other value in the strings. You said you removed the sensitive information, but you've done it so thoroughly that I can't tell whether what you supplied was sensible.
Have you used SETNET32 to set any Informix environment values - a question that only applies to Windows platforms (Unix uses regular environment variables and not a central registry)? If so, what have you set?
How do the working connection strings for the non-NHibernate package compare with the non-working strings for NHibernate?
Lastly (for now) you showed that you had tried to use both the IBM DB2 .NET driver and the Informix OLEDB driver. You need to be aware that the DB2 .NET driver uses the DRDA protocol to talk to IDS, whereas the Informix driver uses the SQLI protocol. By default, IDS only listens for SQLI connections - you would have to configure IDS to accept the DRDA connections. Fixing that gets into some of the details of IDS administration - which I hope we don't need to deal with, but we may.
How to get the information onto SO? I suggest you edit your question, adding the extra information to the question so other people can easily see the issues. (I'm not an expert with IDS on Windows; my backyard is Unix-based. I will likely have to get other people to assist in providing the answer, but the necessary information isn't yet available.)
Commentary on question as amended
IDS supports BOOLEAN types in an unorthodox way - it does not recognize true or false (or unknown) as boolean values; it uses 't' and 'f' instead. Consequently, the code generated by NHibernate is not valid for IDS (though I'd accept arguments to the effect that it should be). I'm not clear whether there's a good way to resolve the problem. If you can persuade NHibernate to pass the quoted characters instead of true and false, then you're in with a fighting chance.
Because of this:
fieldValue = 'T'
I'm drawing conclusion that fieldValue have Char(1) type in database, not boolean.
If so you should use:
Restrictions.Eq("FieldValue", "T")
I haven't tested this but I think that your issue is not Informix - specific. You'll get the same error with different database.
This is because every NHibernate dialect provides ToBooleanValueString method. By default boolean values are mapped to "0" and "1" (so you can't expect "T" in other databases) and for Informix it is mapped to "t" and "f". Apparently this method is not used by Restrictions.Eq which is not Informix fault.
Probably Restrictions.Eq is using internally boolean variable (because your argument is boolean) and calls its "ToString" method because database column is of character type.
Result of "ToString" for true value is just "true".
The problem is happening because the newer Informix drivers are using the different connections, and it is misunderstanding the booleans.
The solution that I found is to create a new NHibernate Driver, inherited from NHibernate.Driver.IfxDriver. Treating the parameters just before the query execution.
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NHibernate;
using NHibernate.Driver;
using NHibernate.SqlCommand;
using NHibernate.SqlTypes;
namespace DataAccess
{
public class NHibernateCustomDriver : NHibernate.Driver.IfxDriver
{
public override IDbCommand GenerateCommand(CommandType type, SqlString sqlString, SqlType[] parameterTypes)
{
IDbCommand cmd = CreateCommand();
cmd.CommandType = type;
SetCommandTimeout(cmd);
SetCommandText(cmd, sqlString, parameterTypes);
SetCommandParameters(cmd, parameterTypes);
return cmd;
}
private void SetCommandText(IDbCommand cmd, SqlString sqlString, SqlType[] parameterTypes)
{
SqlStringFormatter formatter = GetSqlStringFormatter();
formatter.Format(sqlString);
int index = 0;
int count = 0;
string newCommand = formatter.GetFormattedText();
index = newCommand.IndexOf("?");
while (index >= 0)
{
if (parameterTypes[count].DbType == DbType.Boolean)
{
newCommand = newCommand.Substring(0, index) + "CAST(? AS BOOLEAN)" + newCommand.Substring(index + 1);
index = newCommand.IndexOf("?", index + 1);
}
count++;
index = newCommand.IndexOf("?", index + 1);
}
cmd.CommandText = newCommand;
}
private void SetCommandParameters(IDbCommand cmd, SqlType[] sqlTypes)
{
for (int i = 0; i < sqlTypes.Length; i++)
{
string paramName = ToParameterName(i);
IDbDataParameter dbParam = GenerateParameter(cmd, paramName, sqlTypes[i]);
cmd.Parameters.Add(dbParam);
}
}
private static string ToParameterName(int index)
{
return "p" + index;
}
}
}
The NHibernate configuration must be set to the new class.
cfg.SetProperty("connection.driver_class", "DataAccess.NHibernateCustomDriver, DataAccess");
You also have to create a custom type, to handle those booleans:
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NHibernate.Type;
namespace DataAccess
{
public class NHibernateUnixBooleanType : BooleanType
{
public override void Set(IDbCommand cmd, object value, int index)
{
cmd.Parameters[index] = CloneParameter(cmd, cmd.Parameters[index] as IDbDataParameter, value as bool?);
}
private IDbDataParameter CloneParameter(IDbCommand cmd, IDbDataParameter originalParameter, bool? value)
{
var clone = cmd.CreateParameter();
clone.DbType = DbType.String;
clone.Value = value.Value ? "t" : "f";
clone.ParameterName = originalParameter.ParameterName;
return clone;
}
}
}
And set this type on the mapping file:
<property name="Property" column="column" type="DataAccess.NHibernateUnixBooleanType, DataAccess"></property>

How can I just get an IfxBlob from am Informix query in .net?

I can't seem to find a way to get just the locator object of a column under .Net. It seems that Informix is automatically converting the blob column to Byte[] and not leaving a way to change that behavior.
IBM.Data.Informix.IfxConnection c =
new IBM.Data.Informix.IfxConnection("...");
c.Open();
IBM.Data.Informix.IfxCommand cmd =
new IBM.Data.Informix.IfxCommand("SELECT id,data FROM aaa", c);
IBM.Data.Informix.IfxDataReader r = cmd.ExecuteReader();
while (r.Read()) {
Debug.WriteLine(r.GetValue(1).GetType());
}
c.Close();
results:
System.Byte[]
System.Byte[]
System.DBNull
System.DBNull
I expected:
IBM.Data.Informix.IfxBlob
or something similar.
I asked a colleague about this, and this is his response to me. Since it isn't my handiwork, I've made the answer 'Community Wiki' so I don't get the credit (beyond knowing where to ask).
To answer the question... the following program was written using the Common Informix Provider (the IBM.Data.Informix.dll which uses the DRDA communication protocol... you can get it in the "IBM Data Server Driver for CLI, ODBC, and .NET" package). Something very similar should be able to be done with the Legacy Informix Provider (the IBM.Data.Informix.dll which uses the SQLI communication protocol... you can get it in the "Informix Client SDK" package).
Here's an example program:
using System;
using System.Collections.Generic;
using System.Text;
using System.Data;
using IBM.Data.Informix;
namespace InformixClob
{
class Program
{
static void Main(string[] args)
{
try
{
IfxConnection tConn = new IfxConnection("database=idsdb;server=my-system:9089;uid=informix;pwd=********");
tConn.Open();
IfxCommand tCmd = tConn.CreateCommand();
// create table mytesttab (col1 integer, col2 clob)
tCmd.CommandText = "select * from mytesttab";
IfxDataReader tRdr = tCmd.ExecuteReader();
while (tRdr.Read())
{
Console.WriteLine("Col1 is a {0}", tRdr.GetValue(0).GetType());
Console.WriteLine("Col2(GetValue) is a {0}", tRdr.GetValue(1).GetType());
Console.WriteLine("Col2(GetIfxValue) is a {0}", tRdr.GetIfxValue(1).GetType());
Console.WriteLine("Col2(GetIfxClob) is a {0}", tRdr.GetIfxClob(1).GetType());
}
tRdr.Close();
tConn.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
Console.Write("Press ENTER"); Console.ReadLine();
}
}
}
}
And here's the output it generates:
Col1 is a System.Int32
Col2(GetValue) is a System.String
Col2(GetIfxValue) is a IBM.Data.Informix.IfxClob
Col2(GetIfxClob) is a IBM.Data.Informix.IfxClob
Press ENTER
The IfxDataReader.GetValue(int) method is going to return the column value in a native .NET Framework data type. To get the column value returned as an Informix type, you must request that it be returned as such by either calling the GetIfxValue(int) method, or if you can be more specific, by the GetIfxClob(int) method.