I'm stuck at the moment trying to figure out a method of inserting into SQL Server from F#.
I have an F# function that iterates through all files inside a folder following a user-defined pattern. Then I can use the returned data to put in a list or (ideally) insert into a database.
I already have a working insert-into-sql function that works properly:
let execNonQuery conn s =
let comm =
new SqlCeCommand(s, conn)
try
comm.ExecuteNonQuery() |> ignore
with e ->
printf "Error : %A\n" e
let string = "insert into MyTable (MyColumn) values ('test .. again')"
execNonQuery conn string; // works
I'm trying to get this method to work properly:
let rec getAllFiles dir pattern =
seq { yield! Directory.EnumerateFiles(dir, pattern)
for d in Directory.EnumerateDirectories(dir) do
yield! getAllFiles d pattern }
let getApplications (dir : string) (extension : string) =
getAllFiles dir extension
//|> Seq.toList // If I need to create a list of returned values
|> Seq.iter (fun s -> SQLInsertString s) // This does not work as it complains about the function not being of type unit
If I use Seq.toList only and call the function as per below, it works:
getApplications "C:\Admin" "*.txt" // works
The other thing I don't understand is how can you create a working insert command that takes in a string for Value. For example:
let SQLInsertString s = "insert into MyTable (MyColumn) values (%s)" //does not work
You're almost there. The problem is sqlInsertString returns string which is not legal to use in Seq.iter.
What you're doing with sqlInsertString is to create a string using string formats. It fits nicely with sprintf function:
let sqlInsertString s =
sprintf "insert into MyTable (MyColumn) values (%s)" s
Now you can use execNonQuery on results of sqlInsertString to actually insert data into database. Since execNonQuery returns unit, it could be easily used in Seq.iter:
// Assuming conn is a global and already defined variable.
let getApplications (dir : string) (extension : string) =
getAllFiles dir extension
|> Seq.iter (fun s -> execNonQuery conn (sqlInsertString s))
Since type annotation is redundant, your code could be rewritten in a more idiomatic way:
let getApplications dir extension conn =
getAllFiles dir extension
|> Seq.iter (sqlInsertString >> execNonQuery conn)
The best way to pass parameters to a query is to use SqlCeParameter. This is easier than composing strings (because you don't need to encode strings and escape quotes) and it is also safer, because you avoid SQL injection attack. Here is a basic sample:
let sqlInsertString value =
// Create and open connection ('use' makes sure it gets closed at the end)
use conn = new SqlCeConnection("...");
conn.Open()
// Create a command with a parameter named '#str'
let cmd = new SqlCeCommand("INSERT INTO MyTable (MyColumn) values (#str)", conn)
// Create parameter '#str' with string value 'value' and add it to the command
let param = new SqlCeParameter("#str", SqlDbType.NVarChar, value)
cmd.Parameters.Add(param)
// Now run the command (exception handling omitted)
cmd.ExecuteNonQuery() |> ignore
Using this function, you should now be able to use Seq.iter. The function takes a string to be inserted and returns unit (no value), so it can be passed to Seq.iter:
let getApplications (dir : string) (extension : string) =
getAllFiles dir extension
|> Seq.iter (fun s -> sqlInsertString s)
Alternatively, you can write the last line just as |> Seq.iter sqlInsertString. If you do that, you're basically saying that the argument s should be directly passed to the sqlInsertString function.
Related
I found this SQLHelper online that I would like to run a SQL query with.
But the helper wants an list instead of an string.
and I cannot seem to figure out how to make the executeNonQuery to work.
type SqlHelper (connection) =
let exec bind parametres query =
use conn = new SqlConnection (connection)
conn.Open()
use cmd = new SqlCommand (query, conn)
parametres |> List.iteri (fun i p ->
cmd.Parameters.AddWithValue(sprintf "#p%d" i, box p) |> ignore)
bind cmd
member __.Execute = exec <| fun c -> c.ExecuteNonQuery() |> ignore
member __.Scalar = exec <| fun c -> c.ExecuteScalar()
member __.Read f = exec <| fun c -> [ let read = c.ExecuteReader()
while read.Read() do
yield f read ]
let sql = new SqlHelper (connectionString)
The query I have is for dopping the tables
and I'm trying to execute like this.
let emptyDb =
let query =
"SET NOCOUNT ON
DROP TABLE IF EXISTS #STUFF
...
...
END"
sql.Execute [query ]
This compiles, but nothing happens when I execute it.
Any ideas?
Thanks in advance
Edit: sql.Read function works perfect
let GetToken Id=
sql.Read (fun r -> { token = unbox r.[0] })
[Id;]
"SELECT Token
FROM [dbo].[Token]
WHERE id= 0"
GetToken "1337"
You are not providing enough parameters for sql.Execute.
Look closely:
exec takes three parameters - bind, parametres (btw, typo), and query
In the body of Execute you give it one parameter - bind
Therefore, the result of Execute is a function that still expects the other two parameters - parametres and query
But when you're calling sql.Execute, you're only giving it one parameter - [query], which will end up bound to parametres
Therefore, the result of calling sql.Execute [query] is yet another function, which still expects the final parameter to be provided before its body will be executed. In fact, if you pay close attention to compiler warnings, you will see that the compiler actually tells you as much:
This expression is a function value, i.e. is missing arguments. Its type is ...
To fix, provide the correct parameters. Judging by the little piece of your query that I can see, I assume that it's not supposed to have any parametres, so I'll put an empty list there:
sql.Execute [] query
I have an SQL statement that I'm executing through OleDb, the statement is something like this:
INSERT INTO mytable (name, dept) VALUES (#name, #dept);
I'm adding parameters to the OleDbCommand like this:
OleDbCommand Command = new OleDbCommand();
Command.Connection = Connection;
OleDbParameter Parameter1 = new OleDbParameter();
Parameter1.OleDbType = OleDbType.VarChar;
Parameter1.ParamterName = "#name";
Parameter1.Value = "Bob";
OleDbParameter Parameter2 = new OleDbParameter();
Parameter2.OleDbType = OleDbType.VarChar;
Parameter2.ParamterName = "#dept";
Parameter2.Value = "ADept";
Command.Parameters.Add(Parameter1);
Command.Parameters.Add(Parameter2);
The problem I've got is, if I add the parameters to command the other way round, then the columns are populated with the wrong values (i.e. name is in the dept column and vice versa)
Command.Parameters.Add(Parameter2);
Command.Parameters.Add(Parameter1);
My question is, what is the point of the parameter names if parameters values are just inserted into the table in the order they are added command? The parameter names seems redundant?
The Problem is that OleDb (and Odbc too) does not support named parameters.
It only supports what's called positional parameters.
In other words: The name you give a parameter when adding it to the commands parameters list does not matter. It's only used internally by the OleDbCommand class so it can distinguish and reference the parameters.
What matters is the order in which you add the parameters to the list. It must be the same order as the parameters are referenced in the SQL statement via the question mark character (?).
But here is a solution that allows you to use named parameters in the SQL statement. It basically replaces all parameter references in the SQL statement with question marks and reorders the parameters list accordingly.
It works the same way for the OdbcCommand class, you just need to replace "OleDb" with "Odbc" in the code.
Use the code like this:
command.CommandText = "SELECT * FROM Contact WHERE FirstName = #FirstName";
command.Parameters.AddWithValue("#FirstName", "Mike");
command.ConvertNamedParametersToPositionalParameters();
And here is the code
public static class OleDbCommandExtensions
{
public static void ConvertNamedParametersToPositionalParameters(this OleDbCommand command)
{
//1. Find all occurrences of parameter references in the SQL statement (such as #MyParameter).
//2. Find the corresponding parameter in the commands parameters list.
//3. Add the found parameter to the newParameters list and replace the parameter reference in the SQL with a question mark (?).
//4. Replace the commands parameters list with the newParameters list.
var newParameters = new List<OleDbParameter>();
command.CommandText = Regex.Replace(command.CommandText, "(#\\w*)", match =>
{
var parameter = command.Parameters.OfType<OleDbParameter>().FirstOrDefault(a => a.ParameterName == match.Groups[1].Value);
if (parameter != null)
{
var parameterIndex = newParameters.Count;
var newParameter = command.CreateParameter();
newParameter.OleDbType = parameter.OleDbType;
newParameter.ParameterName = "#parameter" + parameterIndex.ToString();
newParameter.Value = parameter.Value;
newParameters.Add(newParameter);
}
return "?";
});
command.Parameters.Clear();
command.Parameters.AddRange(newParameters.ToArray());
}
}
Parameter NAMES are generic in the SQL support system (i.e. not OleDb specific). Pretty much ONLY OleDb / Odbc do NOT use them. They are there because OleDb is a specific implementation of the generic base classes.
I am trying to use the Entity Framework Core interpolated SQL query function in F# which requires a FormattableString. However to my surprise it doesn't function as I cannot find a way to convert a regular F# string to that type. I figured just doing what you do in C# would work but it doesn't. Here is the code I currently have:
let fromDbUser (u : Entity.User) =
{
name = u.Name
age = u.Age
phone = u.Phone
}
let mname = "Foo"
let ctx = new Entity.DatabaseContext()
ctx.User.FromSqlInterpolated($"Select * FROM User Where name = {mname};")
|> Seq.map(fromDbUser)
|> printfn "%A"
Running that block of code yields a compile error:
This token is reserved for future use
I have been trying to google around but I was unable to find any way to get this to work, any help would be most appreciated!
When this was asked, F# didn't have string interpolation.
Today though, there is an RFC for it that is merged in in F# 5.0 allowing string interpolation in F#.
The error was because the $ symbol is reserved (for 6 years+ as of writing), and will probably be used for string interpolation when it is added.
As Dave pointed out, interpolation isn't implemented yet.
But for methods that absolutely require an FormattableString or an IFormattable, you can use FormattableStringFactory.Create
let query (sql: FormattableString) =
printfn "%s" (sql.ToString(null, null))
let mname = "Foo"
let fstr = FormattableStringFactory.Create("Select * FROM User Where name = {0};", mname)
query fstr
It's available from F# 5.0.
> let mname = "Foo" ;;
val mname : string = "Foo"
> let str = $"Select * FROM User Where name = {mname};" ;;
val str : string = "Select * FROM User Where name = Foo;"
Check this out. https://learn.microsoft.com/en-us/dotnet/fsharp/whats-new/fsharp-50#string-interpolation
I'm trying to get data from an Azure SQL Server. I've Been able to get the data through this method:
let db = dbSchema.GetDataContext()
let serviceType = db.table
serviceType
I then go on to do some type casting where I get the actual data. But as my program has progressed I need a new way of getting the data.
I'm able to get a list of column names with this piece of code:
let columnList = (new SqlCommandProvider<"select * from Information_schema.Columns where table_name = #tableName",connectionString).Execute(tableName)
I'm wondering if there's a similar way to get the data.
I've tried:
let data = (new SqlCommandProvider<"select * from #tableName",connectionstring).Execute(tableName)
But I get this error: "Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of object. This may allow the lookup to be resolved."
I came up with this.
let GetData (tableName : string) =
let cn = new SqlConnection(connectionstring)
cn.Open()
let sQL = "select * from [" + tableName + "]"
let db = new SqlCommand(sQL, cn)
db.ExecuteReader()
From here you can access your data. So assign db.ExecuteReader() to a variable then...
let dataSource = db.ExecuteReader()
let mutable tableData = List.empty
while dataSource.Read() do
let rowLength = dataSource.FieldCount
let rowData = Array.zeroCreate(rowLength)
for i = 0 to dataSource.FieldCount-1 do
rowData.SetValue(dataSource.GetValue(i),i)
tableData <- rowData :: tableData
tableData |> List.toArray
This returns all the values in the table
This is one way (using the SqlClient type provider) to select all data from a table. If your table is too large it will return a lot. So I just select the Top 1000 rows. Replace tablenn with your choice of table name. You can parametrize the columns, but if you need to put the table name itself as a parameter you might need to use a Stored Procedure that takes a Table as a parameter (or use quotations in other type providers).
However, this is a string so it's very easy to build it. Then again, it has to be a literal.
#r #"..\packages\FSharp.Data.SqlClient.1.8.2\lib\net40\FSharp.Data.SqlClient.dll"
open FSharp.Data
open System
[<Literal>]
let connectionString = #"Data Source=(localdb)\MSSQLLocalDB;AttachDbFilename=C:\Users\userName\Documents\Test.sdf.mdf;Integrated Security=True;Connect Timeout=10"
[<Literal>]
let tblnn = "MyBigTable"
[<Literal>]
let qry = "SELECT TOP 1000 * FROM " + tblnn
let cmd = new SqlCommandProvider<qry, connectionString>(connectionString)
cmd.Execute() |> Seq.toArray
You can also use the stock SqlDataConnection type provider. It also makes it easy to select all data from a table.
I am working on access a database using F# and my initial attempt at creating a function to create the update query is flawed.
let BuildUserUpdateQuery (oldUser:UserType) (newUser:UserType) =
let buf = new System.Text.StringBuilder("UPDATE users SET ");
if (oldUser.FirstName.Equals(newUser.FirstName) = false) then buf.Append("SET first_name='").Append(newUser.FirstName).Append("'" ) |> ignore
if (oldUser.LastName.Equals(newUser.LastName) = false) then buf.Append("SET last_name='").Append(newUser.LastName).Append("'" ) |> ignore
if (oldUser.UserName.Equals(newUser.UserName) = false) then buf.Append("SET username='").Append(newUser.UserName).Append("'" ) |> ignore
buf.Append(" WHERE id=").Append(newUser.Id).ToString()
This doesn't properly put a , between any update parts after the first, for example:
UPDATE users SET first_name='Firstname', last_name='lastname' WHERE id=...
I could put in a mutable variable to keep track when the first part of the set clause is appended, but that seems wrong.
I could just create an list of tuples, where each tuple is oldtext, newtext, columnname, so that I could then loop through the list and build up the query, but it seems that I should be passing in a StringBuilder to a recursive function, returning back a boolean which is then passed as a parameter to the recursive function.
Does this seem to be the best approach, or is there a better one?
UPDATE:
Here is what I am using as my current solution, as I wanted to make it more generalized, so I just need to write an abstract class for my entities to derive from and they can use the same function. I chose to split up how I do the function so I can pass in how to create the SET part of the update so I can test with different ideas.
let BuildUserUpdateQuery3 (oldUser:UserType) (newUser:UserType) =
let properties = List.zip3 oldUser.ToSqlValuesList newUser.ToSqlValuesList oldUser.ToSqlColumnList
let init = false, new StringBuilder()
let anyChange, (formatted:StringBuilder) =
properties |> Seq.fold (fun (anyChange, sb) (oldVal, newVal, name) ->
match(oldVal=newVal) with
| true -> anyChange, sb
| _ ->
match(anyChange) with
| true -> true, sb.AppendFormat(",{0} = '{1}'", name, newVal)
| _ -> true, sb.AppendFormat("{0} = '{1}'", name, newVal)
) init
formatted.ToString()
let BuildUserUpdateQuery (oldUser:UserType) (newUser:UserType) (updatequery:UserType->UserType->String) =
let buf = StringBuilder("UPDATE users SET ");
buf.AppendFormat(" {0} WHERE id={1}", (updatequery oldUser newUser), newUser.Id)
let UpdateUser conn (oldUser:UserType) (newUser:UserType) =
let query = BuildUserUpdateQuery oldUser newUser BuildUserUpdateQuery3
execNonQuery conn (query.ToString())
Is this the tuple solution you had in mind?
let BuildUserUpdateQuery (oldUser:UserType) (newUser:UserType) =
let buf = StringBuilder("UPDATE users set ")
let properties =
[(oldUser.FirstName, newUser.FirstName, "first_name")
(oldUser.LastName, newUser.LastName, "last_name")
(oldUser.UserName, newUser.UserName, "username")]
|> Seq.map (fun (oldV, newV, field) ->
if oldV <> newV
then sprintf "%s='%s'" field newV
else null)
|> Seq.filter (fun p -> p <> null)
|> Seq.toArray
if properties.Length = 0
then None
else
bprintf buf "%s" (String.Join(", ", properties))
bprintf buf " where id=%d" newUser.Id
Some <| buf.ToString()
I don't see how the recursive solution could be simpler than this...
BTW I would strongly advise to use proper SQL parameters instead of just concatenating the values, you might become vulnerable to injection attacks...
Just for completeness, here is a version that does the same thing directly using the fold function. This can be done quite elegantly, because methods of StringBuilder return the StringBuilder (which allows you to chain them in C#). This can be also used nicely for folding.
Let's assume that we have the list of tuples from the solution by Mauricio:
let properties =
[ (oldUser.FirstName, newUser.FirstName, "first_name")
(oldUser.LastName, newUser.LastName, "last_name")
(oldUser.UserName, newUser.UserName, "username") ]
Now you can write the following code (it also returns a flag whether anything has changed):
let init = false, new StringBuilder()
let anyChange, formatted =
properties |> Seq.fold (fun (anyChange, sb) (oldVal, newVal, name) ->
if (oldVal = newVal) anyChange, sb
else true, sb.AppendFormat("{0} = '{1}'", name, newVal)) init
The state kept during folding has type bool * StringBuilder and we start with an initial value containing empty string builder and false. In each step, we either return the original state (if value is the same as previous) or a new state containing true and a new version of the StringBuilder returned by AppendFormat.
Using recursion explicitly would also work, but when you can use some built-in F# function, it is usually easier to use this approach. If you needed to process nested entities of each entity, you could use the Seq.collect function together with recursion to get a list of properties that you need to process using fold. Pseudo-code might look like this:
let rec processEntities list names =
// Pair matching entity with the name from the list of names
List.zip list names
|> List.collect (fun (entity, name) ->
// Current element containing old value, new value and property name
let current = (entity.OldValue, entity.NewValue, name)
// Recursively proces nested entitites
let nested = processEntities entity.Nested
current::nested)
This can be more elegantly written using sequence expressions:
let rec processEntities list =
seq { for entity, name in List.zip list names do
yield (entity.OldValue, entity.NewValue, name)
yield! processEntities entity.Nested }
Then you could simply call processEntities which returns a flat list of entities and process the entities using fold as in the first case.
I like both Mauricio's and Tomas's solutions, but perhaps this is more like what you originally envisioned?
let sqlFormat (value:'a) = //'
match box value with
| :? int | :? float -> value.ToString()
| _ -> sprintf "'%A'" value // this should actually use database specific escaping logic to make it safe
let appendToQuery getProp (sqlName:string) (oldEntity,newEntity,statements) =
let newStatements =
if (getProp oldEntity <> getProp newEntity) then (sprintf "%s=%s" sqlName (sqlFormat (getProp newEntity)))::statements
else statements
(oldEntity, newEntity, newStatements)
let createUserUpdate (oldUser:UserType) newUser =
let (_,_,statements) =
(oldUser,newUser,[])
|> appendToQuery (fun u -> u.FirstName) "first_name"
|> appendToQuery (fun u -> u.LastName) "last_name"
|> appendToQuery (fun u -> u.UserName) "username"
// ...
let statementArr = statements |> List.toArray
if (statementArr.Length > 0) then
let joinedStatements = System.String.Join(", ", statementArr)
Some(sprintf "UPDATE users SET %s WHERE ID=%i" joinedStatements newUser.ID)
else
None
If you have lots of properties to check, this may be a bit more concise. One benefit to this approach is that it works even if you're checking properties of multiple types, whereas the other approaches require all properties to have the same type (since they're stored in a list).