I am getting the error "Conversion failed when converting date and/or time from character string" when I'm trying to get my database versioned by EF migrations. The problem is the date string that EF generated ('2012-03-21T18:23:13.525Z') for new migration script entries is not supported by my version of MS Sql Server (Microsoft SQL Server Express Edition with Advanced Services (64-bit)). Is this a bug? Is there a work around?
CREATE TABLE [__MigrationHistory] (
[MigrationId] [nvarchar](255) NOT NULL,
[CreatedOn] [datetime] NOT NULL,
[Model] [varbinary](max) NOT NULL,
[ProductVersion] [nvarchar](32) NOT NULL,
CONSTRAINT [PK___MigrationHistory] PRIMARY KEY ([MigrationId])
)
BEGIN TRY
EXEC sp_MS_marksystemobject '__MigrationHistory'
END TRY
BEGIN CATCH
END CATCH
INSERT INTO [__MigrationHistory] ([MigrationId], [CreatedOn], [Model], [ProductVersion])
VALUES ('201203210144184_init', '2012-03-21T18:23:13.525Z', 0x33, '4.3.1');
EDIT
Nevermind. This has nothing to do with EntityFramework. If my SQL Server database is configured to be backwards compatible to SQL Server 2000, it won't accept that date format. I guess unless I can get EF to output it's date string in another format (or get SQL Server to both be backwards compatable to 2000 and still understand the EF date string), I won't be able to use EF migrations with my database :-(. Please let me know if someone has worked out a way to use EF with a database with compatibility level SQL Server 2000.
If you still want to pass in a Date you can use the defaultSqlValue optional parameter with a valid SQL DateTime. For example:
public partial class AddTransactionDate : DbMigration
{
public override void Up()
{
// defaultValueSql is where the magic happens.
AddColumn("dbo.StorageTransaction", "Created", c => c.DateTime(nullable: false, defaultValueSql: DateTimeToSql(DateTime.UtcNow)));
}
public override void Down()
{
DropColumn("dbo.StorageTransaction", "Created");
}
private static string DateTimeToSql(DateTime dt)
{
// Use the .Net SqlDateTime class to create a valid
// SQL server DATETIME. We also need to wrap it in quotes
// because EF prints out your values verbatim (e.g. you could
// also use GETUTCDATE() etc.).
return string.Format("N'{0}'",
new System.Data.SqlTypes.SqlDateTime(dt).ToSqlString());
}
}
And voila:
ALTER TABLE [dbo].[StorageTransaction] ADD [Created] [datetime] NOT NULL DEFAULT N'2012-11-26 03:06:09 PM'
INSERT [dbo].[__MigrationHistory]([MigrationId], [ContextKey], [Model], [ProductVersion])
VALUES ('201211261300214_AddTransactionDate', 'Symblr.Migrations.Configuration',
Just so this question has an answer, the answer is:
Switch the SQL Server database compatibility mode from SQL Server 2000 to at minimum SQL Server 2005 and the issue with the date string parsing will go away.
Related
I have the following method that uses a NamedParameterJdbcTemplate to execute an SQL query in Spring Boot:
#Service
class MyRepository(
val jdbcTemplate: NamedParameterJdbcTemplate
) {
fun loadData(myKey: List<Int>): List<MyRow> {
return jdbcTemplate.query(
"""
select
io.KEY as itemKey,
art.ARTICLE_NR as articleNumber,
art.PRICE as price,
concat(
concat(
concat(art.BEST_B, BEST_A),
lpad(BEST_B, 2, '0')),
lpad(BEST_A, 2, '0')) as group
from
BUY.OPTION io
INNER JOIN BUYING.ART art ON (to_char(art.id) = io.keyb)
where
io.KEY IN (:MYKEY)
""".trimIndent(),
parameters
) { rs, rowNum ->
MyRow(
itemOption = ItemOption(rs.getString("ITEMOPTION")),
articleNumber = rs.getString("ARTICLENUMBER"),
price = rs.getBigDecimal("PRICE")
group = rs.getString("GROUP")
)
}
}
}
Is this method already protected against SQL injection since it's using NamedParameterJdbcTemplate? Or do I have to do some extra steps for that?
Using parameters in a NamedParameterJdbcTemplate will use JDBC prepared statement with parameters, which will - in general - protect you against SQL injection.
I'm saying "in general", because the actual protection depends on the implementation of the specific JDBC driver used. Mainstream JDBC drivers will - bar any bugs - protect you against SQL injection because they either keep statements and parameter values separate (the statement is prepared with parameter placeholders, and on execute only the values are sent), or otherwise properly escape things when generating the actual query (i.e. default behaviour in the MySQL Connector/J driver). However, nothing keeps someone from writing a naive JDBC driver that will use string interpolation to generate a query without actually preventing SQL injection.
I would like to create a bunch of SQL views at my Quarkus application startup and I wrote
#Inject
EntityManager em;
public void exec(... sql) {
logger.info("EXECUTING " ...);
sql = "BEGIN\n" + sql + "\nEND";
Query q = em.createNativeQuery(sql);
q.executeUpdate();
}
And my SQL file is following:
CREATE OR REPLACE VIEW My_view_ex AS
SELECT cc.*, If(cc.beautifulName IS NULL, cc.name, cc.beautifulName) canonName FROM My_table cc;
CREATE OR REPLACE VIEW MyView2 AS
...
Unfortunately, it throws exception, saying
java.sql.SQLException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CREATE OR REPLACE VIEW My_view_ex AS
SELECT cc.*, If(cc.beautifulName IS NU...' at line 2
If I copy paste the content of sql varible to database client and execute it, then it works, i.e. there is no actually syntax error. Probably it dislikes multiple queries or line ends or something.
What else can be suggested?
I have relatively large table with 7,517,717 records in my Azure SQL DB (which is on Azure V12 server, and it is on P4 pricing tier).
When I created table I forgot to define the MaxLength for a property, so now it is of type nvarchar(max).
I wanted to change this so I created migration (using FluentAPI) which looks like this:
public partial class FeedTitleMaxLength : DbMigration
{
public override void Up()
{
AlterColumn("dbo.Feed", "Title", c => c.String(maxLength: 100));
}
public override void Down()
{
AlterColumn("dbo.Feed", "Title", c => c.String());
}
}
I've run Update-Database command like this:
Update-Database -Verbose -ConnectionString "Data Source=serverUrl,1433;Database=dbName;User ID=myusername;Password=mypassword;Trusted_Connection=False;Encrypt=True;Connection Timeout=3000;MultipleActiveResultSets=True;" -ConnectionProviderName "System.Data.SqlClient"
My connection string in web.config looks the same. And I've also defined timeout like this:
this.Database.CommandTimeout = 3000;
But no matter what I do or try I get this error after 30/40 seconds:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. This failure occurred while attempting to connect to the routing destination. The duration spent while attempting to connect to the original server was - [Pre-Login] initialization=97; handshake=110; [Login] initialization=0; authentication=0; [Post-Login] complete=32;
Is there any way I can run this migration? Or some workaround? Can I, for example, create migration but run the AlterColumn command in SQL and then Update-Database command. Would this mess things or not?
I have a problem with JPA and SQlite.
I have created an Entity from Table. My generated Entity looks like:
#Entity
#Table(name="sqliteTestTable")
public class Test implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column(name="id")
private int id;
#Column(name="name")
private String name;
public Test() {
}
------
}
When i try to persist a few Test objects i get following error:
(I have executed the same code on mysql without problems)
Exception in thread "main" Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.4.1.v20121003-ad44345): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: no such table: SEQUENCE
Error Code: 0
Call: UPDATE SEQUENCE SET SEQ_COUNT = SEQ_COUNT + ? WHERE SEQ_NAME = ?
bind => [50, SEQ_GEN_TABLE]
Query: DataModifyQuery(name="SEQ_GEN_TABLE" sql="UPDATE SEQUENCE SET SEQ_COUNT = SEQ_COUNT + ? WHERE SEQ_NAME = ?")
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
How can i solve this problem?
IDENTITY will default to TABLE if IDENTITY is not supported in the database platform.
For TABLE you need to have a sequence table created. If you have ElipseLink create your tables it will automatically be created for you, otherwise you need to create it yourself.
If SQlite has IDENTITY support, then you can create your own DatabasePlatform subclass that adds the identity support.
In general I would never recommend using IDENTITY, it does not support pre-allocation and will lead to poor performance.
sqlite uses the table sqlite_sequence for AUTOINCREMENT field
replace:
#GeneratedValue(strategy=GenerationType.IDENTITY)
with
#GeneratedValue(generator="sqlite")
#TableGenerator(name="sqlite", table="sqlite_sequence",
pkColumnName="name", valueColumnName="seq",
pkColumnValue="sqliteTestTable")
If you are using multiple tables then the generators have to be unique names. With a new table, there is no sqlite_sequence record so you have to create yourself.
insert into sqlite_sequence (name,seq) values ("sqliteTestTable",1);
EDIT:
The solution was to make a view that mirrored the table in question and converted the date to varchar and then convert it back to date with a matching collation
END OF EDIT
Can anyone tell me why OData4j reads datetime values fine from one of my WCF Data Service server but runs into an illegal argument exception (bad valueString as part of keyString) when reading exact same datetime type with same format from another WCF Data Service?
java.lang.IllegalArgumentException: bad valueString
[datetime'2012-01-24T14%3A57%3A22.243'] as part of keyString
The other problem is that when I ask for a JSON response from the service from which OData4j had no problem reading datetime types I get another illegal argument exception and the error message is - Odd number of characters.
java.lang.IllegalArgumentException:
org.odata4j.repack.org.apache.commons.codec.DecoderException: Odd
number of characters.
Because WCF Data Services can't have multiple sources, I've made 2 projects with each it's own Entity Data Model source (from existing database). And like I mentioned above, I'm getting these annoying errors.
To conclude...
Example 1: bad valueString as part of keyString - when reading datetime. Also happens with FormatType.JSON.
ODataConsumer customerInfoServices = ODataConsumer
.newBuilder("http://10.0.2.2:41664/CustomerInfoWCFDataServices.svc/")
.setFormatType(FormatType.ATOM)
.build();
customer = customerInfoServices
.getEntities("Customers")
.select("name, id")
.filter("id eq " + 5)
.execute()
.firstOrNull();
Example 2: Odd number of characters. Only happends with FormatType.JSON and no problem reading datetime.
ODataConsumer businessServices = ODataConsumer
.newBuilder("http://10.0.2.2:35932/BusinessWCFDataServices.svc/")
.setFormatType(FormatType.JSON)
.build();
Enumerable<?> ordrer = businessServices
.getEntities("Orders")
.filter("custId eq " + customer.getProperty("id").getValue())
.execute();
What I want is to receive JSON responses (ATOM is still too bloaded for android) and no problems reading datetime properties.
No body able to help me?
I've been wearing my fingers down trying to find a solution on Google, without any luck.
The collation on the database without datetime problems is "Danish_Norwegian_CI_AS" and on the database with reading errors is "SQL_Danish_Pref_CP1_CI_AS". I don't know if this has any meaning but I have a suspescion it has something to do with it.
The solution was to make a view that mirrored the table in question and converted the date to varchar and then convert it back to date with a matching collation. :-)
I ran into this issue today, and after running around for about three hours Googling and looking at code I managed to figure out what is going on. Here's my setup/situation and what I found:
The Setup
(OData Service) Microsoft IIS 8.0 on Windows Server 2012 using
default application pool.
(OData Producer) Microsoft WCF middle-tier
using Entity Framework and Web Data Services.
(OData Consumer)
Android client using OData4J v0.8 SNAPSHOT.
The Problem
In my Middle-tier (the OData Producer) I am using Entity Framework 5.0 to define a simple table with an Edm.DateTime column. My MessageTable.edmx file generates a simple table:
CREATE TABLE [dbo].[MessageTable] (
[Id] int IDENTITY(1,1) NOT NULL,
[Date1] datetime NULL
);
In my middle-tier's WCF data service (OData Producer) I intercept the OData POST from my Android client application. I manually set the Date1 column in the middle-tier using some C# code:
private static void ProcessMessage(ODataMessage message)
{
.
.
.
conn.Open();
cmd.ExecuteNonQuery();
int returnCode = (int)cmd.Parameters["#result"].Value;
if (returnCode == 0)
{
message.Processed = true;
message.Date1 = DateTime.Now;
}
.
.
.
}
Notice that I used C#'s DateTime.Now static property. The MSDN documentation states that DateTime.Now:
Gets a DateTime object that is set to the current date and time on this computer, expressed as the local time.
The key thing to realize is that local time means the C# date-time structure now includes local time zone information.
So, after the middle-tier code finishes, the WCF service inserts the OData POST into my table. Microsoft then sends the [Date1] column back to my Android client (the OData Consumer). It appears that Microsoft encodes the date as an OData DateTimeOffset because it includes local time-zone information. I.e. as 2013-08-26T17:30:00.0000000-7:00
The Code
In OData4j there is a package org.odata.internal that handles parsing OData date-time strings. In version 0.6 I found the following comment on lines 40-44 about the DATETIME_PATTERN regex pattern used to parse date-time strings:
40 // Since not everybody seems to adhere to the spec, we are trying to be
41 // tolerant against different formats
42 // spec says:
43 // Edm.DateTime: yyyy-mm-ddThh:mm[:ss[.fffffff]]
44 // Edm.DateTimeOffset: yyyy-mm-ddThh:mm[:ss[.fffffff]](('+'|'-')hh':'mm)|'Z'
45 private static final Pattern DATETIME_PATTERN =
46 Pattern.compile("(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2})(:\\d{2})?(\\.\\d{1,7})?((?:(?:\\+|\\-)\\d{2}:\\d{2})|Z)?");
47
48
In OData4j v0.7 the DATETIME_PATTERN has become DATETIME_XML_PATTERN:
40
41 private static final Pattern DATETIME_XML_PATTERN = Pattern.compile("" +
42 "^" +
43 "(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2})" + // group 1 (datetime)
44 "(:\\d{2})?" + // group 2 (seconds)
45 "(\\.\\d{1,7})?" + // group 3 (nanoseconds)
46 "(Z)?" + // group 4 (tz, ignored - handles bad services)
47 "$");
48
49 private static final Pattern DATETIMEOFFSET_XML_PATTERN = Pattern.compile("" +
50 "^" +
51 "(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2})" + // group 1 (datetime)
52 "(\\.\\d{1,7})?" + // group 2 (nanoSeconds)
53 "(((\\+|-)\\d{2}:\\d{2})|(Z))" + // group 3 (offset) / group 6 (utc)
54 "$");
I think the comment on line 46 explains everything:
... // group 4 (tz, ignored - handles bad services)
I interpret this as saying 'Any time zone information is going to be ignored - this handles bad services (i.e. Microsoft) that send time zone info'
It appears to me that the OData4j authors have decided to stick to their guns and only accept the correct Edm.DateTime string format.
The Solution
The fix is very simple if you have access to the OData Producer code:
private static void ProcessMessage(ODataMessage message)
{
.
.
.
conn.Open();
cmd.ExecuteNonQuery();
int returnCode = (int)cmd.Parameters["#result"].Value;
if (returnCode == 0)
{
message.Processed = true;
message.Date1 = DateTime.UtcNow;
}
.
.
.
}
Use the DateTime.UtcNow property instead. The MSDN documentation states:
Gets a DateTime object that is set to the current date and time on this computer, expressed as the Coordinated Universal Time (UTC).
If you do not have access to the OData producer, the only solution I can see is to change the OData4j code. Alter the regex pattern for DATETIME_XML_PATTERN:
41 private static final Pattern DATETIME_XML_PATTERN = Pattern.compile("" +
42 "^" +
43 "(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2})" + // group 1 (datetime)
44 "(:\\d{2})?" + // group 2 (seconds)
45 "(\\.\\d{1,7})?" + // group 3 (nanoseconds)
46 "((?:(?:\\+|\\-)\\d{2}:\\d{2})|Z)?" +
47 "$");
Conclusion
I think this is actually a mistake on Microsoft's part. I'm sure they have some involved justification for sending an Edm.DateTimeOffset but my MessageTable.edmx file specifies [Date1] as an Edm.DateTime, so I think that the Microsoft code should either do the conversion to UTC for me, or throw an exception. An exception would have warned me that I was using an incorrect DateTime structure and helped me see the DateTime.UtcNow solution much quicker.