I want to be able to post something like this
{
...
"foo": "http://localhost:8081/foos/2",
...
}
but want to store only the 2 as a Long in the database. Therefore, I created a JsonDeserializer and declared the field like this:
#JsonDeserialize(using = RestResourceURLSerializer.class)
private Long foo;
The RestResourceURLSerializer looks like this: (not really important, I think, but to give a complete picture)
public class RestResourceURLSerializer extends JsonDeserializer<Long> {
#Override
public Long deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
URL url = p.readValueAs(URL.class);
String path = url.getPath();
return Long.parseLong(path.substring(path.lastIndexOf('/') + 1));
}
}
My problem is that I get this error message posting an entity: https://pastebin.com/JszRnaCc
Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.lang.Long` from String "http://localhost:8081/foos/2": not a valid Long value
at [Source: (org.apache.catalina.connector.CoyoteInputStream); line: 1, column: 2935] (through reference chain: com.demo.Bar["foo"])
Interestingly, I have a different entity in the same project that does the same thing and is not having these issues. Any ideas how to troubleshoot?
Related
I'm working on a generic code to add .NET 6 Entity Framework DbSet<...> records, deserialized from JSON strings. The original code is (much) more elaborated, below are just samples to demonstrated the issue - the following method:
public static void AddRecord(dynamic dbSet, Type entityType, string json)
{
var dataRecord = System.Text.Json.JsonSerializer.Deserialize(json, entityType);
dbSet.Add(dataRecord);
}
results in a run-time error at dbSet.Add(dataRecord) call:
"The best overloaded method match for
'Microsoft.EntityFrameworkCore.DbSet<Northwind.Models.Category>.Add
(Northwind.Models.Category)' has some invalid arguments"}
...
This exception was originally thrown at this call stack:
System.Dynamic.UpdateDelegates.UpdateAndExecuteVoid2<T0, T1>
(System.Runtime.CompilerServices.CallSite, T0, T1)
if you call it, e.g., this way:
using (var ctx = ...)
{
...
var json = ...
...
AddRecord(ctx.Categories, typeof(Category), json);
}
I have intentionally, for clarity, used in the above code concrete dbSet (ctx.Categories) and compile time typedef (typeof(Category)) - in actual code these are run-time defined variables.
If you "unroll" the method code and write it this way:
using (var ctx = ...)
{
...
var json = ...
...
var dataRecord = System.Text.Json.JsonSerializer.Deserialize(json, typeof(Category));
ctx.Categories.Add(dataRecord);
}
you would still get the mentioned above run-time error for the .Add method.
But if you write:
var dataRecord = System.Text.Json.JsonSerializer.Deserialize<Category>(json);
ctx.Categories.Add(dataRecord);
or
var dataRecord = System.Text.Json.JsonSerializer.Deserialize(json);
ctx.Categories.Add((Category)dataRecord);
the code will work without any issues.
Finally, an attemp to use Convert.ChangeType doesn't help:
var dataRecord = System.Text.Json.JsonSerializer.Deserialize(json);
ctx.Categories.Add(Convert.ChangeType(dataRecord, typeof(Category)));
So, it looks like an explicit object type casting is compiled to and makes on rum-time some "special object interfaces arrangements", which dynamic object type casting doesn't?
[Update]
Okan Karadag's prompt answer below gave me a hint how to change AddRecord(...) method to workaround the subject issue:
public static void AddRecord(DbContext dbContext, Type entityType, string json)
{
var dataRecord = System.Text.Json.JsonSerializer.Deserialize(json, entityType);
dbContext.Add(dataRecord);
}
This method works flawlessly. Although it doesn't answer the subject question, why the original AddRecord(...) method
public static void AddRecord(dynamic dbSet, Type entityType, string json)
{
var dataRecord = System.Text.Json.JsonSerializer.Deserialize(json, entityType);
dbSet.Add(dataRecord);
}
results in 'The best overloaded method match for 'Microsoft.EntityFrameworkCore.DbSet<...>.Add(...)' has some invalid arguments...' runtime error at
dbSet.Add(dataRecord);
code line.
You can use generic for dynamic.
public void AddEntity<T>(string json) where T:class
{
var entity = JsonSerializer.Deserialize<T>(json);
ArgumentNullException.ThrowIfNull(entity);
dbContext.Add<T>(entity);
dbContext.SaveChanges();
}
You can see link for problem error.
Based on the EF Core docs (https://learn.microsoft.com/en-us/ef/core/modeling/value-conversions), I'm trying to use EF Core's new Value Conversion on an enumeration. I want to save the enumeration as a string in the SQL Database table.
Here's the entity and enumeration.
public enum InputSetType
{
TypeA, TypeB
}
public class MonthlyInputSet
{
public int Id { get; set; }
public InputSetType Type { get; set; }
}
Here is where I configure the MonthlyInputSet Entity:
public class MonthlyInputSetConfiguration : IEntityTypeConfiguration<MonthlyInputSet>
{
public void Configure(EntityTypeBuilder<MonthlyInputSet> builder)
{
builder.Property(mis => mis.Type).HasConversion(v => v.ToString(), v => (InputSetType)Enum.Parse(typeof(InputSetType), v));
}
}
So, I try to run a basic query to get this data and it fails. The query is:
var saved = await _context.MonthlyInputSets.Include(mis => mis.InsertedBy)
.Include(mis => mis.UpdatedBy)
.Include(mis => mis.MonthlyInputs)
.ThenInclude(mi => mi.EmissionsUnit)
.FirstOrDefaultAsync(mis => mis.Id == id);
But, an error is thrown on the first line of this query that says, "ArgumentException: Must specify valid information for parsing in the string." So my guess is that I have not properly configured the conversion of the string in the table to the enum in C#.
Full raw stack trace:
I verified that the correct string value is being returned from the database. It is not null and it is not a blank string. The string value returned matches a member of the enum perfectly.
public void Configure(EntityTypeBuilder<MonthlyInputSet> builder)
{
builder.Property(mis => mis.Type).HasConversion(
convertToProviderExpression: v => v.ToString(),
convertFromProviderExpression: v => Troubleshooting(v)
);
}
private InputSetType Troubleshooting(string v)
{
return (InputSetType)Enum.Parse(typeof(InputSetType), v);
}
these two images show that the text in every row in the database type column is identical to the 2nd member of the enum that is mapped to this field.
The only way I can reproduce this error is for v to be an empty string, ie "". Not a null.
This :
Enum.Parse(typeof(InputSetType),"xx")
Throws Requested value 'xx' was not found..
While this:
Enum.Parse(typeof(InputSetType),"")
Throws :
Must specify valid information for parsing in the string. (Parameter 'value')
It seems some rows contain an empty string instead of null.
You'll have to decide what to do with those values - are they bad data? Or should the application handle them and replace them with some default value?
If you decide to use a default value you could use String.IsEmptyorString.IsNullOrWhitespace` to handle them, eg:
v=> String.NullOrWhitespace(v)?InputSetType.Unknown:Enum.Parse(typeof(InputSetType),v)
Probably this exception is caused by invalid data in the database - data, that is not present in the enum, null, empty string or so on. This error is common in Enum.Parse() so I think it is related to this method.
When creating a Dataprovider that returns Iterator I have it in my test method, but my intellij-idea marks this return type as invalid and shows the message:
"Data provider must return either Object[][] or Iterator[], or Iterator".
Here is my class/ method:
public class TradeTestDataProvider {
#DataProvider(name = "experimental")
public Iterator<TestCase> createCases() throws IOException {
List<TestCase> test = DataReader.generateCasesFromJson("src/test/resources/json/experimental_test_case");
return test.iterator();
}
}
Please advise, if I am missing something or it is related to TestNG/IDE issue?
Update:
I created a post to discuss this issue with plugin:
topic
I'm using p6spy to log the sql statements generated by my program. The format for the outputted spy.log file looks like this:
current time|execution time|category|statement SQL String|effective SQL string
I'm just wondering if anyone knows if there's a way to alter the spy.properties file and have only the last column, the effective SQL string, output to the spy.log file? I've looked through the properties file but haven't found anything that seems to support this.
Thanks!
In spy.properties there is a property called logMessageFormat that you can set to a custom implementation of MessageFormattingStrategy. This works for any type of logger (i.e. file, slf4j etc.).
E.g.
logMessageFormat=my.custom.PrettySqlFormat
An example using Hibernate's pretty-printing SQL formatter:
package my.custom;
import org.hibernate.jdbc.util.BasicFormatterImpl;
import org.hibernate.jdbc.util.Formatter;
import com.p6spy.engine.spy.appender.MessageFormattingStrategy;
public class PrettySqlFormat implements MessageFormattingStrategy {
private final Formatter formatter = new BasicFormatterImpl();
#Override
public String formatMessage(int connectionId, String now, long elapsed, String category, String prepared, String sql) {
return formatter.format(sql);
}
}
There is no such option provided to achieve it via configuration only yet. I think you have 2 options here:
fill a new bug/feature request report (which could bring benefit to others using p6spy as well) on: https://github.com/p6spy/p6spy/issues?state=open or
provide custom implementation.
For the later option, I believe you could achieve it via your own class (depending on the logger you use, let's assume you use Log4jLogger).
Well, if you check relevant part of the Log4jLogger github as well as sourceforge version, your implementation should be rather straightforward:
spy.properties:
appender=com.EffectiveSQLLog4jLogger
Implementation itself could look like this:
package com;
import com.p6spy.engine.logging.appender.Log4jLogger;
public class EffectiveSQLLog4jLogger extends Log4jLogger {
public void logText(String text) {
super.logText(getEffectiveSQL(text));
}
private String getEffectiveSQL(String text) {
if (null == text) {
return null;
}
final int idx = text.lastIndexOf("|");
// non-perfect detection of the exception logged case
if (-1 == idx) {
return text;
}
return text.substring(idx + 1); // not sure about + 1, but check and see :)
}
}
Please note the implementation should cover github (new project home, no version released yet) as well as sourceforge (original project home, released 1.3 version).
Please note: I didn't test the proposal myself, but it could be a good starting point and from the code review itself I'd say it could work.
I agree with #boberj, we are used to having logs with Hibernate formatter, but don't forget about batching, that's why I suggest to use:
import com.p6spy.engine.spy.appender.MessageFormattingStrategy;
import org.hibernate.engine.jdbc.internal.BasicFormatterImpl;
import org.hibernate.engine.jdbc.internal.Formatter;
/**
* Created by Igor Dmitriev on 1/3/16
*/
public class HibernateSqlFormatter implements MessageFormattingStrategy {
private final Formatter formatter = new BasicFormatterImpl();
#Override
public String formatMessage(int connectionId, String now, long elapsed, String category, String prepared, String sql) {
if (sql.isEmpty()) {
return "";
}
String template = "Hibernate: %s %s {elapsed: %sms}";
String batch = "batch".equals(category) ? ((elapsed == 0) ? "add batch" : "execute batch") : "";
return String.format(template, batch, formatter.format(sql), elapsed);
}
}
In p6Spy 3.9 this can be achieved quite simply. In spy.properties set
customLogMessageFormat=%(effectiveSql)
You can patch com.p6spy.engine.spy.appender.SingleLineFormat.java
removing the prepared element and any reference to P6Util like so:
package com.p6spy.engine.spy.appender;
public class SingleLineFormat implements MessageFormattingStrategy {
#Override
public String formatMessage(final int connectionId, final String now, final long elapsed, final String category, final String prepared, final String sql) {
return now + "|" + elapsed + "|" + category + "|connection " + connectionId + "|" + sql;
}
}
Then compile just the file
javac com.p6spy.engine.spy.appender.SingleLineFormat.java
And replace the existing class file in p6spy.jar with the new one.
In HBase database I want to create a secondary index by using additional "linking" table. I have followed the example given in this answer: Create secondary index using coprocesor HBase
I am not very well familiar with the entire concept of HBase, and I had read some examples on the issue of creating secondary indexes. I am attaching the coprocessor to single table only, like this:
disable 'Entry2'
alter 'Entry2', METHOD => 'table_att', 'COPROCESSOR' => '/home/user/hbase/rootdir/hcoprocessors.jar|com.acme.hobservers.EntryParentIndex||'
enable 'Entry2'
The source code of it, is as follows:
public class EntryParentIndex extends BaseRegionObserver{
private static final Log LOG = LogFactory.getLog(CoprocessorHost.class);
private HTablePool pool = null;
private final static String INDEX_TABLE = "EntryParentIndex";
private final static String SOURCE_TABLE = "Entry2";
#Override
public void start(CoprocessorEnvironment env) throws IOException {
pool = new HTablePool(env.getConfiguration(), 10);
}
#Override
public void prePut(
final ObserverContext<RegionCoprocessorEnvironment> observerContext,
final Put put,
final WALEdit edit,
final boolean writeToWAL)
throws IOException {
try {
final List<KeyValue> filteredList = put.get(Bytes.toBytes ("data"),Bytes.toBytes("parentId"));
byte [] id = put.getRow(); //Get the Entry ID
KeyValue kv=filteredList.get( 0 ); //get Entry PARENT_ID
byte[] parentId=kv.getValue();
HTableInterface htbl = pool.getTable(Bytes.toBytes(INDEX_TABLE));
//create row key for the index table
byte[] p1=concatTwoByteArrays(parentId, ":".getBytes()); //Insert a semicolon between two UUIDs
byte[] rowkey=concatTwoByteArrays(p1, id);
Put indexput = new Put(rowkey);
//The following call is setting up a strange? recursion, resulting
//...in thesame prePut method invoken again and again. Interestingly
//...the recursion limits itself up to 6 times. The actual row does not
//...get inserted into the INDEX_TABLE
htbl.put(indexput);
htbl.close();
}
catch ( IllegalArgumentException ex) { }
}
#Override
public void stop(CoprocessorEnvironment env) throws IOException {
pool.close();
}
public static final byte[] concatTwoByteArrays(byte[] first, byte[] second) {
byte[] result = Arrays.copyOf(first, first.length + second.length);
System.arraycopy(second, 0, result, first.length, second.length);
return result;
}
}
This executes when I perform put on the SOURCE_TABLE.
There is a comment in the code (please seek it): "The following call is setting up a strange".
I set a debugging print in the log confirming that the prePut method is being executed only on the SOURCE_TABLE, and never on the INDEX_TABLE. Yet I don't understand why this strange recursion is happening despite in the coprocessor I only execute one put on the INDEX_TABLE.
I have also confirmed that the put action on the source table is again only one.
I have fixed my problem. It came out to be that I was adding multiple times the same observer mistakenly thinking that it is getting lost after Hbase restart.
Also the reason why the .put call to the INDEX_TABLE was not working is because I did not set a value to it, but only a rowkey, mistakenly thinking that this is possible. Yet HBase did not throw any excepiton whatsoever, just did not perform the PUT, no info given, which may be confusing for newcommers to this technology.