my first SO post !
I'm trying to set a Stempel Analyzer (ES analyzer for polish language) for a string field. I can do it through PUT request:
{
"doc": {
"_source": {
"enabled": false
},
"properties": {
"file": {
"type": "attachment",
**"analyzer": "polish"**,
"fields": {
"content": {
"type": "string",
"term_vector": "with_positions_offsets"
}
}
}
}
}
}
and it works fine. Trying to do the same thing through NEST.
[ElasticProperty(Name = "_content", TermVector = TermVectorOption.WithPositionsOffsets, Analyzer = "polish")]
public string Content { get; set; }
is not working neither do:
client.CreateIndex(index, b => b.AddMapping<DocInES>(m => m
.MapFromAttributes()
.Properties(props => props
.String(s => s
.Name(p => p.File.Content)
.Analyzer("polish")
))));
When I'm using
var result = client.Analyze(a => a.Index("doc").Analyzer("polish").Text("...text..."));
It works fine, so .NET is detecting this analyzer.
I'm using ES 2.1.1. &
NEST 1.7.1
EDIT:
from what I inspected it seems that NEST is not doing Mapping of Attributes of Attachment class created in .NET. It does Map Attributes of Document class
[ElasticType(Name = "docInES")]
public class DocInES {
public int InstitutionId { get; set;}
public int DocumentId { get; set; }
[ElasticProperty(Store = true, Analyzer = "polish")]
public string Title { get; set; }
[ElasticProperty(Type = FieldType.Attachment)]
public Attachment File { get; set; }
}
But not the Attachment class:
public class Attachment {
[ElasticProperty(Name = "content2", Store = true)]
public string Content { get; set; }
[ElasticProperty(Name = "content_type2")]
public string ContentType { get; set; }
[ElasticProperty(Name = "name2", Analyzer = "english")]
public string Name { get; set; }
}
You should probably check the compatibility matrix on Github.
Nest 1.7.1 is not compatible with ES 2.1.1
Related
I've been searching ways to seed data on a .Net Core 3.1 MVC app. I've found many samples, starting from the documentation. What I haven't found was good, real-life full-examples using (Json/XML) file. The closest one I found doesn't show how to properly get a (Json/XML) file's location no matter the platform used. I haven't been able to successfully integrate DI solutions found on other stack overflow questions makes me think it's not possible. Hoping someone can help.
Thanks!
The closest one I found doesn't show how to properly get a (Json/XML) file's location no matter the platform used.
If your json file exists in the physical disks,you just use the original physical location:
using (StreamReader r = new StreamReader(#"C:\xxx\xxx\test.json"))
If your json file exists in the project and in the root project,use the following location:
using (StreamReader r = new StreamReader(#"test.json"))
If it exists in the folders in the project,you could use the location like below:
using (StreamReader r = new StreamReader(#"wwwroot/test.json"))
And another thing you need to know,althogh you have multiple models with relationships,be sure the json should contain only one model's data.
Here is a example code:
1.Model:
public class Test
{
public int Id { get; set; }
public string Name { get; set; }
public int UserId { get; set; }
public List<User> User { get; set; }
}
public class User
{
public int Id { get; set; }
public int Age { get; set; }
}
2.DbContext:
public class YourDbContext : DbContext
{
public YourDbContext(DbContextOptions<YourDbContext> options)
: base(options)
{
}
public DbSet<Test> Test { get; set; }
public DbSet<User> User { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Test>().HasData(SeedTestData());
}
public List<Test> SeedTestData()
{
var tests = new List<Test>();
using (StreamReader r = new StreamReader(#"test.json"))
{
string json = r.ReadToEnd();
tests = JsonConvert.DeserializeObject<List<Test>>(json);
}
return tests;
}
public List<User> SeedUserData()
{
var users = new List<User>();
using (StreamReader r = new StreamReader(#"test2.json"))
{
string json = r.ReadToEnd();
users = JsonConvert.DeserializeObject<List<User>>(json);
}
return users;
}
}
3.Run following command in Package Manager Console:
>PM Add-Migration init
>PM Update-Database
test.json:
[
{
"id": 1,
"name": "aa"
},
{
"id": 2,
"name": "bb"
},
{
"id": 3,
"name": "cc"
}
]
test2.json:
[
{
"id": 1,
"age": 34
},
{
"id": 2,
"age": 21
}
]
I am calling the API and trying to store the Jsonresult into a IEnumerable model class. The json result has group header column consolidated_weather.
when I run the program the following error is coming
JsonSerializationException: Cannot deserialize the current JSON object into type 'System.Collections.Generic.IEnumerable`.
How can I call the given json result into the IEnumerable model class after avoiding group header from json result.
I am using the following two model.
public class Weather
{
[Key]
public long id { get; set; }
public string weather_state_name { get; set; }
public string weather_state_abbr { get; set; }
public string wind_direction_compass { get; set; }
public DateTime created { get; set; }
public DateTime applicable_date { get; set; }
public decimal min_temp { get; set; }
public decimal max_temp { get; set; }
public decimal the_temp { get; set; }
public double wind_speed { get; set; }
public decimal wind_direction { get; set; }
public decimal air_pressure { get; set; }
public decimal Humidity { get; set; }
public string Visibllity { get; set; }
public decimal Predictability { get; set; }
}
public class WeatherList
{
public IEnumerable<Weather> consolidated_weather { get; set; }
}
public async Task<IEnumerable<Weather>> GetWeatherAsync(string woied)
{
var url = SD.APIBaseUrl + woied;
var request = new HttpRequestMessage(HttpMethod.Get, url);
var client = _clientFactory.CreateClient();
HttpResponseMessage response = await client.SendAsync(request);
IEnumerable<Weather> weather = new List<Weather>();
WeatherList weatherList = new WeatherList();
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
var jsonString = await response.Content.ReadAsStringAsync();
weatherList = JsonConvert.DeserializeObject<WeatherList>(jsonString);
return weatherList;
}
return null;
}
The API result is coming as
<!-- language: lang-html -->
{
"consolidated_weather": [
{
"id": 4577625064341504,
"weather_state_name": "Heavy Rain",
"weather_state_abbr": "hr",
"wind_direction_compass": "WSW",
"created": "2020-07-14T19:35:14.577740Z",
"applicable_date": "2020-07-14",
"min_temp": 11.11,
"max_temp": 15.05,
"the_temp": 14.32,
"wind_speed": 6.570953330777592,
"wind_direction": 254.13274105640758,
"air_pressure": 1016.5,
"humidity": 85,
"visibility": 7.654361031575599,
"predictability": 77
},
{
"id": 4896540210495488,
"weather_state_name": "Showers",
"weather_state_abbr": "s",
"wind_direction_compass": "WNW",
"created": "2020-07-14T19:35:17.569534Z",
"applicable_date": "2020-07-15",
"min_temp": 12.31,
"max_temp": 17.03,
"the_temp": 16.509999999999998,
"wind_speed": 7.600821124862802,
"wind_direction": 284.49357944800784,
"air_pressure": 1015.5,
"humidity": 82,
"visibility": 13.558008729022509,
"predictability": 73
},
]
"title": "Texas",
"location_type": "City",
"timezone": "US"
<!-- end snippet -->
From your json string, we can see that there is a list of weather object in object consolidated_weather.
So you need to parse json to WeatherList.
public class WeatherList {
public IEnumerable<Weather> consolidated_weather { get; set; }
}
[HttpPost("/weather")]
public async Task<IEnumerable<Weather>> GetWeatherAsync()
{
string jsonString = "{\"consolidated_weather\":[{\"id\":4577625064341504,\"weather_state_name\":\"Heavy Rain\",\"weather_state_abbr\":\"hr\",\"wind_direction_compass\":\"WSW\",\"created\":\"2020-07-14T19:35:14.577740Z\",\"applicable_date\":\"2020-07-14\",\"min_temp\":11.11,\"max_temp\":15.05,\"the_temp\":14.32,\"wind_speed\":6.570953330777592,\"wind_direction\":254.13274105640758,\"air_pressure\":1016.5,\"humidity\":85,\"visibility\":7.654361031575599,\"predictability\":77},{\"id\":4896540210495488,\"weather_state_name\":\"Showers\",\"weather_state_abbr\":\"s\",\"wind_direction_compass\":\"WNW\",\"created\":\"2020-07-14T19:35:17.569534Z\",\"applicable_date\":\"2020-07-15\",\"min_temp\":12.31,\"max_temp\":17.03,\"the_temp\":16.509999999999998,\"wind_speed\":7.600821124862802,\"wind_direction\":284.49357944800784,\"air_pressure\":1015.5,\"humidity\":82,\"visibility\":13.558008729022509,\"predictability\":73}]}";
IEnumerable<Weather> weather = new List<Weather>();
WeatherList weatherList = new WeatherList();
weatherList = JsonConvert.DeserializeObject<WeatherList>(jsonString);
return weatherList.consolidated_weather;
}
Test Result
UPDATE
public class WeatherList
{
public IEnumerable<Weather> consolidated_weather { get; set; }
public string title { get; set; }
public string location_type { get; set; }
public string timezone { get; set; }
}
There are two missings in your json string:
<!-- language: lang-html -->
{
"consolidated_weather": [
{
"id": 4577625064341504,
...
},
{
"id": 4896540210495488,
...
},
] <!-- missing , -->
"title": "Texas",
"location_type": "City",
"timezone": "US"
<!-- missing } -->
<!-- end snippet -->
Assuming that you cannot change the content of the Json, you can create a ConsolidatedWeather class that contains the List of Weathers
public class ConsolidatedWeather
{
public List<Weather> Consolidated_Weather { get; set; }
}
and in your deserialize part
return JsonConvert.DeserializeObject<ConsolidatedWeather>(jsonString).Consolidated_Weather;
I am currently trying to progress with EF Core with a one-to-many (a user has many items). A tutorial or three later I managed to get things working with two very small and simple tables; however, I got a json exception: A possible object cycle was detected which is not supported which indicated that I had circular references.
Here is my code that gets around the issue using DTO objects, but is there a more cleaner way I can get around this issue as typing the, though it works, felt a bit wrong.
User:
namespace TestWebApplication.Database
{
public class User
{
[Key]
public int UserId { get; set; }
public string UserName { get; set; }
public string Dob { get; set; }
public string Location { get; set; }
public ICollection<Items> Items { get; set; }
}
}
Items:
namespace TestWebApplication.Database
{
public class Items
{
[Key]
public int ItemId { get; set; }
public string Item { get; set; }
public string Category { get; set; }
public string Type { get; set; }
public virtual User User { get; set; }
}
}
DtoItems:
namespace TestWebApplication.Database.DTOs
{
public class DtoItems
{
public string Item { get; set; }
public string Category { get; set; }
public string Type { get; set; }
public DtoUser User { get; set; }
}
}
DtoUser:
namespace TestWebApplication.Database.DTOs
{
public class DtoUser
{
public string UserName { get; set; }
public string Dob { get; set; }
public string Location { get; set; }
}
}
TestController:
[HttpGet]
[Route("getitems")]
public ActionResult<List<Items>> GetItems()
{
List<Items> items = _myContext.Items.Include(i => i.User).ToList();
// DTOs
List<DtoItems> dtoItems = new List<DtoItems>();
foreach (var i in items)
{
var dtoItem = new DtoItems
{
Item = i.Item,
Category = i.Category,
Type = i.Type,
User = new DtoUser
{
UserName = i.User.UserName,
Dob = i.User.Dob,
Location = i.User.Location
}
};
dtoItems.Add(dtoItem);
}
return Ok(dtoItems);
}
The output from endpoint:
[
{
"item": "xxx",
"category": "xxx",
"type": "xxx",
"user": {
"userName": "xxx",
"dob": "xxx",
"location": "xx"
}
},
{
"item": "xxx",
"category": "xxx",
"type": "xxx",
"user": {
"userName": "xxx",
"dob": "xxx",
"location": "xxx"
}
}
]
In my opinion, the use of a DTO is the correct way of dealing with this issue. The fact that your datamodel does not serialize well is trying to hint to you that you should not be serializing your datamodel from the API at all.
I think returning a DTO also solves further issues down the road (What if you want to return all properties of the UserModel except one, maybe it's a sensitive property you don't want to just return from your API, what if your UserModel in the DB gets more navigation properties that you don't want to return?).
There is really only two other ways of handling this.
You can switch to Newtonsoft.Json which has support for handling reference loops and you can configure it one single line
Like so :
services.AddControllers().AddNewtonsoftJson(x => x.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore);
System.Text.Json does not have support for doing this (yet). Follow this Github Issue for more info : https://github.com/dotnet/runtime/issues/30820
You use the JsonIgnore attribute to force non serialization of properties which will work but... It looks weird to have an EntityFramework model have JSON Serialization options on it...
So your best bet, stick with the DTO.
More info :
https://dotnetcoretutorials.com/2020/03/15/fixing-json-self-referencing-loop-exceptions/
https://github.com/dotnet/runtime/issues/30820
https://www.newtonsoft.com/json/help/html/ReferenceLoopHandlingIgnore.htm
I am currently learning how to work with EF Core with a simple one to many setup, where a user can have many items. In terms of retrieving the data from the tables, this is fine with some DTO models; however, when I try and add a user with multiple items via Postman, I noticed that for each item it had duplicated the user that many times (i.e. a user with 3 items will create 3 items and 3 users):
Postman (POST)
{
"username": "xxx",
"dob": "xxx",
"location": "xxx",
"items":[{
"item": "xxx",
"category": "xxx",
"type": "xxx"
},
{
"item": "xxx",
"category": "xxx",
"type": "xxx"
},
{
"item": "xxx",
"category": "xxx",
"type": "xxx"
}]
}
Context:
namespace TestWebApplication.Database
{
public class MyContext : DbContext
{
public DbSet<User> Users { get; set; }
public DbSet<Items> Items { get; set; }
public MyContext(DbContextOptions<MyContext> options)
: base(options)
{
// erm, nothing here
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<User>()
.HasMany(i => i.Items)
.WithOne(u => u.User);
}
public override int SaveChanges()
{
var entities = from e in ChangeTracker.Entries()
where e.State == EntityState.Added
|| e.State == EntityState.Modified
select e.Entity;
foreach (var entity in entities)
{
var validationContext = new ValidationContext(entity);
Validator.ValidateObject(entity, validationContext);
}
return base.SaveChanges();
}
}
}
Controller:
[HttpPost]
[Route("insertuseranditems")]
public ActionResult InsertUserAndItems(User user)
{
if (ModelState.IsValid)
{
using (MyContext myContext = _myContext as MyContext)
{
myContext?.Users?.Add(user);
int changes = myContext.SaveChanges();
if (changes > 0)
{
return Created("User saved", user);
}
}
}
return Accepted();
}
Items:
namespace TestWebApplication.Database
{
public class Items
{
[Key]
public int ItemId { get; set; }
public string Item { get; set; }
public string Category { get; set; }
public string Type { get; set; }
public virtual User User { get; set; }
}
}
Users:
namespace TestWebApplication.Database
{
public class User
{
[Key]
public int UserId { get; set; }
public string UserName { get; set; }
public string Dob { get; set; }
public string Location { get; set; }
public ICollection<Items> Items { get; set; }
}
}
I revisited my code and changed my Items.cs model to:
namespace TestWebApplication.Database
{
public class Items
{
[Key]
public int ItemId { get; set; }
public string Item { get; set; }
public string Category { get; set; }
public string Type { get; set; }
[ForeignKey("User")] // added this
public int? UserId { get; set; } // added this
public virtual User User { get; set; }
}
}
Now, when I send the above json via Postman, I get multiple items with the same user as a Foreign Key.
The site below seemed to help:
The ForeignKey Attribute
There is a note in the developer road map from December of 2013 saying, "Lock/Unlock – We’ve added support for locking and unlocking files into the V2 API."
I've been all through the V2 API (for c#) and cannot find it anywhere. I expected to find something in the BoxFilesManager class or as something you would pass to UpdateInformationAsync within the BoxFileRequest class.
So is there a way to lock/unlock a file?
Great question. In order to see the current lock status of a file do a
GET https://api.box.com/2.0/files/7435988481/?fields=lock
If there is no lock on the file, you'll get something like this back:
{
"type": "file",
"id": "7435988481",
"etag": "0",
"lock": null
}
If you want to lock a file, you need to do a PUT (update) on the /files/ endpoint with a body that tells us what type of lock, and when to release it. Like this:
PUT https://api.box.com/2.0/files/7435988481/?fields=lock
{"lock": {
"expires_at" : "2014-05-29T19:03:04-07:00",
"is_download_prevented": true
}
}
You'll get a response confirming your lock was created:
{
"type": "file",
"id": "7435988481",
"etag": "1",
"lock": {
"type": "lock",
"id": "14516545",
"created_by": {
"type": "user",
"id": "13130406",
"name": "Peter Rexer gmail",
"login": "prexer#gmail.com"
},
"created_at": "2014-05-29T18:03:04-07:00",
"expires_at": "2014-05-29T19:03:04-07:00",
"is_download_prevented": true
}
}
Since there isn't a lock/unlock yet, I created a Lock Manager based on the existing managers:
class BoxCloudLockManager : BoxResourceManager
{
#region Lock/Unlock Classes
[DataContract]
internal class BoxLockRequestInfo
{
[DataMember(Name = "status")]
public string Status { get; set; }
//[DataMember(Name = "expires_at")]
//public string ExpiresAt { get; set; }
[DataMember(Name = "is_download_prevented")]
public bool IsDownloadPrevented { get; set; }
}
[DataContract]
internal class BoxLockRequest
{
[DataMember(Name = "lock")]
public BoxLockRequestInfo Lock { get; set; }
}
#endregion
const string LockFileString = "{0}/?fields=lock";
public BoxCloudLockManager(IBoxConfig config, IBoxService service, IBoxConverter converter, IAuthRepository auth)
: base(config, service, converter, auth)
{
}
public async Task<BoxLockInfo> LockAsync(string documentId,bool isDownloadPrevented = true)
{
var lockRequest = new BoxLockRequest { Lock = new BoxLockRequestInfo { Status = "lock", IsDownloadPrevented = isDownloadPrevented } };
BoxRequest request = new BoxRequest(_config.FilesEndpointUri, string.Format(LockFileString, documentId))
.Method(RequestMethod.Put)
.Payload(_converter.Serialize(lockRequest));
IBoxResponse<BoxLockInfo> response = await ToResponseAsync<BoxLockInfo>(request).ConfigureAwait(false);
return response.ResponseObject;
}
public async Task<BoxLockInfo> UnlockAsync(string documentId)
{
BoxRequest request = new BoxRequest(_config.FilesEndpointUri, string.Format(LockFileString, documentId))
.Method(RequestMethod.Put)
.Payload("{\"lock\":null}");
IBoxResponse<BoxLockInfo> response = await ToResponseAsync<BoxLockInfo>(request).ConfigureAwait(false);
return response.ResponseObject;
}
public async Task<BoxLockInfo> GetLockInfoAsync(string documentId)
{
BoxRequest request = new BoxRequest(_config.FilesEndpointUri, string.Format(LockFileString, documentId))
.Method(RequestMethod.Get);
IBoxResponse<BoxLockInfo> response = await ToResponseAsync<BoxLockInfo>(request).ConfigureAwait(false);
return response.ResponseObject;
}
}
I derived a class from BoxClient, adding a LockManager and instantiate it within the Constructor.
Here is the Lock Info:
[DataContract]
public class BoxLockedBy
{
[DataMember(Name = "type")]
public string Type { get; set; }
[DataMember(Name = "id")]
public string Id { get; set; }
[DataMember(Name = "name")]
public string Name { get; set; }
[DataMember(Name = "login")]
public string Login { get; set; }
}
[DataContract]
public class BoxLockDetails
{
[DataMember(Name = "type")]
public string Type { get; set; }
[DataMember(Name = "id")]
public string Id { get; set; }
[DataMember(Name = "created_by")]
public BoxLockedBy CreatedBy { get; set; }
[DataMember(Name = "created_at")]
public string CreatedAt { get; set; }
[DataMember(Name = "expires_at")]
public string ExpiresAt { get; set; }
[DataMember(Name = "is_download_prevented")]
public bool IsDownloadPrevented { get; set; }
}
[DataContract]
public class BoxLockInfo
{
[DataMember(Name = "type")]
public string Type { get; set; }
[DataMember(Name = "id")]
public string Id { get; set; }
[DataMember(Name = "etag")]
public string Etag { get; set; }
[DataMember(Name = "lock")]
public BoxLockDetails LockDetails { get; set; }
}