How should I store documents for a logging application? - ravendb

I am writing a logging application and I'm using RavenDB as the data store. We have a lot of textual based logging and want to be able to leverage the indexing abilities of RavenDB to search through it.
I started writing my document implementation like this:
public class LogEntry
{
public DateTime Date {get;set;}
public string Message {get;set;}
}
public class Information : LogEntry {}
public class Error : LogEntry {}
I did this so the UI would show different collections for different log levels, however, after thinking about it some more, I would only be using the UI to query the logs.
Since RavenDB stores all of the documents the same behind the scenes, would it be better to just have a single document LogEntry that exposes a severity level property, and then create multiple indexes that group document collections by severity level?
public class LogEntry
{
public DateTime Date {get;set;}
public string Message {get;set;}
public string Severity {get;set;}
}

I would create a single class, but for Severity, how about an enum type? RavenDB will store enums as strings, so it doesn't have any performance impact. But you'll get the benefit of avoiding typos and having Intellisense in your .Net code.
You should just have one index, not multiples. Your map should include the any fields that you want to sort or filter on, so it may be that you map all three fields, Date, Message and Severity.
You might also want to consider marking the Message field as "analyzed" in your index. This will allow full-text searches of your log messages.
public enum Severity
{
Debug,
Info,
Warning,
Error
}
public class LogEntry
{
public DateTime Date { get; set; }
public string Message { get; set; }
public Severity Severity { get; set; }
}
public class LogEntriesIndex : AbstractIndexCreationTask<LogEntry>
{
public LogEntriesIndex()
{
Map = entries => from entry in entries
select new
{
entry.Date,
entry.Message,
entry.Severity
}
Index(x => x.Message, FieldIndexing.Analyzed);
}
}

Related

Does including Collections in Entities violate what an entity is supposed to be?

I am building a Web API using Dapper for .NET Core and trying to adhere to Clean Architecture principles. The API is consumed by an external Angular front-end.
I have repositories that use Dapper to retrieve data from the database, and this data then passes through a service to be mapped into a DTO for display to the user.
It is my understanding that an entity should be an exact representation of the database object, with no extra properties, and that I should use DTOs if I require some additional properties to show the user (or if I wish to obscure certain properties from the user too).
Suppose I have a DTO:
public class StudentDTO
{
public Guid Id { get; set; }
public string Name { get; set; }
public List<Assignment> Assignments { get; set;}
}
and its corresponding Entity:
public class Student
{
public Guid Id { get; set; }
public string Name { get; set; }
}
With this model, should I want to get a student with all of their assignments, I'd need to have two repository calls, and do something like this in the service:
public StudentDTO GetById(Guid id)
{
var student = this.studentRepository.GetById(id);
var assignments = this.assignmentRepository.GetByStudentId(id);
return SomeMapperClass.Map(student, assignments);
}
But this seems inefficient and unnecessary. My question is, should I not just retrieve the Assignments when I get the student entity in the repository, using a JOIN? Or would this violate what an entity is supposed to be?
I apologise, I do realise this is a rather simple question, but I'd really like to know which method is the best approach, or if they both have their use cases
I think it would be more efficient, since map uses reflections, that is slower tens times
public StudentDTO GetById(Guid id)
{
var student = this.studentRepository.GetById(id);
student.Assignments = this.assignmentRepository.GetByStudentId(id);
return student;
}
but the common way is
return _context.Students.Include(i=>i.Assignments).FirstOrDefault(i=> i.Id==id);
This is why the generic repository is a bad idea in the most casses, since it is hard to guess what set of data you will need.

Entity Framework Core: using navigation properties without foreign key

I have following object model:
public class SharingRelation:BaseEntity
{
public Guid? Code { get; set; }
public string Value { get; set; }
}
public class SecondLevelShareEntity : BaseEntity
{
public string Name { get; set; }
public Guid? SharingCode { get; set; }
public List<SharingRelation> SharingRelations { get; set; }
}
In my database (it may be poor db design but I need to answer this question for research), SharingRelation is some sort of dependent entity of SecondLevelShareEntity on Code == SharingCode values. I can have two entities of type SecondLevelShareEntity with same SharingCode value. So, for each of them I need to get all related SharingRelation objects depending on Code and SharingCode values. I can do it using SQL and join on this columns. But how can I do it using EF Core and navigation properties (I want to get all dependent entities using Include() for example)? When I configure my entities like this
public class SharingRelationEntityTypeConfiguration : BaseEntityTypeConfiguration<SharingRelation>
{
public override void Configure(EntityTypeBuilder<SharingRelation> builder)
{
base.Configure(builder);
builder.HasOne<SecondLevelShareEntity>().WithMany(x => x.SharingRelations).HasForeignKey(x => x.Code)
.HasPrincipalKey(x => x.SharingCode);
}
}
EF Core creates foreign key and marks it unique. I am obviously getting an error that that is impossible to have several SecondLevelShareEntity with the same SharingCode
System.InvalidOperationException : The instance of entity type 'SecondLevelShareEntity' cannot be tracked because another instance with the key value '{SharingCode: 8a4da9b3-4b8e-4c91-b0e3-e9135adb9c66}' is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached.
How can I avoid creation of foreign key, but keep using navigation properties (as far, as I see normal queries with navigations generate simple JOIN statements)
UPDATED I can provide real data in database. SecondLevelShareEntity table looks like this:
_id Name SharingCode
----------------------------------------------------------------------
1 "firstSecondLevelEnt" "efcb1c96-0ef1-4bb3-a952-4a6511ab448b"
2 "secondSecondLevelEnt" "efcb1c96-0ef1-4bb3-a952-4a6511ab448b"
And SharingRelation table looks like this:
_id Value Code
----------------------------------------------------------------------
1 "firstSharingRelation" "efcb1c96-0ef1-4bb3-a952-4a6511ab448b"
2 "secondSharingRelation" "efcb1c96-0ef1-4bb3-a952-4a6511ab448b"

RavenDB SaveChanges() not saving properties on derived class ([DataMember] used in other class)

I've recently upgraded to build 2230, and things are working just fine. However, I just updated the RavenDB .NET client assemblies and now I'm having this issue.
This code has been in place for a year or so. This is how I'm saving:
public void Save(EntityBase objectToSave)
{
using (IDocumentSession session = GetOptimisticSession())
{
session.Store(objectToSave, objectToSave.Etag);
session.SaveChanges();
}
}
And this is the object I'm saving.
public class InstallationEnvironment : EntityBase
{
public string Name { get; set; }
public int LogicalOrder { get; set; }
}
Now the base class:
public class EntityBase : NotifyPropertyChangedBase
{
public string Id { get; set; } // Required field for all objects with RavenDB.
}
The problem is that the base class property (Id) is getting persisted in RavenDB, but the derived properties (Name, LogicalOrder) are not.
Why would only the base class properties be saved in RavenDB?
Got it. Through trial and error, I noticed that one derived property was being saved (on a different class than the one shown in my question), and that property was decorated with the [DataMember] attribute. I just recently added it because I'm creating a WCF service for my app, and I started by using that attribute on one property for testing.
As Ayende states here, you have to use [DataMember] on all properties, or on none of them. If [DataMember] exists on a property, all others will be ignored.
Note: This was a problem for me even though [DataMember] was specified on a property in a different class. It seems like if I use [DataMember] anywhere, I have to use it for everything.

RavenDB static index on document with dynamic field

I am trying to create a static index for the following sample class:
public class Board {
...other assorted fields
List<dynamic> Messages {get; set;}
internal Board() {Messages = new List<dynamic>();}
}
The index is to filter boards which have messages which are a older than a certain date. The aim is to perform an "update" operation on messages which are due today, update their content, and persist them back. The index is needed to avoid traversing all the messages for a board for all clients as that may be computationally expensive. Messages is a list of message types which inherit from a base class which contains a property ExpiryDate.
Trying to create an index like follows results in an "An expression tree may not contain a
dynamic operation" error. I know that the dynamic type does not play well with Linq queries hence the need to use LuceneQueries instead of Query() in RavenDB. Is there any way to make this index work with dynamic properties? Thanks!
public class ScanBoardMessagesIndex : AbstractIndexCreationTask<Board>
{
public ScanBoardMessagesIndex () {
Map = boards => from board in boards
where board.Messages.Any(msg => ((MessageItem) msg).ExpiryDate <= DateTime.UtcNow.Date)
select board;
}
}
EDIT:
I ran into a raven serialization issue because the metadata clr-type of existing Board documents was set to a class namespace which was not valid anymore. I am doing a migration project so I went ahead and first issued a patch to change the metadata clr-type of the existing documents before migrating them to the new data structure which uses a base/abstract class for list of Messages instead of type dynamic.
A Map/Reduce index seems more appropriate for the given requirements. Effectively, you want to be able to query boards by the oldest expiry date of messages in the board. This is an aggregating operation, exactly what Map/Reduce was designed to solve. Also, using a base class for messages will allow you to define the index without resorting to the lower level IndexDefinition:
public class Message
{
public DateTime ExpiryDate { get; set; }
}
public class Board
{
public string Id { get; set; }
public List<Message> Messages { get; set; }
}
public class OldestExpiryDateMessageInBoard : AbstractIndexCreationTask<Board, OldestExpiryDateMessageInBoard.Result>
{
class Result
{
public string BoardId { get; set; }
public DateTime OldestExpiryDate { get; set; }
}
public OldestExpiryDateMessageInBoard()
{
this.Map = boards => from board in boards
from message in board.Messages
select new
{
BoardId = board.Id,
OldestExpiryDate = message.ExpiryDate
};
this.Reduce = results => from result in results
group result by result.BoardId into g
select new
{
BoardId = g.Key,
OldestExpiryDate = g.Min(x => x.OldestExpiryDate)
};
}
}
You can then query this index with Lucene syntax.

NHibernate add unmapped column in interceptor

I'm trying to save a mapped entity using NHibernate but my insert to the database fails because the underlying table has a column that does not allow nulls and IS NOT mapped in my domain object. The reason it isn't mapped is because the column in question supports a legacy application and has no relevance to my application - so I'd like to not pollute my entity with the legacy property.
I know I could use a private field inside my class - but this still feels nasty to me. I've read that I can use an NHibernate interceptor and override the OnSave() method to add in the new column right before my entity is saved. This is proving difficult since I can't work out how to add an instance of Nhibernate.type.IType to the types parameter of my interceptor's OnSave.
My Entity roughly looks like this:
public class Client
{
public virtual int Id { get; set; }
public virtual int ParentId { get; set; }
public virtual string Name { get; set; }
public virtual string Phone { get; set; }
public virtual string Email { get; set; }
public virtual string Url { get; set; }
}
And my interceptor
public class ClientInterceptor : EmptyInterceptor
{
public override bool OnSave(object entity, object id, object[] state, string[] propertyNames, NHibernate.Type.IType[] types)
{
if (entity is Client)
{
/*
manually add the COM_HOLD column to the Client entity
*/
List<string> pn_list = propertyNames.ToList();
pn_list.Add("COM_HOLD");
propertyNames = pn_list.ToArray();
List<Object> _state = state.ToList();
_state.Add(false);
state = _state.ToArray();
//somehow add an IType to types param ??
}
return base.OnSave(entity, id, state, propertyNames, types);
}
}
Does anyone have any ideas on how to do this properly?
I can't say for sure since I've never actually done this (like Stefan, I also prefer to just add a private property), but can you just add a NHibernate.Type.BooleanType to the types array?
List<IType> typeList = types.ToList();
typeList.Add(new BooleanType());
types = typesList.ToArray();
EDIT
Yes, it looks like you are right; the types have an internal constructor. I did some digging and found TypeFactory:
Applications should use static
methods and constants on
NHibernate.NHibernateUtil if the
default IType is good enough. For example, the TypeFactory should only
be used when the String needs to have a length of 300 instead of 255. At this point
NHibernate.String does not get you thecorrect IType. Instead use TypeFactory.GetString(300) and keep a
local variable that holds a reference to the IType.
So it looks like what you want is NHibernateUtil:
Provides access to the full range of
NHibernate built-in types. IType
instances may be used to bind values
to query parameters. Also a factory
for new Blobs and Clobs.
typeList.Add(NHibernateUtil.Boolean);
Personally I wouldn't do it so complicated. I would add the private property and assign it a default value - finished. You could also consider a default value in the database, then you don't need to do anything else.
private virtual bool COM_HOLD
{
get { return false; }
set { /* make NH happy */ }
}
Before writing a interceptor for that I would consider to write a database trigger. Because with the Interceptor you are "polluting" your data access layer. It could make it unstable and you could have strange problems.