Using Split in RavenDB index - ravendb

I'd like to use a couple of regexes in an index. This is a simplified example:
Map =
books =>
books.Select(x => new {Sentences = Regex.Split(x.Description, "<br>")})
.Select(x => new {Results = x.Sentences.Where(y => Regex.IsMatch(y, "foo"))})
.Where(x => x.Results.Any())
.Select(x => new {});
However, the following exception is thrown:
An exception of type 'Microsoft.CSharp.RuntimeBinder.RuntimeBinderException' occurred in System.Core.dll but was not handled in user code
Additional information: 'System.Array' does not contain a definition for 'Where'
I've tried .AsEnumerable() but to no avail. Can this be done?

To fix this, you need to call .AsEnumerable() on the thing you're .Select'ing off.
So instead of Regex.Split(...).Select(...), use Regex.Split(...).AsEnumerable().Select(...)
RavenDB relies on an internal class that knows how to map common extension methods.
Sometimes, however, it may not be able to dynamically resolve the extension methods. To help dynamic resolution, call the .AsEnumerable(), then the extension methods will be found.
Your code should look something like this:
Map =
books =>
books.Select(x => new {Sentences = Regex.Split(x.Description, "<br>")})
.AsEnumerable()
.Select(x => new {Results = x.Sentences.Where(y => Regex.IsMatch(y, "foo"))})
.Where(x => x.Results.Any())
.Select(x => new {});

Related

EFCore throws NullReferenceException on complex splitQuery, but only sometimes

I have an EFCore query that, for the same parameter values and same database state, sometimes throws a NullReferenceException. Any ideas how I can identify where the problem lies?
The query looks like this (yes it's a somewhat complicated db structure):
var lines = _context.Calcs.Where(x => x.PlanId == planId && x.IsDeleted == false)
.Include(x => x.Taggings)
.ThenInclude(x => x.CalcTag)
.Include(x => x.Destination)
.ThenInclude(x => x.FormatCode)
.Include(x => x.Destination)
.ThenInclude(x => x.Dimensions.Where(vtd => vtd.PlanId == planId))
.Include(x => x.CalcDimensions)
.ThenInclude(x => x.Dimension)
.Include(x => x.CLVT)
.ThenInclude(x => x.ValueType)
.Include(x => x.CLVT)
.ThenInclude(x => x.CLVTJoinFields)
.ThenInclude(x => x.Dimension)
.Include(x => x.CLVT)
.ThenInclude(x => x.CLVTTagFields)
.Include(x => x.CLVT)
.ThenInclude(clvt => clvt.PreCalc)
.Include(x => x.CLVTPreCalcs)
.ThenInclude(x => x.CLVTJoinFields)
.Include(x => x.CLVTPreCalcs)
.ThenInclude(x => x.CLVTTagFields)
.AsSplitQuery()
.ToList();
For exactly the same planId and database state, sometimes when this executes it results in a NullReferenceException and sometimes it doesn't. It looks like the call stack is always the same when the exception is thrown, which is helpful.
Exception is:
System.NullReferenceException
HResult=0x80004003
Message=Object reference not set to an instance of an object.
Source=Microsoft.EntityFrameworkCore.Relational
StackTrace:
at Microsoft.EntityFrameworkCore.Query.Internal.SplitQueryingEnumerable`1.Enumerator.Dispose()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at MyClass.MyMethod(Int32 planId) in C:\code\server\MyProduct\MyClass.cs:line 94
Searching for other NullReferenceException cases from EFCore it seems like it can happen when there's a null value in the database when the EF model would expect it to be non-nullable. But in this case the same data is being pulled back and often there's no exception, so I'm guessing that's not the case here. What else could cause this?
My next guess is that in the code path that causes the problem something else has happened with the DbContext, e.g. it's cached some objects, causing a different behavior for this same query. But my _context.ChangeTracker.DebugView.ShortView is empty when this exception is thrown, suggesting it's not that. My context usually has QueryTrackingBehavior==NoTracking and AutoDetectChangesEnabled==false unless it's in the middle of an update, but I tested without that (i.e. default tracking behavior) and got the same results.
I've used SQL Server extended events to trace the SQL that's executed. I see about 5 queries being started and I think the exception happens before rpc_completed fires for them all. I haven't yet analysed this thoroughly enough to know if it always throws the exception at the same point with receiving the SQL server results. But given that the results of the queries are the same each time I'm not sure that analysing this further is going to help much.
Any suggestions what could be causing the exception?
I'm using EFCore 5.0.12 (I was on 5.0.5 but upgraded and same problem happens)
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer"
Version="5.0.12" />
UPDATE:
Now I think that there's multiple threads using the same DbContext at once. They're all from the same request but there's some code a bit like this:
foreach (var x in y)
{
tasks.Add(MyAsyncMethodThatUsesDbContext(planId, otherParams));
}
var z = await Task.WhenAll(tasks);
It's a fair bit more convoluted so not as obvious, but I'll see if it's that.
The problem was caused by multiple threads using the same DbContext, which is a naughty thing to do. My DbContext wasn't shared by multiple requests but within a single background Hangfire job processing there was code that created a series of Tasks and then called WaitAll on them, something like this:
foreach (var x in y)
{
tasks.Add(MyAsyncMethodThatUsesDbContext(planId, otherParams));
}
var z = await Task.WhenAll(tasks);
I thought there was something in EFCore that warned when you use the same DbContext from multiple threads, but perhaps it doesn't work either when running from a Hangfire job (i.e. no active asp request) or when the tasks are created all from the same thread?
One solution is to not parallelize the database access. Or alternatively ensure a separate DbContext is used for each task.

EF Core AutoMapper: Filter nested collection in include not applied

I'm using EF Core 5. I want to make a query and use ProjectTo for mapping my result. But I need to make a filter in nested collection like this:
var questions = await _context
.MedicalDatas
.Include(p => p.MedicalDataItems)
.Include(p => p.MedicalAnswers.Where(x => x.AppUserId == userId))
.ProjectTo<MedicalDataDto>(_cp)
.ToListAsync();
The 2 navigation properties are included in destination type but when I check the SQL generated by EF Core, there is no where clause and it selects all medical answers from the database.
BUT without projecting, it works as expected.
What is wrong here?
you don't need include with projectto
var questions = await _context
.MedicalDatas
.Where(p => p.MedicalAnswers.Any(x => x.AppUserId == userId))
.ProjectTo<MedicalDataDto>(_cp)
.ToListAsync();

Iterating over controllers in solution

I am trying to get a list of all public methods that return an ActionResult from any controller in my solution using reflection but I am experiencing some strange behavior.
Assembly asm = Assembly.GetAssembly(typeof(MyDLL.MvcApplication));
var controllerActionList = asm.GetTypes().ToList();
If I run the above code I get a list of all my types including all my models and controllers etc. just like I would expect. However, if I modify it and run the below code my results list comes back empty. Any idea what's going on here? I would think this should filter the types down so I get a list of all the controllers right?
Assembly asm = Assembly.GetAssembly(typeof(MyDLL.MvcApplication));
var controllerActionList = asm.GetTypes()
.Where(type => typeof(Controller).IsAssignableFrom(type)).ToList();
I got it working by using the code below. I think the direct type comparison is failing for me because I believe I have two different .net versions between these 2 projects.
Assembly asm = Assembly.GetAssembly(typeof(SCCView.MvcApplication));
var controllerActionList = asm.GetTypes()
.Where(type => type.BaseType.Name == "Controller")
.SelectMany(type => type.GetMethods())
.Where(
m => m.IsPublic && m.ReturnType.Name == "ActionResult")
.Select(x => new {Controller = x.DeclaringType.Name, Action = x.Name})
.OrderBy(x => x.Controller).ThenBy(x => x.Action).ToList();
The above will give you a paired list of every public method in a controller that returns an ActionResult

Zend Authentication 'Zend\Authentication\Adapter\DbTable' gives error while authenticating

I used a simple instruction from
http://framework.zend.com/manual/2.0/en/modules/zend.authentication.adapter.dbtable.html#advanced-usage-by-example for Zend Authentication.
Here is my code:
$adapter = $sm->get('adapter');
$authAdapter = new DbTable($adapter);
$authAdapter -> setTableName('users')->setIdentityColumn('username')->setCredentialColumn('password');
$authAdapter -> setIdentity('admin')-> setCredential('password');
$authAdapter -> authenticate();
The above code generates error as follows:
The supplied parameters to DbTable failed to produce a valid sql statement, please check table and column names for validity.
I know if makes sense to use ZF-Commons and ZF-Users module and not reinvent the wheel... but being relatively new to ZF2 I want to try it myself.
The answer's right there in the error message, Zend\Authentication\Adapter\DbTable expects more than just an adapter as a parameter, it also needs a table name, and the name of the identifier and credential columns...
$authAdapter = new DbTable($dbAdapter,
'tableName',
'identifierColumnName',
'credentialColumnName'
);
This info is covered in the docs, which is always a good starting point -> http://zf2.readthedocs.org/en/release-2.1.4/modules/zend.authentication.adapter.dbtable.html
I know this is an old one and probably you have the answer now. Even though, I will put my answer here for further users. I myself faced lots of similar issues where the tutorials expect us to have some preconditions in place.
This error occur because you are not completely right in the statement to get the DB adapter. You need to call a service management instance with an string defined in your locals or globals configurations. For example:
I have this factory defined in my global.php file:
return array(
'service_manager' => array(
'factories' => array(
'Zend\Db\Adapter\Adapter'
=> 'Zend\Db\Adapter\AdapterServiceFactory',
),
),
'db' => array(
'driver' => 'Pdo',
'driver_options' => array(
PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
),
),
);
In my local.php I have the credentials and information to reach my DB server as follow:
return array(
'db' => array(
'username' => 'valid_dbusername',
'password' => 'valid_dbpass',
'dsn' => 'mysql:dbname=valid_dbname;host=valid_dbserver',
),
);
This is enough to define my service calle 'Zend\Db\Adapter\Adapter'. Then, to instantiate my Db adapter I have the following lines inside any function in my controller:
if (!$this->adapter) {
$sm = $this->getServiceLocator();
$this->adapter = $sm->get('Zend\Db\Adapter\Adapter');
}
It is important to note adapter here is a class variable. So, it must be defined inside my controller class like:
public $adapter;
This steps are enough to you get an DB adapter. I am assuming you dont have a factory called 'adapter'. This should make your example work.

Working around Duplicate association path bug in Nhibernate with Query Over

I've got a bit of code that tries to access same association path twice and they're really same aliases but because I'm using query objects, I have them in two different places and I am not really sure how to get the alias.
May be some code can clear the confusion:
var privateBlogQuery = new BlogQuery()
.ByUser(1)
.RemoveHidden()
.FetchUserDetails();
//<-------- In Blog Query object class: ------>
/// Gets all the private blogs the user has permissions to view
public BlogQuery ByUser(int userId)
{
var authorisedUsers = null;
this.Left.JoinQueryOver(r => r.AuthorisedUsers, () => authorisedUsers)
.Where(r => r.User.Id == userId);
return this;
}
/// Removes all private blogs user has marked to be hidden
public BlogQuery RemoveHidden()
{
var authorisedUsers = null;
this.Left.JoinQueryOver(r => r.AuthorisedUsers, () => authorisedUsers)
.Where(r => !r.IsHidden);
return this;
}
/// Loads up details of all users who have permission
/// to view the private blog
public BlogQuery FetchUserDetails()
{
var users = null;
var authorisedUsers = null;
this.Left.JoinQueryOver(r => r.AuthorisedUsers, () => authorisedUsers)
.Left.JoinQueryOver(r => r.User, () => users);
return this;
}
There are times when I'm using all 3 criteria individually and the sql generated is precisely what I need and everything is nice and dandy as long as they are used separately.
Now I need to use them all together, at the same time and nhibernate throws an exception duplicate alias and I changed up the alias on these three functions but then I am greeted with the duplicate association path exeception.
A bit of googling and I learnt that it is a bug in hibernate and I also found a few workarounds on this bug
Trouble is I am using Query objects and hence Query over and I am not really sure how to get the association path / alias here.
So how do I go about this please?
make authorisedUsers a membervariable of BlogQuery and use a marker/flag to know if ByUser and RemoveHidden should do the Join
use JoinAlias
example
AuthorisedUser authorisedUser;
bool authorisedUsersJoined;
public BlogQuery RemoveHidden()
{
if (!authorisedUsersJoined)
this.Left.JoinAlias(r => r.AuthorisedUsers, () => authorisedUser);
this.Where(() => !authorisedUser.IsHidden);
return this;
}
FetchUserDetails is mutual exclusive with the other two because filtering on an association prevents NH from initializing the association. You'll need to subquery with the filter and query the resulting Ids and initialize then.
/// Loads up details of all users who have permission
/// to view the private blog
public BlogQuery FetchUserDetails()
{
this.Query = QueryOver.Of<Blog>()
.WhereRestrictionOn(b => b.Id).IsIn(this.Query.Select(b => b.Id))
.Fetch(r => r.AuthorisedUsers).Eager
.ThenFetch(au => au.User).Eager;
return this;
}