RavenDB Saving to disk query - ravendb

Before I start let me say I am new to RavenDB.
I am just evaluating it at the moment, and am using the RavenDB-Build-616 build. I have the server running, that is I have manually started "Raven.Server.exe", and have the following test code
public class RavenFullTextSearchDemo
{
private DocumentStore documentStore;
private List<string> firstNames = new List<string>() { "John", "Peter", "Paul", "Sam", "Brendon" };
private List<string> lastNames = new List<string>() { "Simons", "Black", "Benson", "Jones", "Breckwell" };
private Random rand = new Random();
public RavenFullTextSearchDemo(DocumentStore documentStore)
{
this.documentStore = documentStore;
}
public void Run()
{
IndexCreation.CreateIndexes(typeof(RavenFullTextSearchDemo).Assembly, this.documentStore);
using (IDocumentSession session = documentStore.OpenSession())
{
//add some random Users
for (int i = 0; i < 5; i++)
{
string name = string.Format("{0} {1},",
firstNames[rand.Next(firstNames.Count)], lastNames[rand.Next(lastNames.Count)]);
User newUser = new User { Name = name };
session.Store(newUser);
}
session.SaveChanges();
//Seem to have to give it some time to persist??? Seems very odd
//If I take the following line out, I dont get any users back at all
Thread.Sleep(3000);
PrintCurrentUsers(session);
var searchName = firstNames[rand.Next(firstNames.Count)];
Console.WriteLine(string.Format("Looking for users with Name starting with '{0}'\r\n", searchName));
Console.WriteLine("Simple starts with");
foreach (var person in Queryable.Where(session.Query<User, User_ByName_FullTextSearch>(), x => x.Name.StartsWith(searchName)))
{
Console.WriteLine(person.Name);
}
Console.WriteLine("\r\n");
Console.WriteLine("Complex starts with");
IQueryable<User> query = session.Query<User, User_ByName_FullTextSearch>();
query = searchName.Split().Aggregate(query, (current, part) => current.Where(x => x.Name.StartsWith(part)));
foreach (var person in query)
{
Console.WriteLine(person.Name);
}
}
}
private void PrintCurrentUsers(IDocumentSession session)
{
IList<User> users = new List<User>();
Console.WriteLine("The current list of users is :\r\n");
foreach (User user in session.Query<User>().ToList())
{
Console.WriteLine(string.Format("UserName : {0}", user.Name));
}
Console.WriteLine("\r\n\r\n");
}
}
public class User_ByName_FullTextSearch : AbstractIndexCreationTask<User>
{
public User_ByName_FullTextSearch()
{
Map = users => from user in users
select new { user.Name };
Index(x => x.Name, FieldIndexing.Analyzed);
}
}
Where this gets called like this
using (var documentStore = new DocumentStore { Url = documentStoreLocation, DefaultDatabase = "ravenTest-" + DateTime.Now.Ticks })
{
documentStore.Initialize();
RavenFullTextSearchDemo ravenFullTextSearchMessAround = new RavenFullTextSearchDemo(documentStore);
ravenFullTextSearchMessAround.Run();
}
There is something weird going on, as I seem to need the Thread.Sleep in order for the new User objects to be persisted to disk. If I do not have that Thread.Sleep in there, I never see anything printed out in the "PrintCurrentUsers" method call.
This is the output I get with no Thread.Sleep
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
The current list of users is :
Looking for users with Name starting with 'Paul'
Simple starts with
Paul Jones,
Complex starts with
Paul Jones,
Whilst this is the output I get with the Thread.Sleep
The current list of users is :
UserName : Paul Black,
UserName : Paul Benson,
UserName : Paul Jones,
UserName : Peter Black,
UserName : Paul Simons,
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Looking for users with Name starting with 'Paul'
Simple starts with
Paul Black,
Paul Benson,
Paul Jones,
Paul Simons,
Complex starts with
Paul Black,
Paul Benson,
Paul Jones,
Paul Simons,
What am I doing wrong. I have some other code elsewhere that inserts a bunch of Users and gets them straight away, and that seems to work ok.
Any clues anyone?

The Users are being persisted, but the index is updated in the background, and you are querying against the index (this is by-design). See: http://ravendb.net/docs/client-api/querying
You can tell Raven to wait until all stale data is in the query.
See this page: http://ravendb.net/docs/client-api/querying/stale-indexes for an example of how to wait until the index is up to date. (Specifially, you need to change your method to:
private void PrintCurrentUsers(IDocumentSession session)
{
IList<User> users = new List<User>();
Console.WriteLine("The current list of users is :\r\n");
foreach (User user in session.Query<User>()
.Customize(x => x.WaitForNonStaleResults(TimeSpan.FromSeconds(5)))
.ToList())
{
Console.WriteLine(string.Format("UserName : {0}", user.Name));
}
Console.WriteLine("\r\n\r\n");
}

Related

RavenDB: How to properly query/filter a nested value from a MultiMapIndex?

My application has a requirement that is should be able to filter/search for Pairs by the Number of the related Contact.
A Pair always has a reference to a Contact stored, but the number of the contact is not, and will not, be stored in the reference. So I tried to create a custom index for this, because the Pair and Contact are stored in different collections.
A simplified example of the index looks like this.
public class Pairs_Search : AbstractMultiMapIndexCreationTask<Pairs_Search.Result>
{
public class Result
{
public string Id { get; set; }
public string Workspace { get; set; }
public ContactResult Contact { get; set; }
public bool HasContactDetails { get; set; }
}
public class ContactResult
{
public string Id { get; set; }
public string Name { get; set; }
public int Number { get; set; }
}
public Pairs_Search()
{
AddMap<Pair>(pairs => pairs
.Select(p => new
{
p.Id,
p.Workspace,
Contact = new
{
p.Contact.Id,
p.Contact.Name,
Number = 0
},
// Mark this items as WITHOUT contact details.
HasContactDetails = false,
}
)
);
AddMap<Contact>(contacts => contacts
.Select(c => new
{
Id = (string) null,
Workspace = (string) null,
Contact = new
{
c.Id,
Name = c.DisplayName,
c.Number
},
// Mark this items as WITH contact details.
HasContactDetails = true,
}
)
);
Reduce = results => results
// First group by the contact ID. This will
// create a group with 2 or more items. One with the contact
// details, and one or more with pair details.
// They are all marked by a boolean flag 'HasContactDetails'.
.GroupBy(x => x.Contact.Id)
// We are going to enrich each item in the current group, that is
// marked as 'HasContactDetails = false', with the contact number.
// We need that so that we can filter on it later.
.Select(group =>
group
.Select(i => new
{
i.Id,
i.Workspace,
Contact = new
{
i.Contact.Id,
i.Contact.Name,
// Does the current item have the contact details?
Number = i.HasContactDetails
// Yes, in this case we use the previously set contact number.
? i.Contact.Number
// No, find the item with the contact details and grab the number.
: group.Single(x => x.HasContactDetails).Contact.Number
},
// Pass on the flag that indicates wheter or not
// this item has the contact details. We are going
// to need it later.
i.HasContactDetails
}
)
// We don't need the items with the contact details
// anymore, so filter them out.
.Where(x => !x.HasContactDetails)
)
// Flatten all the small lists to one big list.
.SelectMany(x => x);
// Mark the following fields of the result as searchable.
Index(x => x.Contact.Number, FieldIndexing.Search);
}
}
I've setup a full example that reproduces the issues I am having. You can find the example here.
Creating the index works fine. Querying the index works fine also as it properly matched the pair and contact and enriched the index result with the number of the contact. But when I try to use a .Where() or .Search() on the nested Number property it fails to properly filter the result dataset from the index.
The index without any filtering works as you can see in below code example (also available in the full example).
private static async Task ThisOneWorks()
{
using (var session = Store.OpenAsyncSession())
{
var results = await session
.Query<Pairs_Search.Result, Pairs_Search>()
.ToListAsync();
LogResults("ThisOneWorks()", results);
}
// Output:
// ThisOneWorks(): Pair 'Harry Potter' with number '70'
// ThisOneWorks(): Pair 'Harry Potter' with number '70'
// ThisOneWorks(): Pair 'Hermione Granger' with number '71'
// ThisOneWorks(): Pair 'Albus Dumbledore' with number '72'
}
Filtering on a non-nested value also works (also available in the full example). As you can see it filters out the one with a different workspace.
private static async Task ThisOneWithWorkspaceFilterWorks()
{
using (var session = Store.OpenAsyncSession())
{
var results = await session
.Query<Pairs_Search.Result, Pairs_Search>()
.Where(x => x.Workspace == "hogwarts")
.ToListAsync();
LogResults("ThisOneWithWorkspaceFilterWorks()", results);
}
// Output:
// ThisOneWithWorkspaceFilterWorks(): Pair 'Harry Potter' with number '70'
// ThisOneWithWorkspaceFilterWorks(): Pair 'Harry Potter' with number '70'
// ThisOneWithWorkspaceFilterWorks(): Pair 'Hermione Granger' with number '71'
}
When I try to filter/search on the Workspace and Number properties I would expect two results that are related to the contact Harry Potter. But instead I just get an empty dataset back.
private static async Task ThisOneWithWorkspaceAndNumberFilterDoesntWork()
{
using (var session = Store.OpenAsyncSession())
{
var results = await session
.Query<Pairs_Search.Result, Pairs_Search>()
.Where(x => x.Workspace == "hogwarts")
.Where(x => x.Contact.Number == 70)
.ToListAsync();
LogResults("ThisOneWithWorkspaceAndNumberFilterDoesntWork()", results);
}
// Output:
// ThisOneWithWorkspaceAndNumberFilterDoesntWork(): EMPTY RESULTS!
}
Can anyone tell me what I am doing wrong here? Any help would be greatly appreciated!
The way to go about it is to store ContactResult in a different collection,
which is what is called a related document in this case,
and when you create the index then you 'Index the Related Document'
Learn from the demo example in:
https://demo.ravendb.net/demos/csharp/related-documents/index-related-documents
The example is for a basic map index but the principle is the same for Multi-Map.
Remove the public class ContactResult from the index class
and define the index with something like:
select new Result
{
....
Number = LoadDocument<Contact>(Pair.Contact).Number
....
}

looking for a good example to create a db record with dapper/sproc

Can you point me to a good example for creating a db record with Dapper via sproc? Ideally I would take an OO approach similar to EF like this:
var user = new User{ FirstName="John", LastName="Smith"}
var userId = dapper.Create(User);
return userId;
Of course, I'm open to the "right" way of doing this in dapper if dapper recommends a different approach
Assuming you have a stored procedure to add a record to the User table like
CREATE PROCEDURE SaveUser(#firstName VARCHAR(10),#lastName VARCHAR(10))
AS
BEGIN
INSERT INTO [User] (FirstName,LastName) VALUES (#firstName ,#lastName);
SELECT SCOPE_IDENTITY()
END
Assuming your User table has Id column (PK) set to Identity.
Now, you can execute your stored procedure like this
var conStr="Replace your connection string here";
using (var con = new SqlConnection(conStr))
{
var userId = con.Query<int>("SaveUser",
new {#firstName = "John", #lastName = "Smith"},
commandType: CommandType.StoredProcedure
).First();
}
If you have a class to represent the User table record, you can pass that class object when you call the stored proc.
public class UserDto
{
public string FirstName { set;get;}
public string LastName { set;get;}
}
and simply pass the object of this!
var conStr="Replace your connection string here";
var u = new UserDto { FirstName = "Johny", LastName= "Depp" };
using (var con = new SqlConnection(conStr))
{
var userId = con.Query<int>("SaveUser", u, commandType: CommandType.StoredProcedure)
.First();
}
You might also want to have a look at Drapper (built on top of Dapper) which would allow you to do something like this.
// instance of user created somewhere & procedure called
// on repository method.
var user = new User { FirstName = "John", LastName = "Smith" };
_commander.Execute(user) ? user : null;
Assuming you're using a repository class of some sort. I've taken the liberty of fleshing out a bit more code, adding some constructor injection love.
public class UserRepository : IUserRepository
{
// the IDbCommander is a Drapper construct
private readonly IDbCommander _commander;
public UserRepository(IDbCommander commander)
{
_commander = commander;
}
public User Create(User user)
{
// execute your procedure here (looked up in config)
return _commander.Execute(user) ? user : null;
}
public IEnumerable<User> RetrieveAll()
{
return _commander.Query<User>();
}
}
Your SQL code would then be saved in a separate file (json or xml config).
This keeps your C# separate from your SQL making life easier in the long run - changes to your SQL don't involve recompiling your code, your repository class becomes more testable, etc. etc. etc.
{
"Namespaces": [
{
"Namespace": "Company.Product.Namespace",
"ConnectionString": {
"ProviderName": "System.Data.SqlClient",
"ConnectionString": "Data Source=(LocalDb)\\mssqllocaldb;Initial Catalog=DrapperTests;Integrated Security=true"
},
"Types": [
{
"Name": "Company.Product.Namespace.UserRepository",
"Commands": {
"Create": {
"CommandText": "[dbo].[usp_CreateUser]",
"CommandType": 4
},
"RetrieveAll": {
"CommandText": "select * from [User];",
"CommandType": 1
}
}
}
]
}
]
}

How to do WaitAll with Akka.Net?

I have a hierarchy of actors in Akka.Net and am wondering whether I've chosen the right way to do something, or if there are better/simpler ways to achieve what I want.
My specific example is that I'm constructing a User actor in response to a user logging into the system, and when constructing this actor there are two pieces of data I need in order to complete the construction of the actor.
If this were regular .NET code I might have something like the following...
public Task<User> LoadUserAsync (string username)
{
IProfileService profileService = ...;
IMessageService messageService = ...;
var loadProfileTask = profileService.GetUserProfileAsync(username);
var loadMessagesTask = messageService.GetMessagesAsync(username);
Task.WaitAll(loadProfileTask, loadMessagesTask);
// Now construct the user from the result of both tasks
var user = new User
{
Profile = loadProfileTask.Result,
Messages = loadMessagesTask.Result
}
return Task.FromResult(user);
}
Here I use WaitAll to wait for the subordinate tasks to complete, and let them run concurrently.
My question is - if I wanted to do the same in Akka.Net, would the following be the most regular way to do this? Pictorially I've created the following...
When I create my User actor, I then construct a (temporary) User Loader Actor, whose job it is to get the full user details by calling to the Profile actor and the Messages actor. The leaf actors that get the data are as follows...
public class UserProfileLoader : ReceiveActor
{
public UserProfileLoader()
{
Receive<LoadUserRequest>(msg =>
{
// Load the user profile from somewhere
var profile = new UserProfile();
// And respond to the Sender
Sender.Tell(profile);
Self.Tell(PoisonPill.Instance);
});
}
}
public class UserMessagesLoader : ReceiveActor
{
public UserMessagesLoader()
{
Receive<LoadUserRequest>(msg =>
{
// Load the messages from somewhere
var messages = new List<Message>();
// And respond to the Sender
Sender.Tell(messages);
Self.Tell(PoisonPill.Instance);
});
}
}
It doesn't really matter where they get the data from for this discussion, but both simply respond to a request by returning some data.
Then I have the actor that coordinates the two data gathering actors...
public class UserLoaderActor : ReceiveActor
{
public UserLoaderActor()
{
Receive<LoadUserRequest>(msg => LoadProfileAndMessages(msg));
Receive<UserProfile>(msg =>
{
_profile = msg;
FinishIfPossible();
});
Receive<List<Message>>(msg =>
{
_messages = msg;
FinishIfPossible();
});
}
private void LoadProfileAndMessages(LoadUserRequest msg)
{
_originalSender = Sender;
Context.ActorOf<UserProfileLoader>().Tell(msg);
Context.ActorOf<UserMessagesLoader>().Tell(msg);
}
private void FinishIfPossible()
{
if ((null != _messages) && (null != _profile))
{
_originalSender.Tell(new LoadUserResponse(_profile, _messages));
Self.Tell(PoisonPill.Instance);
}
}
private IActorRef _originalSender;
private UserProfile _profile;
private List<Message> _messages;
}
This just creates the two subordinate actors, sends them a message to get cracking, and then waits for both to respond before sending back all the data that's been gathered to the original requestor.
So, does this seem like a reasonable way to coordinate two disparate responses, in order to combine them? Is there an easier way to do this than craft it up myself?
Thanks in advance for your responses!
Thanks folks, so I've now simplified the actor significantly into the following, based on both Roger and Jeff's suggestions...
public class TaskBasedUserLoader : ReceiveActor
{
public TaskBasedUserLoader()
{
Receive<LoadUserRequest>(msg => LoadProfileAndMessages(msg));
}
private void LoadProfileAndMessages(LoadUserRequest msg)
{
var originalSender = Sender;
var loadPreferences = this.LoadProfile(msg.UserId);
var loadMessages = this.LoadMessages(msg.UserId);
Task.WhenAll(loadPreferences, loadMessages)
.ContinueWith(t => new UserLoadedResponse(loadPreferences.Result, loadMessages.Result),
TaskContinuationOptions.AttachedToParent & TaskContinuationOptions.ExecuteSynchronously)
.PipeTo(originalSender);
}
private Task<UserProfile> LoadProfile(string userId)
{
return Task.FromResult(new UserProfile { UserId = userId });
}
private Task<List<Message>> LoadMessages(string userId)
{
return Task.FromResult(new List<Message>());
}
}
The LoadProfile and LoadMessages methods will ultimately call a repository to get the data, but for now I have a succinct way to do what I wanted.
Thanks again!
IMHO that's a valid process, as you fork action and then join it.
BTW you could use this.Self.GracefulStop(new TimeSpan(1)); instead of sending poison pill.
You could use a combination of Ask, WhenAll and PipeTo:
var task1 = actor1.Ask<Result1>(request1);
var task2 = actor2.Ask<Result2>(request2);
Task.WhenAll(task1, task2)
.ContinueWith(_ => new Result3(task1.Result, task2.Result))
.PipeTo(Self);
...
Receive<Result3>(msg => { ... });

EntityFramework, Insert if not exist, otherwise update

I'm having a Entity-Set Countries, reflecting a database table '<'char(2),char(3),nvarchar(50> in my database.
Im having a parser that returns a Country[] array of parsed countries, and is having issues with getting it updated in the right way. What i want is: Take the array of countries, for those countries not already in the database insert them, and those existing update if any fields is different. How can this be done?
void Method(object sender, DocumentLoadedEvent e)
{
var data = e.ParsedData as Country[];
using(var db = new DataContractEntities)
{
//Code missing
}
}
I was thinking something like
for(var c in data.Except(db.Countries)) but it wount work as it compares on wronge fields.
Hope anyone have had this issues before, and have a solution for me. If i cant use the Country object and insert/update an array of them easy, i dont see much benefict of using the framework, as from performers i think its faster to write a custom sql script that inserts them instead of ect checking if an country is already in the database before inserting?
Solution
See answer of post instead.
I added override equals to my country class:
public partial class Country
{
public override bool Equals(object obj)
{
if (obj is Country)
{
var country = obj as Country;
return this.CountryTreeLetter.Equals(country.CountryTreeLetter);
}
return false;
}
public override int GetHashCode()
{
int hash = 13;
hash = hash * 7 + (int)CountryTreeLetter[0];
hash = hash * 7 + (int)CountryTreeLetter[1];
hash = hash * 7 + (int)CountryTreeLetter[2];
return hash;
}
}
and then did:
var data = e.ParsedData as Country[];
using (var db = new entities())
{
foreach (var item in data.Except(db.Countries))
{
db.AddToCountries(item);
}
db.SaveChanges();
}
I would do it straightforward:
void Method(object sender, DocumentLoadedEvent e)
{
var data = e.ParsedData as Country[];
using(var db = new DataContractEntities)
{
foreach(var country in data)
{
var countryInDb = db.Countries
.Where(c => c.Name == country.Name) // or whatever your key is
.SingleOrDefault();
if (countryInDb != null)
db.Countries.ApplyCurrentValues(country);
else
db.Countries.AddObject(country);
}
db.SaveChanges();
}
}
I don't know how often your application must run this or how many countries your world has. But I have the feeling that this is nothing where you must think about sophisticated performance optimizations.
Edit
Alternative approach which would issue only one query:
void Method(object sender, DocumentLoadedEvent e)
{
var data = e.ParsedData as Country[];
using(var db = new DataContractEntities)
{
var names = data.Select(c => c.Name);
var countriesInDb = db.Countries
.Where(c => names.Contains(c.Name))
.ToList(); // single DB query
foreach(var country in data)
{
var countryInDb = countriesInDb
.SingleOrDefault(c => c.Name == country.Name); // runs in memory
if (countryInDb != null)
db.Countries.ApplyCurrentValues(country);
else
db.Countries.AddObject(country);
}
db.SaveChanges();
}
}
The modern form, using later EF versions would be:
context.Entry(record).State = (AlreadyExists ? EntityState.Modified : EntityState.Added);
context.SaveChanges();
AlreadyExists can come from checking the key or by querying the database to see whether the item already exists there.
You can implement your own IEqualityComparer<Country> and pass that to the Except() method. Assuming your Country object has Id and Name properties, one example of that implementation could look like this:
public class CountryComparer : IEqualityComparer<Country>
{
public bool Equals(Country x, Country y)
{
return x.Name.Equals(y.Name) && (x.Id == y.Id);
}
public int GetHashCode(Country obj)
{
return string.Format("{0}{1}", obj.Id, obj.Name).GetHashCode();
}
}
and use it as
data.Countries.Except<Country>(db, new CountryComparer());
Although, in your case it looks like you just need to extract new objects, you can use var newCountries = data.Where(c => c.Id == Guid.Empty); if your Id is Guid.
The best way is to inspect the Country.EntityState property and take actions from there regarding on value (Detached, Modified, Added, etc.)
You need to provide more information on what your data collection contains i.e. are the Country objects retrieved from a database through the entityframework, in which case their context can be tracked, or are you generating them using some other way.
I am not sure this will be the best solution but I think you have to get all countries from DB then check it with your parsed data
void Method(object sender, DocumentLoadedEvent e)
{
var data = e.ParsedData as Country[];
using(var db = new DataContractEntities)
{
List<Country> mycountries = db.Countries.ToList();
foreach(var PC in data)
{
if(mycountries.Any( C => C.Name==PC.Name ))
{
var country = mycountries.Any( C => C.Name==PC.Name );
//Update it here
}
else
{
var newcountry = Country.CreateCountry(PC.Name);//you must provide all required parameters
newcountry.Name = PC.Name;
db.AddToCountries(newcountry)
}
}
db.SaveChanges();
}
}

SQL CLR stored procedure output

there is a simple class called User and List of its objects
public class User
{
public int ID;
public string UserName;
public string UserPassword;
}
...
List userList = new List();
Can i make this list of User objects as result of execution SLQ CLR stored procedure ?
e.g. i want to get this
ID UserName UserPassword
1 Ted SomePassword
2 Sam Password2
3 Bill dsdsd
[SqlProcedure]
public static void GetAllocations()
{
// what here ??
}
P.S. Please do not advice me to use Sql functions. It does not suit me because it does not support output parameters
P.S.2 i will be very appreciated for any help !
Try to create a virtual table with SqlDataRecord and send it over the Pipe property of SqlContext object:
[SqlProcedure]
public static void GetAllocations()
{
// define table structure
SqlDataRecord rec = new SqlDataRecord(new SqlMetaData[] {
new SqlMetaData("ID", SqlDbType.Int),
new SqlMetaData("UserName", SqlDbType.VarChar),
new SqlMetaData("UserPassword", SqlDbType.VarChar),
});
// start sending and tell the pipe to use the created record
SqlContext.Pipe.SendResultsStart(rec);
{
// send items step by step
foreach (User user in GetUsers())
{
int id = user.ID;
string userName = user.UserName;
string userPassword = user.UserPassword;
// set values
rec.SetSqlInt32(0, id);
rec.SetSqlString(1, userName);
rec.SetSqlString(2, userPassword);
// send new record/row
SqlContext.Pipe.SendResultsRow(rec);
}
}
SqlContext.Pipe.SendResultsEnd(); // finish sending
}