RavenDB lazy search against Index returns uninitialized statistiscs - ravendb

I am trying to run lazy queries against raven db and get the counts on total matching results. I am finding when I query against a static index, a lazy search does not initialize the statistics when the query is materialized, but otherwise it comes back all right.
Below is the test to prove this behaviour.
[TestFixture]
public class CanSearchLazily
{
private const int ServerPort = 8085;
private readonly string _serverAddress = #"http://localhost:{0}".For(ServerPort);
[Test]
public void CanGetTotalResultsFromStatisticsOnLazySearchAgainstDynamicIndex()
{
CanGetTotalResultsFromStatisticsOnLazySearchAgainstAnIndex();
}
[Test]
public void CanGetTotalResultsFromStatisticsOnLazySearchAgainstStaticIndex()
{
CanGetTotalResultsFromStatisticsOnLazySearchAgainstAnIndex("UserByFirstName");
}
private void CanGetTotalResultsFromStatisticsOnLazySearchAgainstAnIndex(string indexName = "")
{
BuilderSetup.DisablePropertyNamingFor<User, string>(x => x.Id);
var users = Builder<User>.CreateListOfSize(2000).All()
.With(x => x.FirstName = GetRandom.FirstName())
.With(x => x.LastName = GetRandom.LastName())
.Build();
using (GetNewServer())
using (var store = new DocumentStore { Url = _serverAddress }.Initialize())
{
using (var session = store.OpenSession())
{
users.ForEach(session.Store);
session.SaveChanges();
IndexCreation.CreateIndexes(typeof(UserByFirstName).Assembly, store);
session.Query<User, UserByFirstName>().Customize(x => x.WaitForNonStaleResults()).ToList();
}
using (var session = store.OpenSession())
{
var names = session.Query<User>().Select(u => u.FirstName).Distinct().Take(15).ToList();
RavenQueryStatistics stats;
var query = string.IsNullOrEmpty(indexName)
? session.Query<User>().Statistics(out stats).Where(x => x.FirstName.In(names))
: session.Query<User>(indexName).Statistics(out stats).Where(x => x.FirstName.In(names));
var results = query.Take(8).Lazily();
Assert.AreEqual(8, results.Value.ToList().Count);
Assert.AreEqual(DateTime.Now.Year, stats.IndexTimestamp.Year, "the index should have the current year on its timestamp");
Assert.IsTrue(stats.TotalResults > 0, "The stats should return total results");
}
}
}
protected RavenDbServer GetNewServer(bool initializeDocumentsByEntitiyName = true)
{
var ravenConfiguration = new RavenConfiguration
{
Port = ServerPort,
RunInMemory = true,
DataDirectory = "Data",
AnonymousUserAccessMode = AnonymousUserAccessMode.All
};
if (ravenConfiguration.RunInMemory == false)
IOExtensions.DeleteDirectory(ravenConfiguration.DataDirectory);
var ravenDbServer = new RavenDbServer(ravenConfiguration);
if (initializeDocumentsByEntitiyName)
{
using (var documentStore = new DocumentStore
{
Url = _serverAddress
}.Initialize())
{
new RavenDocumentsByEntityName().Execute(documentStore);
}
}
return ravenDbServer;
}
}
[Serializable]
public class User
{
public string Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
public class UserByFirstName : AbstractIndexCreationTask<User>
{
public UserByFirstName()
{
Map = users => from user in users
select new {user.FirstName};
}
}

Related

Data variable is equal to null, even though there are elements in the database

I'm seeding sql tables with the use of context factory, It shows up and there is an element in mssql, but when I write a controller, I get an empty list with no entries in the table. Could it perhaps be the inheritance that is causing an issue? Classical concert is inherited from Concert.
ConcertController:
public class ConcertController : Controller
{
private readonly AppDbContext _context;
public ConcertController(AppDbContext context)
{
_context = context;
}
public IActionResult Index()
{
var data = _context.Concerts.ToList(); //this variable is null
return View();
}
}
DbContext:
public class AppDbContext : DbContext
{
public virtual DbSet<Concert> Concerts { get; set; }
public virtual DbSet<Ticket> Tickets { get; set; }
public AppDbContext(DbContextOptions<AppDbContext> options) : base(options)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Party>().HasBaseType<Concert>();
modelBuilder.Entity<ClassicalConcert>().HasBaseType<Concert>();
base.OnModelCreating(modelBuilder);
}
}
Data seeding:
var factory = new AppContextFactory();
using var context = factory.CreateDbContext();
var builder = WebApplication.CreateBuilder(args);
await AddData();
async Task AddData()
{
ClassicalConcert mozart;
await context.AddRangeAsync(new[]
{
mozart = new ClassicalConcert()
{
PerformerName = "Some random dude",
TicketsCount= 1000,
PerformanceDate= DateTime.Now,
Location = "Rnd location",
Description = "some rnd desc",
ImageURL = "some https",
VoiceType = Centaurea.Data.Enums.VoiceTypes.Bass,
ConcertName = "Mozarts back",
ComposersName = "Mozart",
}
});
await context.SaveChangesAsync();
}
Context Factory:
public class AppContextFactory : IDesignTimeDbContextFactory<AppDbContext>
{
public AppDbContext CreateDbContext(string[] args = null)
{
var configuration = new ConfigurationBuilder().AddJsonFile("appsettings.json").Build();
var optionsBuilder = new DbContextOptionsBuilder<AppDbContext>();
optionsBuilder.UseSqlServer(configuration["ConnectionStrings:DefaultConnection"]);
return new AppDbContext(optionsBuilder.Options);
}
}
Picture of the Debuger:

Autocad.net Save complex Object

I'm working on a project in AutoCAD using c#, my application data is stored in complex objects
(String, double, objectId, arrays, list...) and I would like to save data for later using (serialize or saved in AutoCAD drawing) and if I re-open AutoCAD and reload my project, I can find all data in my object
Sorry for my English
So You need to use XData.
Details and sample You can find here:
https://www.keanw.com/2007/04/adding_xdata_to.html
You could serialize your class into a binary stream and then you can save it in the drawing as a bunch of binary chunks (see this topic)
But most of the time you should directly store data in Xrecords of a DBDictionary.
public abstract class RecordableObject
{
protected ObjectId dictionaryId;
protected Database database;
public string Key { get; }
protected RecordableObject(string key, Database db = null)
{
database = db ?? HostApplicationServices.WorkingDatabase;
Key = key;
using (var tr = database.TransactionManager.StartOpenCloseTransaction())
{
var NOD = (DBDictionary)tr.GetObject(database.NamedObjectsDictionaryId, OpenMode.ForRead);
DBDictionary dictionary;
if (NOD.Contains(Key))
{
dictionaryId = NOD.GetAt(Key);
}
else
{
NOD.UpgradeOpen();
dictionary = new DBDictionary();
dictionaryId = NOD.SetAt(Key, dictionary);
tr.AddNewlyCreatedDBObject(dictionary, true);
}
tr.Commit();
}
}
public abstract void SavePropertiesToDictionary();
public abstract void SetPropertiesFromDictionary();
protected void SaveData(string key, params TypedValue[] values)
{
using (var tr = database.TransactionManager.StartOpenCloseTransaction())
{
var dictionary = (DBDictionary)tr.GetObject(dictionaryId, OpenMode.ForRead);
Xrecord xrecord;
if (dictionary.Contains(key))
{
xrecord = (Xrecord)tr.GetObject(dictionary.GetAt(key), OpenMode.ForWrite);
}
else
{
xrecord = new Xrecord();
dictionary.UpgradeOpen();
dictionary.SetAt(key, xrecord);
tr.AddNewlyCreatedDBObject(xrecord, true);
}
xrecord.Data = new ResultBuffer(values);
tr.Commit();
}
}
protected T GetData<T>(string key)
{
using (var tr = database.TransactionManager.StartOpenCloseTransaction())
{
var dictionary = (DBDictionary)tr.GetObject(dictionaryId, OpenMode.ForRead);
if (dictionary.Contains(key))
{
var xrecord = (Xrecord)tr.GetObject(dictionary.GetAt(key), OpenMode.ForRead);
if (xrecord.Data != null)
return (T)xrecord.Data.AsArray()[0].Value;
}
return default;
}
}
protected T[] GetDataArray<T>(string key)
{
using(var tr = database.TransactionManager.StartOpenCloseTransaction())
{
var dictionary = (DBDictionary)tr.GetObject(dictionaryId, OpenMode.ForRead);
if (dictionary.Contains(key))
{
var xrecord = (Xrecord)tr.GetObject(dictionary.GetAt(key), OpenMode.ForRead);
if (xrecord.Data != null)
return xrecord.Data.AsArray().Select(tv => (T)tv.Value).ToArray();
}
return default;
}
}
}
Derived class example:
public class RecordableExample : RecordableObject
{
public double Size { get; set; }
public ObjectId ObjectId { get; set; }
public int[] Ints { get; set; }
public RecordableExample(string key, Database db = null) : base(key, db) { }
public override void SavePropertiesToDictionary()
{
SaveData(nameof(Size), new TypedValue((int)DxfCode.Real, Size));
SaveData(nameof(ObjectId), new TypedValue((int)DxfCode.Handle, ObjectId.Handle));
if (Ints != null)
SaveData(nameof(Ints), Ints.Select(i => new TypedValue((int)DxfCode.Int32, i)).ToArray());
}
public override void SetPropertiesFromDictionary()
{
Size = GetData<double>(nameof(Size));
Ints = GetDataArray<int>(nameof(Ints));
var handle = new Handle(Convert.ToInt64(GetData<string>(nameof(ObjectId))));
if (database.TryGetObjectId(handle, out var id))
ObjectId = id;
}
}

How to keep user logged in after browser is closed

Every time I close the browser I need to log in again into this app. It is developed in .NET Core 2.0. I'm trying to let it logged in, like every other regular site.
I checked this post that may be useful, but since the code is quite different from this application I decided to create this post.
This is my security code:
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Claims;
using System.Security.Principal;
using System.Text;
namespace Petito.Common
{
public interface IActivityContext
{
string ActivityID { get; }
IPrincipal User { get; }
}
[JsonObject(MemberSerialization = MemberSerialization.Fields)]
public class ActivityIdentity : IIdentity
{
private string _name = null;
[JsonIgnore()]
private bool _isAuthenticated = false;
[JsonIgnore()]
private string _authenticationType = "";
public ActivityIdentity()
{
}
public ActivityIdentity(string name) : this(name, false, "")
{
}
internal ActivityIdentity(string name, bool isAuthenticated, string authenticationType)
{
this._name = name;
this._isAuthenticated = isAuthenticated;
this._authenticationType = authenticationType;
}
public string Name { get => _name; }
public bool IsAuthenticated { get => _isAuthenticated; }
public string AuthenticationType { get => _authenticationType; }
public static ActivityIdentity Unathenticated => new ActivityIdentity();
}
[JsonObject(MemberSerialization = MemberSerialization.Fields)]
public class ActivityPrincipal : IPrincipal
{
private ActivityIdentity _activityIdentity = null;
private string[] _roles = null;
public ActivityPrincipal() : this(ActivityIdentity.Unathenticated, null)
{
}
public ActivityPrincipal(ActivityIdentity activityIdentity, params string[] roles)
{
_activityIdentity = activityIdentity;
_roles = roles;
}
public ActivityIdentity Identity => _activityIdentity;
IIdentity IPrincipal.Identity => _activityIdentity;
public bool IsInRole(string role)
{
if (_roles != null && _roles.Length > 0)
{
return _roles.Contains(role);
}
return false;
}
}
[JsonObject(MemberSerialization = MemberSerialization.Fields)]
public class ActivityContext : IDisposable, IActivityContext
{
private string _activityID = Guid.NewGuid().ToString();
private DateTime _startDate = DateTime.UtcNow;
private DateTime? _endDate = null;
private ActivityPrincipal _activityPrincipal = null;
public ActivityContext() : this(null)
{
}
public ActivityContext(IPrincipal principal)
{
_activityPrincipal = Convert(principal);
}
private ActivityPrincipal Convert(IPrincipal principal)
{
if (principal == null)
{
return new ActivityPrincipal();
}
var activityPrincipal = principal as ActivityPrincipal;
if (activityPrincipal != null)
{
return activityPrincipal;
}
var claimsPrincipal = principal as ClaimsPrincipal;
if (claimsPrincipal != null)
{
var roles = claimsPrincipal.Claims.Select(x => x.Value);
var p = new ActivityPrincipal(
new ActivityIdentity(claimsPrincipal.Identity.Name, claimsPrincipal.Identity.IsAuthenticated, claimsPrincipal.Identity.AuthenticationType)
, roles.ToArray()
);
return p;
}
throw new NotSupportedException($"Converting {principal.GetType()} not supported");
}
public void Dispose()
{
if (!_endDate.HasValue)
{
_endDate = DateTime.UtcNow;
}
}
public string ActivityID { get => _activityID; }
public DateTime StartDate { get => _startDate; }
public DateTime? EndDate { get => _endDate; }
public IPrincipal User
{
get
{
return _activityPrincipal;
}
}
}
}
Of course, I'll still try to figure out what's wrong with the code. Please if you need another part of the code other from that I posted let me know.
Thanks!

Getting email from oauth authentication (Microsoft)

How can I get the email from microsoft account? I'm doing the following:
public ActionResult ExternalLoginCallback(string returnUrl)
{
AuthenticationResult result = OAuthWebSecurity.VerifyAuthentication(Url.Action("ExternalLoginCallback", new { ReturnUrl = returnUrl }));
//...
string email = null;
if (result.Provider.ToLower() == "google")
{
email = result.ExtraData["email"];
}
else if (result.Provider.ToLower() == "facebook")
{
email = result.ExtraData["username"];
}
else if (result.Provider.ToLower() == "microsoft")
{
email = result.ExtraData["????"];
}
}
For google and facebook I'm able to get the email but I can't with microsoft? What kew should I use?
Solution:
public class MicrosoftScopedClient : IAuthenticationClient
{
private string clientId;
private string clientSecret;
private string scope;
private const string baseUrl = "https://login.live.com/oauth20_authorize.srf";
private const string tokenUrl = "https://login.live.com/oauth20_token.srf";
public MicrosoftScopedClient(string clientId, string clientSecret, string scope)
{
this.clientId = clientId;
this.clientSecret = clientSecret;
this.scope = scope;
}
public string ProviderName
{
get { return "Microsoft"; }
}
public void RequestAuthentication(HttpContextBase context, Uri returnUrl)
{
string url = baseUrl + "?client_id=" + clientId + "&redirect_uri=" + HttpUtility.UrlEncode(returnUrl.ToString()) + "&scope=" + HttpUtility.UrlEncode(scope) + "&response_type=code";
context.Response.Redirect(url);
}
public AuthenticationResult VerifyAuthentication(HttpContextBase context)
{
string code = context.Request.QueryString["code"];
string rawUrl = context.Request.Url.ToString();
//From this we need to remove code portion
rawUrl = Regex.Replace(rawUrl, "&code=[^&]*", "");
IDictionary<string, string> userData = GetUserData(code, rawUrl);
if (userData == null)
return new AuthenticationResult(false, ProviderName, null, null, null);
string id = userData["id"];
string username = userData["email"];
userData.Remove("id");
userData.Remove("email");
AuthenticationResult result = new AuthenticationResult(true, ProviderName, id, username, userData);
return result;
}
private IDictionary<string, string> GetUserData(string accessCode, string redirectURI)
{
string token = QueryAccessToken(redirectURI, accessCode);
if (token == null || token == "")
{
return null;
}
var userData = GetUserData(token);
return userData;
}
private IDictionary<string, string> GetUserData(string accessToken)
{
ExtendedMicrosoftClientUserData graph;
var request =
WebRequest.Create(
"https://apis.live.net/v5.0/me?access_token=" + EscapeUriDataStringRfc3986(accessToken));
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (StreamReader sr = new StreamReader(responseStream))
{
string data = sr.ReadToEnd();
graph = JsonConvert.DeserializeObject<ExtendedMicrosoftClientUserData>(data);
}
}
}
var userData = new Dictionary<string, string>();
userData.Add("id", graph.Id);
userData.Add("username", graph.Name);
userData.Add("name", graph.Name);
userData.Add("link", graph.Link == null ? null : graph.Link.AbsoluteUri);
userData.Add("gender", graph.Gender);
userData.Add("firstname", graph.FirstName);
userData.Add("lastname", graph.LastName);
userData.Add("email", graph.Emails.Preferred);
return userData;
}
private string QueryAccessToken(string returnUrl, string authorizationCode)
{
var entity =
CreateQueryString(
new Dictionary<string, string> {
{ "client_id", this.clientId },
{ "redirect_uri", returnUrl },
{ "client_secret", this.clientSecret},
{ "code", authorizationCode },
{ "grant_type", "authorization_code" },
});
WebRequest tokenRequest = WebRequest.Create(tokenUrl);
tokenRequest.ContentType = "application/x-www-form-urlencoded";
tokenRequest.ContentLength = entity.Length;
tokenRequest.Method = "POST";
using (Stream requestStream = tokenRequest.GetRequestStream())
{
var writer = new StreamWriter(requestStream);
writer.Write(entity);
writer.Flush();
}
HttpWebResponse tokenResponse = (HttpWebResponse)tokenRequest.GetResponse();
if (tokenResponse.StatusCode == HttpStatusCode.OK)
{
using (Stream responseStream = tokenResponse.GetResponseStream())
{
using (StreamReader sr = new StreamReader(responseStream))
{
string data = sr.ReadToEnd();
var tokenData = JsonConvert.DeserializeObject<OAuth2AccessTokenData>(data);
if (tokenData != null)
{
return tokenData.AccessToken;
}
}
}
}
return null;
}
private static readonly string[] UriRfc3986CharsToEscape = new[] { "!", "*", "'", "(", ")" };
private static string EscapeUriDataStringRfc3986(string value)
{
StringBuilder escaped = new StringBuilder(Uri.EscapeDataString(value));
// Upgrade the escaping to RFC 3986, if necessary.
for (int i = 0; i < UriRfc3986CharsToEscape.Length; i++)
{
escaped.Replace(UriRfc3986CharsToEscape[i], Uri.HexEscape(UriRfc3986CharsToEscape[i][0]));
}
// Return the fully-RFC3986-escaped string.
return escaped.ToString();
}
private static string CreateQueryString(IEnumerable<KeyValuePair<string, string>> args)
{
if (!args.Any())
{
return string.Empty;
}
StringBuilder sb = new StringBuilder(args.Count() * 10);
foreach (var p in args)
{
sb.Append(EscapeUriDataStringRfc3986(p.Key));
sb.Append('=');
sb.Append(EscapeUriDataStringRfc3986(p.Value));
sb.Append('&');
}
sb.Length--; // remove trailing &
return sb.ToString();
}
protected class ExtendedMicrosoftClientUserData
{
public string FirstName { get; set; }
public string Gender { get; set; }
public string Id { get; set; }
public string LastName { get; set; }
public Uri Link { get; set; }
public string Name { get; set; }
public Emails Emails { get; set; }
}
protected class Emails
{
public string Preferred { get; set; }
public string Account { get; set; }
public string Personal { get; set; }
public string Business { get; set; }
}
}
AuthConfig.cs
public static class AuthConfig
{
public static void RegisterAuth()
{
Dictionary<string, object> MicrosoftsocialData = new Dictionary<string, object>();
MicrosoftsocialData.Add("Icon", "../Content/icons/microsoft.png");
OAuthWebSecurity.RegisterClient(new MicrosoftScopedClient("XXXXXXXX", "YYYYYYYYYYYYY",
"wl.basic wl.emails"), "Microsoft", MicrosoftsocialData);
//......
}
}
Usage:
public ActionResult ExternalLoginCallback(string returnUrl)
{
AuthenticationResult result = OAuthWebSecurity.VerifyAuthentication(Url.Action("ExternalLoginCallback", new { ReturnUrl = returnUrl }));
//...
string email = null;
if (result.Provider.ToLower() == "google")
{
email = result.ExtraData["email"];
}
else if (result.Provider.ToLower() == "facebook")
{
email = result.ExtraData["username"];
}
else if (result.Provider.ToLower() == "microsoft")
{
email = result.UserName;
}
}
Based on: How OAuthWebSecurity to obtain emails for different oauth clients, but Microsoft Client doesn’t return email, it didn’t include scope “wl.emails”
or even simpler: https://stackoverflow.com/a/22723713/1586498
var mo =
new Microsoft.Owin.Security.MicrosoftAccount.MicrosoftAccountAuthenticationOptions
{
CallbackPath = new Microsoft.Owin.PathString("/Callbacks/External"),//register at oAuth provider
ClientId = "<<yourclientid>>",
ClientSecret = "<<yourclientsecret>>",
Provider = new Microsoft.Owin.Security.MicrosoftAccount.MicrosoftAccountAuthenticationProvider
{
OnAuthenticated = (context) =>
{
context.Identity.AddClaim(new Claim(providerKey, context.Identity.AuthenticationType));
context.Identity.AddClaim(new Claim(ClaimTypes.Name, context.Identity.FindFirstValue(ClaimTypes.Name)));
return System.Threading.Tasks.Task.FromResult(0);
}
}
};
mo.Scope.Add("wl.basic");
mo.Scope.Add("wl.emails"); //HERE IS THE GOLD
app.UseMicrosoftAccountAuthentication(mo);
and my way of grabbing them:
var externalIdentity = await AuthenticationManager.GetExternalIdentityAsync(DefaultAuthenticationTypes.ExternalCookie);
externalIdentity.Claims.FirstOrDefault(c => c.Type.Equals(ClaimTypes.Email));
amp's answer really helped me out.
Also want to mention that you have to check the 'Live SDK support' checkbox when you register your application (https://apps.dev.microsoft.com/) - otherwise the OAuth service complains that you don't have a client secret (even if you do).
Just wanted to add how to do this without using the AuthConfig.cs stuff in case anyone is interested (a bit more manual, but it makes it easier to understand if you're not familiar with the framework):
public ActionResult LoginWithMicrosoftAccount(CancellationToken cancellationToken)
{
var client = new MicrosoftScopedClient(appID, appsecret, "wl.basic wl.emails");
var urlNoQueryString = Request.Url.GetLeftPart(UriPartial.Path);
AuthenticationResult result = null;
if(Request.QueryString["error"]!= null)
{//Microsoft service returns error
return View();
}
if (Request.QueryString["code"] != null)
{
result = client.VerifyAuthentication(this.HttpContext);
//at this point, you should get the username from result.UserName
}
if(Request.QueryString["code"]==null || result.UserName == null)
{//will do the redirection
client.RequestAuthentication(this.HttpContext, new Uri(urlNoQueryString));
}
return View();
}

RavenDB static index on dictionary

I have an application that uses documents, that contain list of attributes in a dictionary, for some reason we need to use a static index and query/filter over these attributes.
A prototype looks like this:
class Program
{
static void Main(string[] args)
{
IDocumentStore store = new DocumentStore() { DefaultDatabase = "Test", Url = "http://localhost:8081" };
store.Initialize();
IndexCreation.CreateIndexes(typeof(Program).Assembly, store);
using (var session = store.OpenSession())
{
session.Store(new Document { Id = "1", Name = "doc_name", Attributes = new Dictionary<string, object> { { "Type", "1" }, { "Status", "Active" } } });
session.SaveChanges();
}
using (var session = store.OpenSession())
{
// works
var l1 = session.Query<Document, Documents_Index>().Where(a => a.Attributes["Type"] == "1").ToList();
// not working
var l2 = session.Query<Document, Documents_Index>().Where(a => a.Attributes["Status"] == "Active").ToList();
}
}
}
public class Documents_Index : AbstractIndexCreationTask<Document>
{
public Documents_Index()
{
Map = docs => docs.Select(a =>
new
{
a.Name,
a.Attributes,
Attributes_Type = a.Attributes["Type"]
});
}
}
[Serializable]
public class Document
{
public string Id { get; set; }
public string Name { get; set; }
public Dictionary<string, object> Attributes { get; set; }
}
But since I need to query using any arbitrary Attribute name/value this index does solve our problem. Actually the list of attributes is known at run-time (so we tried modifying the Map expression to inject any number of attribute names, but so far we weren't successful). Is there a way how to define the index in some dynamic fashion?
You need to write it like:
public class Documents_Index : AbstractIndexCreationTask<Document>
{
public Documents_Index()
{
Map = docs => docs.Select(a =>
new
{
a.Name,
_ = a.Attributes.Select(x=>CreateField("Attributes_"+x.Key, x.Value),
});
}
}