I have added the simplest of indexes, like this:
public class MyDocType_ById : AbstractIndexCreationTask<MyDocType>
{
public MyDocType_ById()
{
Map = myDocs => from mine in myDocs
select new
{
Id = mine.Id
};
Index(x => x.Id, FieldIndexing.Analyzed);
}
}
After compiling and hitting my site, I get the following exception:
[IndexCompilationException: Could not find file 'C:\dev\db\ravendb\CompiledIndexCache\-1836739954.u6DpcVRxqMXarC7cwg26Jg%3d%3d.nodebug.dll'.]
I am creating indexes on my app-startup using
IndexCreation.CreateIndexes(typeof(MyDocType_ById).Assembly, IoC.Container.GetInstance<IDocumentStore>());
In the folder a file is created named -1836739954.u6DpcVRxqMXarC7cwg26Jg%3d%3d.nodebug.dll.cs but the dll is not there.
Steps tried:
ensure correct folder permissions on C:\dev\db\ravendb\CompiledIndexCache\
iisreset + restart ravendb (Windows service)
renaming the index
saying for [CURSE] sake out loud in the office
Any ideas how to fix this?
Thorough googling led me to just 1 possible solution found here (where Ayende has also suggested checking folder permissions, ensure no indexing service/antivirus/backup et.c. is touching the folder): https://groups.google.com/forum/#!topic/ravendb/awh7eV00QNE
OP ended up fixing this by rebooting his machine.
This solved it for me too (although I surely am curious as to how rebooting differs from restarting all services, et.c...).
I tried to recreate the bug by adding an identical documenttype + index, and this time around it worked correctly right away. Only thing left is hoping this will never reoccur (especially not during e.g. production release...).
Update: this seems to occur occasionally on any of my dev-machines only after waking the system from hibernating (Windows 10), so another solution might be to not hibernate your dev-machine...
Related
Is there a way to make the logged user (on superset) to make the queries on impala?
I tried to enable the "Impersonate the logged on user" option on Databases but with no success because all the queries run on impala with superset user.
I'm trying to achieve the same! This will not completely answer this question since it does not still work but I want to share my research in order to maybe help another soul that is trying to use this instrument outside very basic use cases.
I went deep in the code and I found out that impersonation is not implemented for Impala. So you cannot achieve this from the UI. I found out this PR https://github.com/apache/superset/pull/4699 that for whatever reason was never merged into the codebase and tried to copy&paste code in my Superset version (1.1.0) but it didn't work. Adding some logs I can see that the configuration with the impersonation is updated, but then the actual Impala query is with the user I used to start the process.
As you can imagine, I am a complete noob at this. However I found out that the impersonation thing happens when you create a cursor and there is a constructor parameter in which you can pass the impersonation configuration.
I managed to correctly (at least to my understanding) implement impersonation for the SQL lab part.
In the sql_lab.py class you have to add in the execute_sql_statements method the following lines
with closing(engine.raw_connection()) as conn:
# closing the connection closes the cursor as well
cursor = conn.cursor(**database.cursor_kwargs)
where cursor_kwargs is defined in db_engine_specs/impala.py as the following
#classmethod
def get_configuration_for_impersonation(cls, uri, impersonate_user, username):
logger.info(
'Passing Impala execution_options.cursor_configuration for impersonation')
return {'execution_options': {
'cursor_configuration': {'impala.doas.user': username}}}
#classmethod
def get_cursor_configuration_for_impersonation(cls, uri, impersonate_user,
username):
logger.debug('Passing Impala cursor configuration for impersonation')
return {'configuration': {'impala.doas.user': username}}
Finally, in models/core.py you have to add the following bit in the get_sqla_engine def
params = extra.get("engine_params", {}) # that was already there just for you to find out the line
self.cursor_kwargs = self.db_engine_spec.get_cursor_configuration_for_impersonation(
str(url), self.impersonate_user, effective_username) # this is the line I added
...
params.update(self.get_encrypted_extra()) # already there
#new stuff
configuration = {}
configuration.update(
self.db_engine_spec.get_configuration_for_impersonation(
str(url),
self.impersonate_user,
effective_username))
if configuration:
params.update(configuration)
As you can see I just shamelessy pasted the code from the PR. However this kind of works only for the SQL lab as I already said. For the dashboards there is an entirely different way of querying Impala that I did not still find out.
This means that queries for the dashboards are handled in a different way and there isn't something like this
with closing(engine.raw_connection()) as conn:
# closing the connection closes the cursor as well
cursor = conn.cursor(**database.cursor_kwargs)
My gut (and debugging) feeling is that you need to first understand the sqlalchemy part and extend a new ImpalaEngine class that uses a custom cursor with the impersonation conf. Or something like that, however it is not simple (if we want to call this simple) as the sql_lab part. So, the trick is to find out where the query is executed and create a cursor with the impersonation configuration. Easy, isnt'it ?
I hope that this could shed some light to you and the others that have this issue. Let me know if you did find out another way to solve this issue, or if this comment was useful.
Update: something really useful
A colleague of mine succesfully implemented impersonation with impala without touching any superset related, but instead working directly with the impyla lib. A PR was open with the code to change. You can apply the patch directly in the impyla src used by superset. You have to edit both dbapi.py and hiveserver2.py.
As a reminder: we are still testing this and we do not know if it works with different accounts using the same superset instance.
I am using NodaTime's LocalDate in RavenDB index.
Here is an example of the index:
public class TaskIndex : AbstractIndexCreationTask<ScheduleTask>
{
public TaskIndex()
{
Map = tasks => from task in tasks
select new
{
task.Name,
PlannedStartDate = task.PlannedStartDate.AsLocalDate().Resolve(),
PlannedDueDate = task.PlannedDueDate.AsLocalDate().Resolve()
};
Index(x => x.Name, FieldIndexing.Analyzed);
Store(x => x.Name, FieldStorage.Yes);
TermVector(x => x.Name, FieldTermVector.WithPositionsAndOffsets);
}
}
I installed RavenDB-NodaTime bundle as described here.
Here is a piece of code I use to install index:
var assembly = AppDomain.CurrentDomain.Load(new AssemblyName
{
Name = "cs.Scheduling"
});
var catalog = new AssemblyCatalog(assembly);
var provider = new CompositionContainer(catalog);
var commands = documentStore.DatabaseCommands.ForDatabase(dbName);
IndexCreation.CreateIndexes(provider, commands, documentStore.Conventions);
documentStore is configured with default database, but then I use it to install index to different (tenant) database name of which comes in dbName.
During the installation of the index I got an exception: The name 'NodaTimeField' does not exist in the current context.
I have one default database which is completely different from database I try to install index for. So basically the case is similar to one described here but I am using standalone version of RavenDB server.
I tried to find out how I can do suggested there but was not able to do that:
embeddableDocumentStore.ServerIfEmbedded.Options.DatabaseLandlord.SetupTenantConfiguration += configuration =>
{
configuration.Catalog.Catalogs.Add(new TypeCatalog(typeof(DeleteOnConflict)));
configuration.Catalog.Catalogs.Add(new TypeCatalog(typeof(PutOnConflict)));
};
Version of RavenDB I am using is 2.5.2956.
RavenDB.Client.NodaTime - 2.5.10.
Hope for your help. Thanks.
In my case that was a very silly mistake. When I was installing RavenDB server some time ago I installed it into non-default destination. Later some RavenDB updates were installed into default destination (i.e. \Program Files (x86)\RavenDB). And when I was installing RavenDB-NodaTime bundle I put it into incorrect destination (\Program Files (x86)\RavenDB).
After detecting this issue and configuring RavenDB server in my correct destination properly an error described in the heading has gone away.
Hope this answer can help somebody else.
P.S. Later there was a deserialization error during reading data from db (RavenDB was not aware of how to deserialize date from string in "yyyy-MM-dd" format to LocalDate object) which I fixed by calling store.ConfigureForNodaTime(DateTimeZoneProviders.Tzdb); after store.Initialize(); call as Steven suggested in his answer.
I believe the answer is that your tenant database does not have the bundle "activated" Your database document (under settings in Raven 3) should have something like
"Raven/ActiveBundles": "Encryption;Compression;NodaTime"
Also you must call
store.ConfigureForNodaTime(DateTimeZoneProviders.Tzdb);
I call this after store.Initialize(). Once you do both of these things, you may have to fix existing data by re-saving your documents (not sure if there is another way). New data will be properly stored like '2016-2-3' format which should make your index return data.
I have been consolidating 3 Databases into one via prefixes in my mediawiki installation. I got three wikis using the same database like so:
en_interwiki
de_interwiki
es_interwiki
Everything works fine out of visitor perspective... but whenever a USER wants to post a new article or commit edits, the database takes up to 35 seconds to respond. This is unacceptable.
I activated debugging like so:
# Debugging:
$wgDBerrorLog = '/var/log/mediawiki/WikiDBerror.log';
$wgShowSQLErrors = true;
$wgDebugDumpSql = true;
$wgDebugLogFile = '/var/log/mediawiki/WikiDebug.log';
$wgShowDBErrorBacktrace = true;
I am getting debug info, and it seems that pagelinks is the culprit, but i am not one hundred percent sure.
Did anyone ever have this issue before?
Please help me!
Best regards,
Max
I could fix it. In my case, the memcache had the wrong port. Everything is back to normal.
In case anyone uses memcache with their MediaWiki installation: Be sure to use the right port on your server, or you will end up like me, with 30 second-wait-times.
I noticed this in the debug environment where I have to do many re-installs in order to test persistent data storage, initial settings, etc... It may not be relevant in production, but I mention this anyway just to inform other developers.
Any files created by an app in its App Folder are not 'visible' to queries after manual un-install / re-install (from IDE, for instance). The same applies to the 'Encoded DriveID' - it is no longer valid.
It is probably 'by design' but it effectively creates 'orphans' in the app folder until manually cleaned by 'drive.google.com > Manage Apps > [yourapp] > Options > Delete hidden app data'. It also creates problem if an app relies on finding of files by metadata, title, ... since these seem to be gone. As I said, not a production problem, but it can create some frustration during development.
Can any of friendly Googlers confirm this? Is there any other way to get to these files after re-install?
Try this approach:
Use requestSync() in onConnected() as:
#Override
public void onConnected(Bundle connectionHint) {
super.onConnected(connectionHint);
Drive.DriveApi.requestSync(getGoogleApiClient()).setResultCallback(syncCallback);
}
Then, in its callback, query the contents of the drive using:
final private ResultCallback<Status> syncCallback = new ResultCallback<Status>() {
#Override
public void onResult(#NonNull Status status) {
if (!status.isSuccess()) {
showMessage("Problem while retrieving results");
return;
}
query = new Query.Builder()
.addFilter(Filters.and(Filters.eq(SearchableField.TITLE, "title"),
Filters.eq(SearchableField.TRASHED, false)))
.build();
Drive.DriveApi.query(getGoogleApiClient(), query)
.setResultCallback(metadataCallback);
}
};
Then, in its callback, if found, retrieve the file using:
final private ResultCallback<DriveApi.MetadataBufferResult> metadataCallback =
new ResultCallback<DriveApi.MetadataBufferResult>() {
#SuppressLint("SetTextI18n")
#Override
public void onResult(#NonNull DriveApi.MetadataBufferResult result) {
if (!result.getStatus().isSuccess()) {
showMessage("Problem while retrieving results");
return;
}
MetadataBuffer mdb = result.getMetadataBuffer();
for (Metadata md : mdb) {
Date createdDate = md.getCreatedDate();
DriveId driveId = md.getDriveId();
}
readFromDrive(driveId);
}
};
Job done!
Hope that helps!
It looks like Google Play services has a problem. (https://stackoverflow.com/a/26541831/2228408)
For testing, you can do it by clearing Google Play services data (Settings > Apps > Google Play services > Manage Space > Clear all data).
Or, at this time, you need to implement it by using Drive SDK v2.
I think you are correct that it is by design.
By inspection I have concluded that until an app places data in the AppFolder folder, Drive does not sync down to the device however much to try and hassle it. Therefore it is impossible to check for the existence of AppFolder placed by another device, or a prior implementation. I'd assume that this was to try and create a consistent clean install.
I can see that there are a couple of strategies to work around this:
1) Place dummy data on AppFolder and then sync and recheck.
2) Accept that in the first instance there is the possibility of duplicates, as you cannot access the existing file by definition you will create a new copy, and use custom metadata to come up with a scheme to differentiate like-named files and choose which one you want to keep (essentially implement your conflict merge strategy across the two different files).
I've done the second, I have an update number to compare data from different devices and decide which version I want so decide whether to upload, download or leave alone. As my data is an SQLite DB I also have some code to only sync once updates have settled down and I deliberately consider people updating two devices at once foolish and the results are consistent but undefined as to which will win.
I have a completely empty RavenHQ database that's linked to my Appharbor application. The amount of space the database is currently using is 1.1mb out of an available 25mb for my bronze account. The database previously had records in it, but I have deleted them using "delete collection" in the management studio.
The very first time I call session.Store(myobject), and BEFORE I call .SaveChanges(), I get the following error.
System.InvalidOperationException: Url: "/docs/Raven/Hilo/AccItems"
Raven.Database.Exceptions.OperationVetoedException: PUT vetoed by Raven.Bundles.Quotas.Triggers.DatabaseSizeQoutaForDocumetsPutTrigger because: Database size is 45,347 KB, which is over the allowed quota of 25,600 KB. No more documents are allowed in.
Now, the document is definitely not that big, so I don't know what this error can mean, especially as I don't think I've even hit the database at that point since I haven't closed the session by calling SaveChanges(). Any ideas? Here's the code itself.
XDocument doc = XDocument.Parse(rawXml);
var accItems = ExtractItemsFromFeed(doc);
using (IDocumentSession session = _store.OpenSession())
{
var dbItems = session.Query<AccItem>().ToList();
foreach (var item in accItems)
{
var existingRecord = dbItems.SingleOrDefault(x => x.Source == x.SourceId == cottage.SourceId);
if (existingRecord == null)
{
session.Store(item);
_logger.Info("Saved new item {0}.", item.ShortName);
}
else
{
existingRecord.ShortName = item.ShortName;
_logger.Info("Updated item {0}.", item.ShortName);
}
session.SaveChanges();
}
}
Any other comments about the style of this code would be most welcome, as I was unsure of the best way to approach the "update existing item or create if it isn't there" scenario.
The answer here was as follows.
RavenHQ support found that the database was indeed oversized, but it seemed that the size reported in the Appharbor-branded RavenHQ control panel was incorrect. I had filled up the database way over the limit with a previous faulty version of the code posted above, so the error message I received was actually correct.
Fixing this problem without paying to upgrade the database wasn't straightforward, as it's not possible to shrink the database. As I also wasn't able to delete my single Appharbor/RavenHQ database or create another one that left me with the choice of creating an entirely new Appharbor application, or registering directly with RavenHQ for a new account. I chose the latter. The RavenHQ-branded control panel is slightly different to the Appharbor one, in that it has the ability to create and delete databases.
So to summarize: there doesn't seem to be any benefit to using RavenHQ as an add-on to Appharbor - you might as well go and get a proper free RavenHQ account.