in line 357-358 of ContactServices.updatePostalAddress:
<set field="newPostalAddress" from="origPostalAddress.cloneValue()"/>
<entity-set value-field="newPostalAddress" include="nonpk"/>
In my case, address2 had a value in origPostalAddress and does not in the context, so it does not get changed in the entity-set because set-if-empty defaults to false and therefore it is deemed unchanged and no update takes place.
I guess this is a conscious decision not to have fields that are not passed (or empty) affect the update. Is there a work around?
There isn't really a good workaround for this. The update#PartyPostalAddress service in Mantle Business Artifacts is a generic service but with specific behavior, and in this case I think setting set-if-empty=true is fine. This is changed in commit #7e98f27.
Related
Is there a way to make the logged user (on superset) to make the queries on impala?
I tried to enable the "Impersonate the logged on user" option on Databases but with no success because all the queries run on impala with superset user.
I'm trying to achieve the same! This will not completely answer this question since it does not still work but I want to share my research in order to maybe help another soul that is trying to use this instrument outside very basic use cases.
I went deep in the code and I found out that impersonation is not implemented for Impala. So you cannot achieve this from the UI. I found out this PR https://github.com/apache/superset/pull/4699 that for whatever reason was never merged into the codebase and tried to copy&paste code in my Superset version (1.1.0) but it didn't work. Adding some logs I can see that the configuration with the impersonation is updated, but then the actual Impala query is with the user I used to start the process.
As you can imagine, I am a complete noob at this. However I found out that the impersonation thing happens when you create a cursor and there is a constructor parameter in which you can pass the impersonation configuration.
I managed to correctly (at least to my understanding) implement impersonation for the SQL lab part.
In the sql_lab.py class you have to add in the execute_sql_statements method the following lines
with closing(engine.raw_connection()) as conn:
# closing the connection closes the cursor as well
cursor = conn.cursor(**database.cursor_kwargs)
where cursor_kwargs is defined in db_engine_specs/impala.py as the following
#classmethod
def get_configuration_for_impersonation(cls, uri, impersonate_user, username):
logger.info(
'Passing Impala execution_options.cursor_configuration for impersonation')
return {'execution_options': {
'cursor_configuration': {'impala.doas.user': username}}}
#classmethod
def get_cursor_configuration_for_impersonation(cls, uri, impersonate_user,
username):
logger.debug('Passing Impala cursor configuration for impersonation')
return {'configuration': {'impala.doas.user': username}}
Finally, in models/core.py you have to add the following bit in the get_sqla_engine def
params = extra.get("engine_params", {}) # that was already there just for you to find out the line
self.cursor_kwargs = self.db_engine_spec.get_cursor_configuration_for_impersonation(
str(url), self.impersonate_user, effective_username) # this is the line I added
...
params.update(self.get_encrypted_extra()) # already there
#new stuff
configuration = {}
configuration.update(
self.db_engine_spec.get_configuration_for_impersonation(
str(url),
self.impersonate_user,
effective_username))
if configuration:
params.update(configuration)
As you can see I just shamelessy pasted the code from the PR. However this kind of works only for the SQL lab as I already said. For the dashboards there is an entirely different way of querying Impala that I did not still find out.
This means that queries for the dashboards are handled in a different way and there isn't something like this
with closing(engine.raw_connection()) as conn:
# closing the connection closes the cursor as well
cursor = conn.cursor(**database.cursor_kwargs)
My gut (and debugging) feeling is that you need to first understand the sqlalchemy part and extend a new ImpalaEngine class that uses a custom cursor with the impersonation conf. Or something like that, however it is not simple (if we want to call this simple) as the sql_lab part. So, the trick is to find out where the query is executed and create a cursor with the impersonation configuration. Easy, isnt'it ?
I hope that this could shed some light to you and the others that have this issue. Let me know if you did find out another way to solve this issue, or if this comment was useful.
Update: something really useful
A colleague of mine succesfully implemented impersonation with impala without touching any superset related, but instead working directly with the impyla lib. A PR was open with the code to change. You can apply the patch directly in the impyla src used by superset. You have to edit both dbapi.py and hiveserver2.py.
As a reminder: we are still testing this and we do not know if it works with different accounts using the same superset instance.
We've seen a resurrection of this issue in a recent update of Elasticsearch (https://github.com/elastic/elasticsearch-net/issues/1937).
We set the SourceSerializer when creating the Client connection but that doesn't seem to help.
Debugging in, I see that RequestResponseSerializer defaults to Nest.InternalSerializer. This JSON serializer has the DateParseHandling field set to DateTime when we want DateTimeOffset. I suspect that this may be the cause of my problem.
Is there a way to set RequestResponseSerializer to verify my theory?
ADDITION: I was able to verify my theory above by altering the NEST code directly. I edited the InternalSerializer::CreateSettings() method to include DateParseHandling = DateParseHandling.DateTimeOffset and that solved the issue.
Now how to set/modify this value for RequestResponseSerializer without modifying NEST code directly...
Turns out my issue was the same as https://github.com/elastic/elasticsearch-net/issues/3164 and seemed to be fixed in v6.2.0 (https://github.com/elastic/elasticsearch-net/pull/3278).
I was running v6.1.0
Upgraded my version to v6.3.1 and all looks well.
Problem
I would like to trace the Uri that will be generated by a LINQ query executed against a Microsoft.WindowsAzure.StorageClient.TableServiceContext object. TableServiceContext just extends System.Data.Services.Client.DataServiceContext with a couple of properties.
The issue I am having is that the query executes fine against our Azure Table Storage instance when we run the web role on a dev machine in debug mode (we are connecting to Azure storage in the cloud not using Dev Storage). I can get the resulting query Uri using Fiddler or just hovering over the statement in the debugger.
However, when we deploy the web role to Azure the query fails against the exact same Azure Table Storage source with a ResourceNotFound DataServiceClientException. We have had ResoureNotFound errors before that dealt with the behavior of FirstOrDefault() on empty tables. This is not the problem here.
As one approach to the problem, I wanted to compare the query Uri that is being generated when the web role is deployed versus when it is running on a dev machine.
Question
Does anyone know a way to get the query Uri for the query that will be sent when the FirstOrDefault() method is called. I know that you can call ToString() on the IQueryable returned from the TableServiceContext but my concern is that when FirstOrDefault() is called the Uri might be further optimized and ToString() on IQueryable might not be what is ultimately sent to the server when FirstOrDefault() is called.
If someone has another approach to the problem I am open to suggestions. It seems to be a general problem with LINQ when trying to determine what will happen when the expression tree is finally evaluated. I am open to suggestions here as well because my LINQ skills could use some improvement.
Sample Code
public void AddSomething(string ProjectID, string Username) {
TableServiceContext context = new TableServiceContext();
var qry = context.Somethings.Where(m => m.RowKey == Username
&& m.PartitionKey == ProjectID);
System.Diagnostics.Trace.TraceInformation(qry.ToString());
// ^ Here I would like to trace the Uri that will be generated
// and sent to the server when the qry.FirstOrDefault() call below is executed.
if (qry.FirstOrDefault() == null) {
// ^ This statement generates an error when the web role is running
// in the fabric
...
}
}
Edit Update and Answer
Steve provided the write answer. Our problem was as exactly described in this post which describes an issue with PartitionKey/RowKey ordering in Single Entity query which was fixed with an update to the Azure OS. This explains the discrepancy between our dev machines and when the web role was deployed to Azure.
When I indicated we had dealt with the ResourceNotFound issue before in our existence checks, we had dealt with it in two ways in our code. One way was using exception handling to deal with the ResourceNotFound error the other way was to put the RowKey first in the LINQ query (as some MS people had indicated was appropriate).
It turns out we have several places where the RowKey was first instead of using the exception handling. We will address this by refactoring our code to target .NET 4 and using the .IgnoreResourceNotFoundException = true property of theTableServiceContext .
Lesson learned (more than once): Don't depend on quirky undocumented behavior.
Aside
We were able to get the query Uri's. They did turn out to be different (as indicated they would be in the blog post). Here are the results:
Query Uri from Dev Fabric
`https://ourproject.table.core.windows.net/Somethings()?$filter=(RowKey eq 'test19#gmail.com') and (PartitionKey eq '41e0c1ae-e74d-458e-8a93-d2972d9ea53c')
Query Uri from Azure Fabric
`https://ourproject.table.core.windows.net/Somethings(RowKey='test19#gmail.com',PartitionKey='41e0c1ae-e74d-458e-8a93-d2972d9ea53c')
I can do one better... I think I know what the problem is. :)
See http://blogs.msdn.com/b/windowsazurestorage/archive/2010/07/26/how-wcf-data-service-changes-in-os-1-4-affects-windows-azure-table-clients.aspx.
Specifically, it used to be the case (in previous Guest OS builds) that if you wrote the query as you did (with the RowKey predicate before the PartitionKey predicate), it resulted in a filter query (while the reverse, PartitionKey preceding RowKey) resulted in the kind of query that raises an exception if the result set is empty.
I think the right fix for you (as indicated in the above blog post) is to set the IgnoreResourceNotFoundException to true on your context.
So I currently have 2 WSDLs added as Service References in my solution. They look like this in my app.config file (I removed the "bindings" field, because it's uninteresting):
<system.serviceModel>
<client>
<endpoint address="http://localhost:8080/query-service/jse" binding="basicHttpBinding" bindingConfiguration="QueryBinding" contract="QueryService.Query" name="QueryPort" />
<endpoint address="http://localhost:8080/dataimport-service/jse" binding="basicHttpBinding" bindingConfiguration="DataImportBinding" contract="DataService.DataImport" name="DataImportPort" />
</client>
</system.serviceModel>
When I utilize a WSDL, it looks something like this:
using (DataService.DataClient dClient = new DataService.DataClient())
{
DataService.importTask impt = new DataService.importTask();
impt.String_1 = "someData";
DataService.importResponse imptr = dClient.importTask(impt);
}
In the "using" statement, when instantiating the DataClient object, I have 5 constructors available to me. In this scenario, I use the default constructor:
new DataService.DataClient()
which uses the built-in Endpoint Address string, which I assume is pulled from app.config. But I want the user of the application to have the option to change this value.
1) What's the best/easiest way of programatically obtaining this string?
2) Then, once I've allowed the user to edit and test the value, where should I store it?
I'd prefer having it be stored in a place (like app.config or equivalent) so that there is no need for checking whether the value exists or not and whether I should be using an alternate constructor. (Looking to keep my code tight, ya know?)
Any ideas? Suggestions?
EDIT
Maybe I should ask about these Alternate constructors as well.
For example, one of them looks like this:
new DataService.DataClient(string endPointConfigurationName,
string remoteAddress)
What values could get passed for "endPointConfigurationName" and "remoteAddress"?
EDIT2
Answering my own questions here, the "endPointConfigurationName" appears to be the same as the "name" in the app.config XML and the "remoteAddress" is formatted the same as "endpoint address" in the app.config XML.
Also! The answer to my first question about getting the EndPointAddresses is the following:
ClientSection clSection =
ConfigurationManager.GetSection("system.serviceModel/client") as ClientSection;
ChannelEndpointElementCollection endpointCollection =
clSection.ElementInformation.Properties[string.Empty].Value as ChannelEndpointElementCollection;
Dictionary<string, string> nameAddressDictionary =
new Dictionary<string, string>();
foreach (ChannelEndpointElement endpointElement in endpointCollection)
{
nameAddressDictionary.Add(endpointElement.Name,
endpointElement.Address.ToString());
}
EDIT3
Ok, I think I've figured out the 2nd half (and thus, full solution) to my problem. I found this on another website and I modified it to meet my needs:
Configuration configuration;
ServiceModelSectionGroup serviceModelSectionGroup;
ClientSection clientSection;
configuration =
ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
serviceModelSectionGroup =
ServiceModelSectionGroup.GetSectionGroup(configuration);
clientSection = serviceModelSectionGroup.Client;
foreach (ChannelEndpointElement endPt in clientSection.Endpoints)
{
MessageBox.Show(endPt.Name + " = " + endPt.Address);
}
configuration.Save();
With this code, we have access to the clientSection.Endpoints and can access and change all the member properties, like "Address". And then when we're done changing them, we can do configuration.Save() and all the values get written out to a user file.
Now here's the catch. In debug mode, the "configuration.save()" does not appear to actually persist your values from execution to execution, but when running the application normal (outside of debug mode), the values persist. (Which is good.) So that's the only caveat.
EDIT4
There is another caveat. The changes made to the WSDLs do not take effect during runtime. The application needs to be restarted to re-read the user config file values into memory (apparently.)
The only other thing that I might be interested in is finding a way (once the values have been changed) to revert the values to their defaults. Sure, you can probably delete the user file, but that deletes all of the custom settings.
Any ideas?
EDIT5
I'm thinking Dependency Injection might be perfect here, but need to research it more...
EDIT 6
I don't have comment privileges but you need to run
ConfigurationManager.RefreshSection("client");
to have the cache updated so that changes happen immediately.
If you're using Microsoft Add Web Reference to create your service reference, then I think you may have trouble changing the connection programmatically. Even if you did change the auto generated code, as soon as you did an Update Service Reference it'd be overwritten.
You're best bet is to scratch Microsoft's auto generated code and build your own WCF classes. It's not difficult, and offers lots of flexibility / scalability.
Here's an excellent article on this very subject.
As for storing the custom addresses, it would depend on your app whether it's a Silverlight, Windows or web app. My personal choice is the database.
In my asp.net application, I open and close/flush the session at the beginning/ending of each request.
With this setup, I thought it would result in things like:
Entity e = EntityDao.GetById(1);
e.Property1 = "blah";
EntityDao.MakePersistant(e);
e = EntityDao.GetById(1);
e.Property1 // this won't be blah, it will be the old value since the request hasn't flushed
But I noticed that the value returned was the most recent updated value.
Someone responded that because of they way I have my identity setup?
Can someone explain this behaviour? So I don't need to call flush to ensure it is persisted to the db?
I belive (but could be mistaken) that nHibernate's caching and change-tracking is the reasons for this.
Since this Session instance is still active, nHibernate is tracking the changes you made to 'e'. When you ask for it again it includes those changes in the object returned. What happened when you called MakePersistant (which I assume calls Session.SaveOrUpdate(e)) is that it told that Session instance you are going to save those changes, thus it knows about them and will still show them when you call Session.Get(id) again.
If you instead started another Session and then called Session.Get(id) you wouldn't see those changes if you had not called Flush (or closed the Transaction, since you should be using a Transaction here) as it knows nothing of the changes made in the other Session.
To answer your other question, yes you still need to call Flush or close a Transaction to ensure changes are written to the database. One thing that is neat, is that you don't actually need to call SaveOrUpdate(e). For example this code will cause an Update to the database:
using (var session = SessionFactory.OpenSession())
using (var trans = session.BeginTransaction()){
var e = session.Get(id);
e.Name = "New Name";
trans.Commit();
}
nHibernate knows to update 'e' since it was tracking the changes that were made to during that Session. When I commit that transaction they are written. Note, this is the default behavior and I believe it can be changed if you want to require that .SaveOrUpdate() is called.