This is happening intermittently (usually at start up). I get the above error message when executing the following code.
var arr = ORMExecuteQuery( "FROM priority WHERE active = 1 ORDER BY sortOrder" );
var qry = entityToQuery( arr );
The first line executes fine, but the second line blows up. The solution is to run ormreload();
The problem keeps coming up in an unpredictable way though. Even when no changes have been made to the beans or gateways that are using ORM. Completely unpredictable and impossible to replicate on purpose. Is there something else that can mess with the hibernate mappings that could cause this type of problem.
Other info that may be pertinent:
This is a MURA plugin based on a recent version of FW/1.
ormreload() is a persistent fix (until it fails again)
My current solution is to put ormreload() in the setupApplication() method of application.cfc
I just want to understand better what could be causing this problem.
Related
I am attempting to use clairvoyant's db-cleanup dag to clear metadata in our xcom table, but when I run it, I receive the following warning, printed thousands of times before I manually stop the job in order to not take down our mysql instance:
SAWarning: Loading context for <BaseXCom at 0x7f26f789b370> has changed within a load/refresh handler, suggesting a row refresh operation took place. If this event handler is expected to be emitting row refresh operations within an existing load or refresh operation, set restore_load_context=True when establishing the listener to ensure the context remains unchanged when the event handler completes.
The other cleanup tasks work fine, but it is the xcom table in particular I am having trouble with. We have hundreds/thousands of active dags and so the xcom table is constantly being written to nearly every second or two. I think that is what is causing this error, the fact that the data is continually changing while it is being queried.
I have been unable to find the cause of this or any examples of how this can be resolved. I tried adding a "restore_load_context":True line as per SQLAlchemy docs but it did not work.
Here are the snippets I attempted to add to the database object and the cleanup task:
{
"airflow_db_model": XCom,
"age_check_column": XCom.execution_date,
"keep_last": False,
"keep_last_filters": None,
"keep_last_group_by": None,
"restore_load_context":True
},
....
def cleanup_function(**context):
logging.info("Retrieving max_execution_date from XCom")
max_date = context["ti"].xcom_pull(
task_ids=print_configuration.task_id, key="max_date"
)
max_date = dateutil.parser.parse(max_date) # stored as iso8601 str in xcom
airflow_db_model = context["params"].get("airflow_db_model")
state = context["params"].get("state")
age_check_column = context["params"].get("age_check_column")
keep_last = context["params"].get("keep_last")
keep_last_filters = context["params"].get("keep_last_filters")
keep_last_group_by = context["params"].get("keep_last_group_by")
restore_load_context = context["params"].get("restore_load_context")
In order to not paste too much code here, I am using the same code in the db-cleanup dag. Has anyone encountered this and found a way to resolve?
I am very inexperienced with sqlalchemy and am entirely unsure where else to place this code or how to go about it.
I am trying to run the example "optaplanner-mixedvrp-experiment" developed by Geoffrey De Smet and when I run it it throws me the following error:
Caused by: java.lang.IllegalStateException: The entity (MY) has a
variable (previousStandstill) with value (MUNO) which has a
sourceVariableName variable (nextVisit) with a value (WERBOMONT) which
is not null. Verify the consistency of your input problem for that
sourceVariableName variable.
I have not made any change, I have only cloned and executed it, I import and solve it and it throws me this error.
Do you know what could be happening?
I am applying it in the development of a variant of VRP with multiple deliveries and collections, but it throws me the same error. I have activated the FULL_ASSERT mode and nextVisit, previousStandstill, visitIndex are always null
It's been a long time since I looked at that code, so it's using an old version of optaplanner. Our goal is still to clean it up and offer an out of the box example for VRPPD (and probably remove some boilerplate along the way, using the upcoming #CollectionPlanningVariabe etc). That being said, we have multiple users&customers who used that optaplanner-mixedvrp-experiment to successfully build VRPPD implementations.
Which dataset did you try?
FWIW, that IllegalStateException says that when A.previous = B, the B.next is not A. So either the dataset importer didn't import it correctly - before calling solve() - especially if it fails before the first CH step in FULL_ASSERT. Or one of the custom moves corrupted the model.
I´ve got a Job that runs a TaskletStep, then a chunk-based step and then another TaskletStep.
In each of these steps, errors (in the form of Exceptions) can occur.
The chunk-based step looks like this:
stepBuilderFactory
.get("step2")
.chunk<SomeItem, SomeItem>(1)
.reader(flatFileItemReader)
.processor(itemProcessor)
.writer {}
.faultTolerant()
.skipPolicy { _ , _ -> true } // skip all Exceptions and continue
.taskExecutor(taskExecutor)
.throttleLimit(taskExecutor.corePoolSize)
.build()
The whole job definition:
jobBuilderFactory.get("job1")
.validator(validator())
.preventRestart()
.start(taskletStep1)
.next(step2)
.next(taskletStep2)
.build()
I expected that Spring Batch somehow picks up the Exceptions that occur along the way, so I can then create a Report including them after the Job has finished processing. Looking at the different contexts, there´s also fields that should contain failureExceptions. However, it seems there´s no such information (especially for the chunked step).
What would be a good approach if I need information about:
what Exceptions did occur in which Job execution
which Item was the one that triggered it
The JobExecution provides a method to get all failure exceptions that happened during the job. You can use that in a JobExecutionListener#afterJob(JobExecution jobExecution) to generate your report.
In regards to which items caused the issue, this will depend on where the exception happens (during the read, process or write operation). For this requirement, you can use one of the ItemReadListener, ItemProcessListener or ItemWriteListener to keep record of the those items (For example, by adding them to the job execution context to be able to get access to them in the JobExecutionListener#afterJob method for your report).
Trying to use the DoubleClick API with the Google Client PHP SDK described here https://developers.google.com/doubleclick-advertisers/v3.1/conversions/batchupdate
I'm finding that instead of an update happening, these conversions are being removed from our reports. I suspect something with De-duplication, but I wouldn't expect that the conversion be completely removed.
I'm basically doing this:
$conversion = new Google_Service_Dfareporting_Conversion();
$conversion->setKind("dfareporting#conversion");
$conversion->setGclid($clickId);
$conversion->setFloodlightActivityId($this->getActivityId());
$conversion->setFloodlightConfigurationId($this->getConfigId());
$conversion->setOrdinal($conversionId);
$conversion->setTimestampMicros($conversionTimestamp);
$conversion->setQuantity(1);
$conversion->setValue($revenue);
$doubleClickSearch = new Google_Service_Dfareporting($this->getGoogleApiClient());
$conversionList = new Google_Service_Dfareporting_ConversionsBatchUpdateRequest();
$conversionList->setKind("dfareporting#conversionsBatchUpdateRequest");
$conversionList->setConversions([$conversion]);
return $doubleClickSearch->conversions->batchupdate($this->getUserProfileId(), $conversionList);
And when this gets run, it works as expected:
{"hasFailures":false,
"kind":"dfareporting#conversionsBatchUpdateResponse",
"status":[
{"kind":"dfareporting#conversionStatus",
"conversion":{
"childDirectedTreatment":null,
"encryptedUserId":null,
"encryptedUserIdCandidates":null,
"floodlightActivityId":"redacted",
"floodlightConfigurationId":"redacted",
"gclid":"redacted",
"kind":"dfareporting#conversion",
"limitAdTracking":null,
"mobileDeviceId":null,
"ordinal":"redactedConversionId",
"quantity":"1",
"timestampMicros":"1532697843000000",
"value":885
}
}
]
}
But then when we run the report afterwards, this particular conversion is no longer part of it: It doesn't just not take the update, the conversion has been completely removed.
I've verified that the Ordinal, timestamp, gclid, activityId, and configuration IDs all match. Any idea where my conversions are going?
I will note that we are running the batchUpdate command for creations as well. I'm not sure if that would have anything to do with it, but the records do get created as normal.
I am experiencing a glitch from OpenDaylight (using Mininet).
Essentially, I am querying flow rules on specific nodes and on specific tables. The relevant code is the following, and is run by 1 separate thread per node that I am polling:
public static final InstanceIdentifier<Nodes NODES_II = InstanceIdentifier
.builder(Nodes.class).build();
public static InstanceIdentifier<Table> makeTableIId(NodeId nodeId, Short tableId) {
return NODES_IID.child(Node.class, new NodeKey(nodeId))
.augmentation(FlowCapableNode.class)
.child(Table.class, new TableKey(tableId));
}
and
InstanceIdentifier<Table> tableIId = makeTableIId(nodeId, tableId);
Optional<Table> tableOptional = dataBroker.newReadOnlyTransaction()
.read(LogicalDatastoreType.OPERATIONAL, tableIId).get();
if(!tableOptional.isPresent()) {
continue;
}
List<Flow> flows = tableOptional.get().getFlow();
The behavior: tableOptional is present, and getFlow() returns an empty list.
The observation: there ARE flow rules installed on ALL nodes on the tables I am querying, but for some reason, some of these nodes show none of these flows on none of the tables (here, tables 3, 4, 5, and 6).
The weirdness: On one of the problematic nodes, I have four rules, installed on tables 9, 13, 17 and 22 respectively. They timeout simultaneously after 150 seconds. After they disappear, the query suddenly begins to "see" the flows installed on tables 3, 4, 5, and 6, returning these for each table.
Question: How is this even possible?
EDIT I just realized that the rules whose timeout "suddenly fix everything" were also rules that generated warnings in ODL's log (OpenFlowPlugin to be more specific). I did not observe any obvious issue, so I'd sort of brushed it aside.
Here is the code relevant to the error:
https://pastebin.com/yJDZesXU
Here are the errors I get every time I install a rule that walks through these lines:
https://pastebin.com/c9HYLBt6
I must stress that these rules work as intended, and that printing them out reveals no evident formatting issue. Again, they appear fine when dumped.
My hypothesis is that this warning is a symptom of ODL "messing up" trying to store the rules in MD-SAL, which ends up messing a lot of rule-reading queries. On uninstallation of the garbage that ensues, rule-reading queries become functional again.
This makes sense to me, but then... I haven't understood how to fix these warnings, or what these warnings were about in the first place.
EDIT 2: By commenting lines suspecting of causing the warnings in the above pastebin:
//ipv4MatchBuilder.setIpv4SourceAddressNoMask(...);
//ipv4MatchBuilder.setIpv4SourcenArbitraryBitmask(...);
The warnings disappear, AND the flows appear correctly on all tables, when pinged. This confirms my hypothesis that somewhere, something wrong happens in the data store.
EDIT 3: I have found that by setting any non-trivial arbitrary bitmask, this error goes away. That is, I have tried setting an arbitrary bitmask which was neither null nor "255.255.255.255", and this error has gone away. The problem is I might like having a bitmask for the source, but an exact match on the destination. Even setting the bitmask to "127.255.255.255" (as I tried) is still unnerving. It really feels to me like this is an OpenFlowPlugin glitch, though.
EDIT 4: Steps for reproducing the bug
Install a rule with ipv4 arbitrary bitmask match, with the destination ip set, and the destination arbitrary bitmask either null or set to 255.255.255.255.
Ipv4MatchArbitraryBitMaskBuilder ipv4MatchBuilder = new Ipv4MatchArbitraryBitMaskBuilder();
ipv4MatchBuilder.setIpv4DestinationAddressNoMask(new Ipv4Address("10.0.0.1"));
ipv4MatchBuilder.setIpv4DestinationArbitraryBitmask(new DottedQuad("255.255.255.255"));
matchBuilder = new MatchBuilder().setEthernetMatch(ethernetMatchBuilder.build()).setLayer3Match(ipv4MatchBuilder.build());
... and so on ...
Extra optional steps: Install one such rule for the destination, one such rule for the source, and install equivalent rules where the bitmask is set to something else, like 127.255.255.255.
Make a query to MDSal to fetch flow information from the node on which you installed the flow rule.
Now, do "log:display" inside your ODL controller. You should have a warning about a malformed destination address. Additionally, the Table object you queried should contain no flows, so tableObject.getFlow() should return an empty list.