So i have an orchestration that successfully does everything i need it to. Now, i want to reuse the logic in the orchestration, but with a set of slightly different data sources. Rather than copy and paste the orchestration into another one and having to use a decision tree to choose which orchestration to call, I was thinking of making my calls to SQL a little more dynamic.
For example, lets say i have a stored procedure called spGetUSCust. I have coded the orchestration to call the SQL server via a send/receive port with an operation of GetCust on it. It was generated using a strongly typed method so the response message is of a type spGetUSCustResponse.
I now want to call spGetCACust on the same SQL server. The responding data is in the exact same format (structure) as the US stored proc, but they have different names.
So my question is, can i do this by setting the action on the message that will be going to the port within the code? Since my response is strongly typed, will it cause a problem that the response will be really coming from the CA procedure and not the US one? If so, how do i solve that? I could go with generic responses, but they return XML ANY fields and i need to map these responses for additional use in the orchestration.
In an Message Assignment Shape, you can set the WCF.Action property on the request message like this:
MySQLRequest(WCF.Action)="TypedProcedure/dbo/spGetUSCust";
You can use the same physical Port and will have to remove any content in the Action Mapping section.
However, because the Request Messages are different types, you will need two Orchestration Ports, one US, one CA.
Related
I am currently working on a REST API for a project. In the process I should search for events. I would like to make an endpoint for searching events in a period. That is, specify two parameters with from - to.
For the search you normally take a GET operation. My question is now it makes sense to specify two parameters in the path or should I rather fall back to a POST operation for something like that.
Example for the path /Events{From}{To}
Is this even feasible with multiple parameters?
If you are not making a change on the resource, you should use GET operation.
More detailed explanation:
If you were writing a plain old RPC API call, they could technically interchangeable as long as the processing server side were no different between both calls. However, in order for the call to be RESTful, calling the endpoint via the GET method should have a distinct functionality (which is to get resource(s)) from the POST method (which is to create new resources).
GET request with multiple parameters: /events?param1=value1¶m2=value2
GET request with an array as parameter: /events?param=value1,value2,value3
The DBCPConnectionPool Service requires 5 connection parameters to establish connection to a database as shown in the picture below [Marked Yellow]
I used UpdateAttribute Processor to manually add these 5 connection parameters and gave them their respective values as shown in the picture below [Marked Yellow]
Now, when I was trying to read the values for the connection parameters in DBCPConnectionPool Service through these attributes (Shown in picture below) , I was unable to read them.
To know the reason why the DBCPConnectionPool Service was unable to read the Flowfile attributes, I went ahead to check the source code for both DBCPConnectionPool Service and UpdateAttribute Processor.
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/src/main/java/org/apache/nifi/dbcp/DBCPConnectionPool.java
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-processor/src/main/java/org/apache/nifi/processors/attributes/UpdateAttribute.java
Souce code for DBCPConnectionPool Service :
Souce code for UpdateAttribute Processor :
Thus, I came to know the reason why it was unable to read the values from FlowFile attributes. This is because the ExpressionLanguageScope is limited to VARIABLE_REGISTRY and not FLOWFILE_ATTRIBUTES.
Now, My Question is that why the ExpressionLanguageScope for DBCPConnectionPool Service is limited to VARIABLE_REGISTRY. What is the reason for this limitation? The reason why I am asking this question is because I want to read the values for the connection parameters through FlowFile attributes.
For the same question that has been asked in the NiFi dev mailing list, Andy had answered it in the best way possible. The reason why DBCPConnectionPool services or any controller services for that matter, uses ExpressionLanguageScope.VARIABLE_REGISTRY is that, the controller services have no access to the flowfiles so it won't read the flowfiles' attributes. And for the question, why it only supports VARIABLE_REGISTRY is:
Just because it doesn't read flowfile attributes doesn't mean that it should not use attributes from elsewhere.
One of the major reasons why VARIABLE_REGISTRY was introduced was to avoid exposing the sensitive values which is the case when we pass around such values as flowfile attributes. The controller services fit this case, because many of them use sensitive properties like Password.
And if you're assuming that you can make it work just changing the the scope for those properties to ExpressionLanguageScope.FLOWFILE_ATTRIBUTES, you're wrong. Changing them makes no sense and doesn't work, the reason is again the controller services never get to access the flowfiles.
If there is a specific requirement for you wherein you need to use different property values for different flowfiles, Andy in the original dev thread had shared some links which I'm posting again:
https://stackoverflow.com/a/49412970/70465
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Using_Custom_Properties
I came to think about this question a few days ago when I desinged an HTML form that submits data via php to an SQL database. I solved my problem, but I am asking here a computer-theoretical question, which might help me (or others) in the future.
I want to protect myself from SQL-injection, and I thought that instead of validating the data by the php on the server side, I can have the javascript validate the data on the client side (I am much more fluent in JS than in PHP) and then send it.
However, a sophisticated user might inspect the javascript (or the HTTPrequest) and then alter it to send his own vicious request to the server.
My question:
Is it theoretically possible to do a computation on the clinet side, where the code is visible to him, and have it sent with some way that ensures that the data was sent from my program and not from an altered code?
Can this be done by an RSA-scheme with public and private keys?
I want to protect myself from SQL-injection, and I thought that instead of validating the data
Don't validate data to protect yourself from SQL Injection. Validate data to make sure it is in the format you want.
Escape data to protect yourself from SQL Injection (and do that escaping via prepared statements and parameterized queries).
Is it theoretically possible to do a computation on the clinet side, where the code is visible to him, and have it sent with some way that ensures that the data was sent from my program and not from an altered code?
No. The client side code can be bypassed entirely. In this arena, it is useful only to quickly tell the user that their data would be rejected if it was submitted to the server.
Can this be done by an RSA-scheme with public and private keys?
No. You have to give one of the keys to the client. It can then be extracted and used independently of your code.
Okay, so my goal is to build a easy to use protocol for sending data over tcp. basically, it would send a message, and an object(of unknown type) over tcp. To send, it would only require one method call and to receive it would only require one also.
So this is how I was thinking to format the "message".
length_of_message - "A string that is a message" - length_of_Object - object
length_of_message would be a set number of bytes. along with length_of_Object.
the actual message string and the actual object would be of variable length.
If the actual class of the object wouldn't be know, could I just declare it as a "generic object" somehow? and then get its class name from the "generic object" and the message would tell the receiver what to do with the object?
It would be simple if it was a constant object type but i want to be able use one send function and one receive function for ever object that needs to be send/recieved.
Any suggestions?
Thanks,
Andrew
Make sure you aren't reinventing the wheel (unless doing so is your primary goal).
With that in mind, consider:
• Implement and use the NSCoding protocol. It allows for the efficient archival of complexly connected object graphs, including cycles.
• Instead of raw TCP, use HTTP. While it adds a bit over overhead in the headers, the body can be straight encoded data. More importantly, HTTP is ubiquitous. It routes through just about anything whereas other protocols might be blocked (think proxy servers).
• Via HTTP, you can leverage compression. If one side of your communication pipe is an existing web server of some kind, it probably already supports gzip'd communication. Compressing an NSData (that would be the result of NSCoding) is trivial.
• Alternatively, stick with straight plists.
Unless you truly have some requirement that makes the above inviable, you are likely better off leveraging the above technologies instead of rolling a new one.
With that said, what you propose is fine. I would add, possibly, a structure like:
[HEADER][MSGID][LEN][TYPE][DATA of len][POST]
Where the POST is a known sequence of bytes that the receiver can verify to make sure that, maybe, all the data was received correctly. Or you could go whole hog and integrate a checksum. Or sub-pieces could be repeated, as needed (i.e. [LEN][TYPE][DATA] over and over.
I'm trying to send a message to the WCF-WSHttp adapter with a dynamic
send port from an orchestration, but BizTalk seems to always be
reverting back to the HTTP Adapter.
According to the docs that I've been able to find, I should just need
to set the transport type from my expression shape to get BizTalk to
use the WCF-WSHttp adapter, and I AM, but it still seems to be
reverting. Below is an example of my expression shape that's setting
the properties (as you can see, I've tried both
Microsoft.XLANGs.BaseTypes.TransportType and
BTS.OutboundTransportType):
Body(BTS.OutboundTransportType) = "WCF-WSHttp";
SendMessagePort(Microsoft.XLANGs.BaseTypes.Address) =
System.String.Format("{0}/Accept{1}", "http://myserver/myservice/
myservice.svc/Accept{0}", messageInfo.MessageType);
SendMessagePort(Microsoft.XLANGs.BaseTypes.TransportType) = "WCF-
WSHttp";
Probably are Craig :-)
When using a dynamic send port, BizTalk uses the "scheme" part of the url to decide which adapter to use.
When your url starts with "Http://" or "Https://" BizTalk would always use the HTTP adapter.
Similarly url's begining with ftp:// will use the FTP adapter.
Same works for custom adapaters as well - when you install the adapter's configuration you register the moniker to use; for example - the open source Scheduled Task adapter uses schedule:// (I believe).
Using dynamic send ports with WCF is slightly more involved than most other adapaters because of the various configuration that's required but you can find detailed explanation here, just scroll down to the "Dynamic Send Ports" section about half way down.
I ended up resolving my issue, but am still unsure of the reasoning for the behavior I saw.
The Expression shape mentioned in the question was located inside of an Atomic Scope. Once the Orchestration exited the scope containing the Expression shape, the Transport Type was reset back to its original value. Moving the Expression out of the atomic scope resolved the issue, in that the TransportType was set correctly.