In which place is best practice to validate parameters sent by the user in an API design? By parameter validation I refer to: checking required params are sent, ensure they have correct format and so... Here are a couple of simple examples that validate an id has been sent. It is Python using Flask to illustrate:
A) Add validation logic in the route definition, within the controller.
#api.route('/job', methods=['GET'])
def get_jobs():
try:
if params["id"] is None:
raise Exception("Invalid param ID parameters.")
job = job_manager.get_job(params["id"])
return jsonify(job)
B) In the core of the app. This is the business layer, where logic is applied to transform data.
class JobManager:
def get_job(self, job_id) -> None:
if job_id is None:
raise Exception("Invalid param ID parameters.")
In more complex scenarios a validator service or decorators could be used, but the question would be the same: At which point of the code is best practice to validate a user's input.
If the answer is none of the scenarios above (or both), please provide more details on your answer. If possible, try to be language agnostic as I'm looking for a best practice that can be applied anywhere.
Parsing, as a rule, should happen at the point where information enters your system, or as close to that point as is practical.
Therefore certainly "application layer" rather than "domain layer/business layer": either invoked by the controller itself, or very close to it. (Not typically "in" the controller, because you should be able to test the parser without being coupled to a bunch of HTTP ceremony.)
#api.route('/job', methods=['GET'])
def get_jobs():
try:
job_id = parse_job_id(params["id"])
job = job_manager.get_job(job_id)
return jsonify(job)
In typed languages, this can make your life a lot easier, because you greatly reduce the number of places you have to ask "does this general purpose data structure have the information I expect?"
Checks against business policy, on the other hand, normally belong in the domain layer.
For example: if your API requires a date, the checks that the date is actually present, and that the date is represented in the appropriate ISO-8601 format, and so on... these kinds of checks all happen as part of the parsing of the input by the controller.
On the other hand, checking that the date is "in the future", or that the date is within warranty, or ... these are checks that belong in your domain code.
Generally, I split validation into phases:
Immediate syntax validation of input data in the REST controller
Business-logic validation in the services
The first validation should only flag things that are definitely wrong; e.g. missing required fields, type mismatch, unparsable strings, any attempts of code injection and the presence (or lack of) of security tokens.
When this validation passes, the input data is at least syntactically correct and may be passed on to the services, where a more strict validation occurs; i.e. does the input data make sense business-wise, does the resource with that ID exist - and so on.
Short version: The first validation looks for things that are obviously wrong, while the following validation ensures input data is correct and meaningful business-wise.
Related
I consider myself that 0 is not a good thing to do when returning information from an API
e.g.
{
userId: int|null
}
I have a colleague that insists in that userId should be 0 or -1, but that forces a system to know that the 0 means "not set", instead of null which is universally known as not set.
The same happens with string params, like logoUrl. However, in this case I think it is acceptable to have an empty string instead of null if the variable is not set or was unset.
Is there bibliography, standards, etc, that I can refer to?
I'm not aware of any standard around that, but the way I take those kind of decisions is by thinking about the way consumer services would read this response, the goal being to provide a very smooth and clean consuming workflow. This exercise can even be transformed into a documentation for your API consumers.
In your case, instead of returning a null/0 field, I would simply remove that field altogether when it's empty and let the consumers explicitly mark that field as optional in the model they use to deserialize this response.
In that way, they'll explicitly have to deal with those optional fields, than relying on the API to always provide a value for them.
Could someone help me to understand the difference between:
Mono.defer()
Mono.create()
Mono.just()
How to use it properly?
Mono.just(value) is the most primitive - once you have a value you can wrap it into a Mono and subscribers down the line will get it.
Mono.defer(monoSupplier) lets you provide the whole expression that supplies the resulting Mono instance. The evaluation of this expression is deferred until somebody subscribes. Inside of this expression you can additionally use control structures like Mono.error(throwable) to signal an error condition (you cannot do this with Mono.just).
Mono.create(monoSinkConsumer) is the most advanced method that gives you the full control over the emitted values. Instead of the need to return Mono instance from the callback (as in Mono.defer), you get control over the MonoSink<T> that lets you emit values through MonoSink.success(), MonoSink.success(value), MonoSink.error(throwable) methods.
Reactor documentation contains a few good examples of possible Mono.create use cases: link to doc.
The general advice is to use the least powerful abstraction to do the job: Mono.just -> Mono.defer -> Mono.create.
Although in general I agree with (and praise) #IlyaZinkovich's answer, I would be careful with the advice
The general advice is to use the least powerful abstraction to do the job: Mono.just -> Mono.defer -> Mono.create.
In the reactive approach, especially if we are beginners, it's very easy to overlook which the "least powerful abstraction" actually is. I am not saying anything else than #IlyaZinkovich, just depicting one detailed aspect.
Here is one specific use case where the more powerful abstraction Mono.defer() is preferable over Mono.just() but which might not be visible at the first glance.
See also:
https://stackoverflow.com/a/54412779/2886891
https://stackoverflow.com/a/57877616/2886891
We use switchIfEmpty() as a subscription-time branching:
// First ask provider1
provider1.provide1(someData)
// If provider1 did not provide the result, ask the fallback provider provider2
.switchIfEmpty(provider2.provide2(someData))
public Mono<MyResponse> provide2(MyRequest someData) {
// The Mono assembly is needed only in some corner cases
// but in fact it is always happening
return Mono.just(someData)
// expensive data processing which might even fail in the assemble time
.map(...)
.map(...)
...
}
provider2.provide2() accepts someData only when provider1.provide1() does not return any result, and/or the method assembly of the Mono returned by provider2.provide2() is expensive and even fails when called on wrong data.
It this case defer() is preferable, even if it might not be obvious at the first glance:
provider1.provide1(someData)
// ONLY IF provider1 did not provide the result, assemble another Mono with provider2.provide()
.switchIfEmpty(Mono.defer(() -> provider2.provide2(someData)))
I was doing some reading here(1,2) and I was wondering if making sure the parameters provided during initialization are correct, violates the guideline that constructors shouldn't do work.
For example (Python):
class Employee:
def __init__(self, empFirstname, empLastname, empEmail):
self._validate_employee(empFirstname, "First Name")
self._validate_employee(empLastname, "Last name")
self._validate_employee(empEmail, "Email")
self._validate_email(empEmail, "Email")
self.empFirstname = empFirstname
self.empLastname = empLastname
self.empEmail = empEmail
#property
def email(self):
return self.empEmail
def _validate_employee(self, parameter, error_message):
if not parameter:
raise TypeError("{0} {1}" .format(error_message, "is missing"))
def _validate_email(self, email, parameter):
if "#" not in email or "." not in email:
raise TypeError("{0} {1}" .format(parameter, " is invalid"))
In my example, I check to make sure the first and last names aren't blank and that the email is valid. Did I violate the guideline?
Update:
I'm not asking if it should throw, I'm asking if I'm violating the guideline that constructors shouldn't do work when it validates my parameters.
The requirement to provide arguments that satisfy certain conditions is a part of Design by Contract, namely it corresponds to preconditions. The client (of a constructor in your case) should guarantee that the arguments are as expected. If this is not the case, the supplier (the constructor in your case) cannot ensure the resulting object is in a valid state after the constructor returns. In languages that support Design by Contract natively, one would associate a class invariant stating that the names are not empty and the email address follows the naming convention. This could be achieved only when the arguments passed to the constructor are valid.
The conditions you mention would be written as preconditions and could be turned on or off depending on the policy established in the software development process. When enabled, they are checked on entry to the constructor, before its real body is executed. From this point of view, your code is not doing any additional work, but makes sure the arguments are correct. After program verification that ensures the preconditions are satisfied all the time, or extensive testing that gives the feeling that the preconditions are not violated by the client, they can be disabled, thus removing any "additional work".
To summarize:
the "additional code" in your examples are preconditions
preconditions are part of software contracts
the checks of the contracts can be enabled or disabled depending on the adopted policy, it is safe to disable them as soon as all clients guarantee the preconditions are never violated, thus removing any "additional work" from the constructor (or any other software component)
No, validating parameters and inputs is vital for all functions, including constructors. Blindly setting parameters without checking/error handling can lead to serious issues ranging from unexpected performance to the ability for malicious actors to manipulate your program in ways that it was not intended.
Are idempotent and deterministic functions both just functions that return the same result given the same inputs?
Or is there a distinction that I'm missing?
(And if there is a distinction, could you please help me understand what it is)
In more simple terms:
Pure deterministic function: The output is based entirely, and only, on the input values and nothing else: there is no other (hidden) input or state that it relies on to generate its output. There are no side-effects or other output.
Impure deterministic function: As with a deterministic function that is a pure function: the output is based entirely, and only, on the input values and nothing else: there is no other (hidden) input or state that it relies on to generate its output - however there is other output (side-effects).
Idempotency: The practical definition is that you can safely call the same function multiple times without fear of negative side-effects. More formally: there are no changes of state between subsequent identical calls.
Idempotency does not imply determinacy (as a function can alter state on the first call while being idempotent on subsequent calls), but all pure deterministic functions are inherently idempotent (as there is no internal state to persist between calls). Impure deterministic functions are not necessarily idempotent.
Pure deterministic
Impure deterministic
Pure Nondeterministic
Impure Nondeterministic
Idempotent
Input
Only parameter arguments (incl. this)
Only parameter arguments (incl. this)
Parameter arguments and hidden state
Parameter arguments and hidden state
Any
Output
Only return value
Return value or side-effects
Only return value
Return value or side-effects
Any
Side-effects
None
Yes
None
Yes
After 1st call: Maybe.After 2nd call: None
SQL Example
UCASE
CREATE TABLE
GETDATE
DROP TABLE
C# Example
String.IndexOf
DateTime.Now
Directory.Create(String)Footnote1
Footnote1 - Directory.Create(String) is idempotent because if the directory already exists it doesn't raise an error, instead it returns a new DirectoryInfo instance pointing to the specified extant filesystem directory (instead of creating the filesystem directory first and then returning a new DirectoryInfo instance pointing to it) - this is just like how Win32's CreateFile can be used to open an existing file.
A temporary note on non-scalar parameters, this, and mutating input arguments:
(I'm currently unsure how instance methods in OOP languages (with their hidden this parameter) can be categorized as pure/impure or deterministic or not - especially when it comes to mutating the the target of this - so I've asked the experts in CS.SE to help me come to an answer - once I've got a satisfactory answer there I'll update this answer).
A note on Exceptions
Many (most?) programming languages today treat thrown exceptions as either a separate "kind" of return (i.e. "return to nearest catch") or as an explicit side-effect (often due to how that language's runtime works). However, as far as this answer is concerned, a given function's ability to throw an exception does not alter its pure/impure/deterministic/non-deterministic label - ditto idempotency (in fact: throwing is often how idempotency is implemented in the first place e.g. a function can avoid causing any side-effects simply by throwing right-before it makes those state changes - but alternatively it could simply return too.).
So, for our CS-theoretical purposes, if a given function can throw an exception then you can consider the exception as simply part of that function's output. What does matter is if the exception is thrown deterministically or not, and if (e.g. List<T>.get(int index) deterministically throws if index < 0).
Note that things are very different for functions that catch exceptions, however.
Determinacy of Pure Functions
For example, in SQL UCASE(val), or in C#/.NET String.IndexOf are both deterministic because the output depends only on the input. Note that in instance methods (such as IndexOf) the instance object (i.e. the hidden this parameter) counts as input, even though it's "hidden":
"foo".IndexOf("o") == 1 // first cal
"foo".IndexOf("o") == 1 // second call
// the third call will also be == 1
Whereas in SQL NOW() or in C#/.NET DateTime.UtcNow is not deterministic because the output changes even though the input remains the same (note that property getters in .NET are equivalent to a method that accepts no parameters besides the implicit this parameter):
DateTime.UtcNow == 2016-10-27 18:10:01 // first call
DateTime.UtcNow == 2016-10-27 18:10:02 // second call
Idempotency
A good example in .NET is the Dispose() method: See Should IDisposable.Dispose() implementations be idempotent?
a Dispose method should be callable multiple times without throwing an exception.
So if a parent component X makes an initial call to foo.Dispose() then it will invoke the disposal operation and X can now consider foo to be disposed. Execution/control then passes to another component Y which also then tries to dispose of foo, after Y calls foo.Dispose() it too can expect foo to be disposed (which it is), even though X already disposed it. This means Y does not need to check to see if foo is already disposed, saving the developer time - and also eliminating bugs where calling Dispose a second time might throw an exception, for example.
Another (general) example is in REST: the RFC for HTTP1.1 states that GET, HEAD, PUT, and DELETE are idempotent, but POST is not ( https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html )
Methods can also have the property of "idempotence" in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property. Also, the methods OPTIONS and TRACE SHOULD NOT have side effects, and so are inherently idempotent.
So if you use DELETE then:
Client->Server: DELETE /foo/bar
// `foo/bar` is now deleted
Server->Client: 200 OK
Client->Server DELETE /foo/bar
// foo/bar` is already deleted, so there's nothing to do, but inform the client that foo/bar doesn't exist
Server->Client: 404 Not Found
// the client asks again:
Client->Server: DELETE /foo/bar
// foo/bar` is already deleted, so there's nothing to do, but inform the client that foo/bar doesn't exist
Server->Client: 404 Not Found
So you see in the above example that DELETE is idempotent in that the state of the server did not change between the last two DELETE requests, but it is not deterministic because the server returned 200 for the first request but 404 for the second request.
A deterministic function is just a function in the mathematical sense. Given the same input, you always get the same output. On the other hand, an idempotent function is a function which satisfies the identity
f(f(x)) = f(x)
As a simple example. If UCase() is a function that converts a string to an upper case string, then clearly UCase(Ucase(s)) = UCase(s).
Idempotent functions are a subset of all functions.
A deterministic function will return the same result for the same inputs, regardless of how many times you call it.
An idempotent function may NOT return the same result (it will return the result in the same form but the value could be different, see http example below). It only guarantees that it will have no side effects. In other words it will not change anything.
For example, the GET verb is meant to be idempotent in HTTP protocol. If you call "~/employees/1" it will return the info for employee with ID of 1 in a specific format. It should never change anything but simply return the employee information. If you call it 10, 100 or so times, the returned format will always be the same. However, by no means can it be deterministic. Maybe if you call it the second time, the employee info has changed or perhaps the employee no longer even exists. But never should it have side effects or return the result in a different format.
My Opinion
Idempotent is a weird word but knowing the origin can be very helpful, idem meaning same and potent meaning power. In other words it means having the same power which clearly doesn't mean no side effects so not sure where that comes from. A classic example of There are only two hard things in computer science, cache invalidation and naming things. Why couldn't they just use read-only? Oh wait, they wanted to sound extra smart, perhaps? Perhaps like cyclomatic complexity?
How do I call the SAP report (for example RSPARAM) with help JCo?
What RFC may be used to remotely call SA38 transaction with RSPARAM (e.t.c.) as parameter and then return results for later work ?
RFC is for calling function modules, not programs. It's possible to use some generic function module to start a report, but since you'll usually want to process the results of the program and the program does not know that it was meant to deliver its results in a machine-readable way, you probably won't get too far this was. What exactly are you trying to do?
With the nearly infinite possible results of calling a transaction, i don't think there is a RFC to execute such an operation and return a result. What would be the result in case of an ALV display, or if the program then wait for some interactions ?
You can display a transaction in SAP portal using transactions Iviews. You're then using the portal page as a HTMLGui for your transaction.
also, some FM can sometime be used to perform operations instead of a full program (ie HR_INFOTYPE_OPERATION instead of pa30).
regards
Guillaume
Edition : since you want the result of RRSPARAM, you could encapsulate the "important" part (form SHOW_ACTUAL_PAR_VALUES_ALV) in a module function accessible by RFC, and returning a table of CST_RSPFPAR_ALV (ie the same structure that is displayed in the report)
regards
If You don't find a function to call, just create one by yourself. Tag it as callable from outside via RFC and in the coding, perform such things as "submit report xyz with param1 = value1 ... and return ... exporting list to memory". Then you can even return list output from this. Define the interface of the freshly created function module as you need (that means, report name as input, list output as a table of strings, e.g.). Attention, there is of course a big security risk, having an remote function accepting variable reportnames. But I am sure You know :-)