For my project I need a new property for json schemas.
I named it "isPropertyOf" and basically what I want is that if I have this:
fruits = {
"banana": "yellow fruit",
"apple": "round fruit"
}
schema = {
"type": "object",
"properties": {
"fav_fruit": {
"type": "string",
"isPropertyOf": fruits
}
}
}
Then schema would validate only objects like {"fav_fruit":"banana"} or {"fav_fruit":"apple"}, but not {"fav_fruit":"salami"}
(I know that for this very example using an enum would make more sense, but assume "fruits" is also used for other stuff and I would rather avoid redundancy)
I read docs about this and figured I need to use jsonschema.validators.extend. I tried something like that:
def is_property_of_callable(validator_instance, property_value, instance, schema):
keys = [k for k in property_value]
return instance in keys
mapping = {"isPropertyOf": is_property_of_callable}
my_validator = jsonschema.validators.extend(jsonschema.Draft7Validator, mapping)
instance = {"fav_fruit": "tobacco"}
my_validator(schema).is_valid(instance)
I was ready to see something go wrong, but apparently the validator wasn't seeing any issue. I tried with an obviously wrong instance that even the standard validator of jsonschema wouldn't accept
my_validator(schema).is_valid({"fav_fruit": 0})
But apparently it looked ok to it.
I thought maybe the way I added this property was so broken that it was making the validator accept anything, so I tried a minimal case of validator extension:
minimal_validator = jsonschema.validators.extend(jsonschema.Draft7Validator, {})
minimal_validator(schema).is_valid(instance)
And this one is also happy with 0 being my favourite fruit.
What am I doing wrong here? How can I make that work?
Related
I have this json:
{
"$type": "System.Collections.Generic.List<MyType>",
"$values": [
{
"$type": "MyType",
"o": 7.54,
"t": 1619002800000,
"n": 3
},
{
"$type": "MyType",
"o": 7.53,
"t": 1619005140000,
"n": 3
}
]
}
I want to deserialize it back into a List<MyType>. I thought there would be an easy way to do that some thing like this:
var myList = json.FromJson<MyType>();
but that doesn't work.
I have figured out a way to accomplish my goal but it's a bit messy so I was wondering if there's a better way that I'm not aware of. Here's the messy way I came up with:
var myListOfObject = (List<object>)((Dictionary<string, object>)JSON.parse(json))["$values"];
var myTypes = myListOfObject.ConvertAll(x => JSON.stringify(x).FromJson<MyType>());
I'm not necessarily looking for fewer lines of code because 2 isn't anything to complain about. I'm just hoping there is a way that doesn't require all the casting and parsing and rather can accept the json as is and get it back to the type it came from. Maybe there's even a parameter I can set to tell it to validate types during the deserialization since the full type names are in the json.
You should use the same serializer you used to serialize the payload to deserialize it. ServiceStack.Text uses __type to embed its type information, in a different schema so you wont be able to use ServiceStack.Text to automatically deserialize it into the embedded type.
This likely used JSON.NET which you should use instead to deserialize it, otherwise yeah you can use ServiceStack's JS Utils to deserialize arbitrary JSON as you're doing.
The following lines show how to generate a key in Tink:
keysetHandle=KeysetHandle.generateNew(AeadKeyTemplates.AES128_GCM)
privateKeysetHandle = KeysetHandle.generateNew(SignatureKeyTemplates.ECDSA_P256)
Could you show me how to construct a key given the parameters such as key bytes and related parameters?
It is also possible to create a key by loading the parameters from JSON:
String keysetFilename = "my_keyset.json";
KeysetHandle keysetHandle = CleartextKeysetHandle.read(
JsonKeysetReader.withFile(new File(keysetFilename)));
How is the key format in JSON defined?
Maarten Bodewes: would you mind tell us what wrong with the APIs, and how you think it should be changed? We're all ears for feedback.
Ursa Major: we don't want users to deal with keys directly, because it's easy to mess up. It's why we provide APIs that generate, persist and load keys. The Java HOWTO [1] shows how to do this.
It looks like you have an existing key, in some other format, that you want to use it with Tink. Tink's keys are stored in protobuf. Each key type is defined in its own protobuf. You can find all definitions at https://github.com/google/tink/tree/master/proto. Tink doesn't work with individual keys, but keysets which are also protobuf. You can convert existing keys to Tink's keysets by providing an implementation of KeysetReader. SignaturePemKeysetReader [2] is an example that converts certain PEM keys to Tink.
If you encounter any further issue, feel free to comment or email the mailing list at tink-users#googlegroups.com.
Hope that helps,
Thai.
[1] https://github.com/google/tink/blob/master/docs/JAVA-HOWTO.md
[2] https://github.com/google/tink/blob/master/java_src/src/main/java/com/google/crypto/tink/signature/SignaturePemKeysetReader.java
edit: update the second link.
I've had a similar problem, but with HMAC in unit tests. Hope it helps.
Example JSON:
{
"primaryKeyId": 2061245617,
"key": [{
"keyData": {
"typeUrl": "type.googleapis.com/google.crypto.tink.HmacKey",
"keyMaterialType": "SYMMETRIC",
"value": "EgQIAxAgGiB9qbGjo1sA41kHHKbELAKmFzj3cNev0GJ3PpvhR00vuw=="
},
"outputPrefixType": "TINK",
"keyId": 2061245617,
"status": "ENABLED"
}]
}
code used to generate it (Scala):
import com.google.crypto.tink.mac.MacConfig
MacConfig.register()
def generate(): Unit = {
import java.io.ByteArrayOutputStream
import java.nio.charset.StandardCharsets
import com.google.crypto.tink.mac.HmacKeyManager
import com.google.crypto.tink.{CleartextKeysetHandle, JsonKeysetWriter, KeysetHandle}
val generatedKeyset = KeysetHandle.generateNew(HmacKeyManager.hmacSha256Template())
val output = new ByteArrayOutputStream
CleartextKeysetHandle.write(generatedKeyset, JsonKeysetWriter.withOutputStream(output))
println(output.toString(StandardCharsets.UTF_8))
}
generate()
Loading the JSON and usage:
import com.google.crypto.tink.{CleartextKeysetHandle, JsonKeysetReader}
val hmacKeyset = CleartextKeysetHandle.read(
JsonKeysetReader.withString(...)
)
val mac = hmacKeyset.getPrimitive(classOf[Mac])
mac.computeMac(...)
Keep in mind this is totally insecure and should never be used outside tests.
Relevant parts of the implementation:
JsonKeysetReader.keyFromJson
hmac.proto
MacIntegrationTest
EDIT:
Even easier way to generate a keyset JSON:
$ tinkey create-keyset --key-template HMAC_SHA256_256BITTAG
{
"primaryKeyId": 1132518908,
"key": [{
"keyData": {
"typeUrl": "type.googleapis.com/google.crypto.tink.HmacKey",
"keyMaterialType": "SYMMETRIC",
"value": "EgQIAxAgGiDwIucBpWJ8WHVIEKIdEVQlfynm+4QS8sKUVUga2JzRlw=="
},
"outputPrefixType": "TINK",
"keyId": 1132518908,
"status": "ENABLED"
}]
}
How can i get my json from web api to format only value or null for Option types and Discriminated Unions preferably using Newtonsoft.
I am currently using Newtonsoft and only have to add this to web api for it to work:
config.Formatters.JsonFormatter.SerializerSettings <- new JsonSerializerSettings()
When i consume the data on my side, i can easily convert it back to an F# item using: JsonConvert.DeserializeObject<'a>(json)
The api will be consumed by NON .NET clients as well so i would like a more standard formatted json result.
I would like to to fix my issue, w/o having to add code or decorators to all of my records/DU in order for it to work. I have lots of records with lots of properties, some are Option.
ex (this is how DU is serializing):
// When value
"animal": {
"case": "Dog"
}
// When no value
"animal": null
This is what I need:
// When value
"animal": "Dog"
// When no value
"animal": null
This is how an Option type is serializing:
"DocumentInfo": {
"case": "Some",
"fields": [
{
"docId": "77fb9dd0-bfbe-42e0-9d29-d5b1f5f0a9f7",
"docType": "Monkey Business",
"docName": "mb.doc",
"docContent": "why cant it just give me the values?"
}
]
}
This is what I need:
"DocumentInfo": {
"docId": "77fb9dd0-bfbe-42e0-9d29-d5b1f5f0a9f7",
"docType": "Monkey Business",
"docName": "mb.doc",
"docContent": "why cant it just give me the values?"
}
Thank you :-)
You could try using Chiron. I haven't used it myself so I can't give you an extensive example, but https://neoeinstein.github.io/blog/2015/12-13-chiron-json-ducks-monads/index.html has some bits of sample code. (And see https://neoeinstein.github.io/blog/2016/04-02-chiron-computation-expressions/index.html as well for some nicer syntax). Basically, Chiron knows how to serialize and deserialize the basic F# types (strings, numbers, options, etc.) already, and you can teach it to serialize any other type by providing that type with two static methods, ToJson and FromJson:
static member ToJson (x:DocumentInfo) = json {
do! Json.write "docId" x.docId
do! Json.write "docType" x.docType
do! Json.write "docName" x.docName
do! Json.write "docContent" x.docContent
}
static member FromJson (_:DocumentInfo) = json {
let! i = Json.read "docId"
let! t = Json.read "docType"
let! n = Json.read "docName"
let! c = Json.read "docContent"
return { docId = i; docType = t; docName = n; docContent = c }
}
By providing those two static methods on your DocumentInfo type, Chiron will automatically know how to serialize a DocumentInfo option. At least, that's my understanding -- but the Chiron documentation is sadly lacking (by which I mean literally lacking: it hasn't been written yet), so I haven't really used it myself. So this may or may not be the answer you need, but hopefully it'll be of some help to you even if you don't end up using it.
I have found the solution that allows me to use Newtonsoft (JSON.NET), apply custom converters for my types where needed and not require any changes to my DU's or Records.
The short answer is, create a custom converter for Json.Net and use the Read/Write Json overrides:
type CustomDuConverter() =
inherit JsonConverter() (...)
Unfortunately the ones I have found online that were already created doesn't work as is for my needs listed above, but will with slight modification. A great example is to look at: https://gist.github.com/isaacabraham/ba679f285bfd15d2f53e
To apply your custom serializer in Web Api for every call, use:
config.Formatters.JsonFormatter.SerializerSettings.Converters.Add(new CustomDuConverter())
To deserialize use (example that will deserialize to DU):
JsonConvert.DeserializeObject<Animal>("Dog", customConverter)
ex:
type Animal = Dog | Cat
json:
"animal": "Dog"
This will allow you to create a clean Api for consumers and allow you to consume 3rd party Json data into your types that use Option, etc.
Is there any tool for node express where you can automatically generate swagger documentation for an existing project? Similar to swashbuckle?
I've been looking into this as well, the project which will help you is swagger-node-express. swagger-ui, should come as a dependency of swagger-node-express. Swagger-node-express wraps express and exposes it as a new interface meaning there will be code changes you to make it work. This is what a route would look like (taken from their docs)
var findById = {
'spec': {
"description" : "Operations about pets",
"path" : "/pet.{format}/{petId}",
"notes" : "Returns a pet based on ID",
"summary" : "Find pet by ID",
"method": "GET",
"parameters" : [swagger.pathParam("petId", "ID of pet that needs to be fetched", "string")],
"type" : "Pet",
"errorResponses" : [swagger.errors.invalid('id'), swagger.errors.notFound('pet')],
"nickname" : "getPetById"
},
'action': function (req,res) {
if (!req.params.petId) {
throw swagger.errors.invalid('id');
}
var id = parseInt(req.params.petId);
var pet = petData.getPetById(id);
if (pet) {
res.send(JSON.stringify(pet));
} else {
throw swagger.errors.notFound('pet');
}
}
};
Where the type "Pet" is still for you to define, I wont rewrite their docs here.
This will then produce a file which swagger-ui can use to give you a self contained self documenting system. The documentation for swagger-node-express is more than good enough to get it setup (dont forget to set swagger path, I did).
swagger.configureSwaggerPaths("", "/docs", "");
Having shown you the tools that in theory offer what your asking for, let me explain why I've come to conclusion that I'm not going to use them.
A lot of code change needed - is it really less work than creating
your own swagger.yml file? I dont think so.
Hand creating a swagger.yml file is much less likely to brake your project.
Whilst swagger-node-express hasnt been depricated it's github repo doesnt exist anymore, its been wrapped into swagger-node, but that project doesnt really mention it
I'd be wary of any tool that means I need to wrap express - it's not something I'd look to do.
TL;DR:
It's possible with a lot of code change - probably not what your after though.
I am trying to save (RavenDB build 960) the names and values of form data items passed into a Nancy Module via its built in Request.Form.
If I save a straightforward instance of a dynamic object (with test properties and values) then everything works and both the property names and values are saved. However, if I use Nancy's Request.Form then only the dynamic property names are saved.
I understand that I will have to deal with further issues to do with restoring the correct types when retrieving the dynamic data (RavenJObjects etc) but for now, I want to solve the problem of saving the dynamic names / values in the first place.
Here is the entire test request and code:
Fiddler Request (PUT)
Nancy Module
Put["/report/{name}/add"] = parameters =>
{
reportService.AddTestDynamic(Db, parameters.name, Request.Form);
return HttpStatusCode.Created;
};
Service
public void AddTestDynamic(IDocumentSession db, string name, dynamic data)
{
var testDynamic = new TestDynamic
{
Name = name,
Data = data
};
db.Store(testDynamic);
db.SaveChanges();
}
TestDynamic Class
public class TestDynamic
{
public string Name;
public dynamic Data;
}
Dynamic contents of Request.Form at runtime
Resulting RavenDB Document
{
"Name": "test",
"Data": [
"username",
"age"
]
}
Note: The type of the Request.Form is Nancy.DynamicDictionary. I think this may be the problem since it inherits from IEnumerable<string> and not the expected IEnumerable<string, object>. I think that RavenDB is enumerating the DynamicDictionary and only getting back the dynamic member-names rather than the member name / value pairs.
Can anybody tell me how or whether I can treat the Request.Form as a dynamic object with respect to saving it to RavenDB? If possible I want to avoid any hand-crafted enumeration of DynamicDictionary to build a dynamic instance so that RavenDB can serialise correctly.
Thank You
Edit 1 #Ayende
The DynamicDictionary appears to implement the GetDynamicMemberNames() method:
Taking a look at the code on GitHub reveals the following implementation:
public override IEnumerable<string> GetDynamicMemberNames()
{
return dictionary.Keys;
}
Is this what you would expect to see here?
Edit 2 #TheCodeJunkie
Thanks for the code update. To test this I have:
Created a local clone of the NancyFx/Nancy master branch from
GitHub
Added the Nancy.csproj to my solution and referenced the project
Run the same test as above
RavenDB Document from new DynamicDictionary
{
"Name": "test",
"Data": {
"$type": "Nancy.DynamicDictionary, Nancy",
"username": {},
"age": {}
}
}
You can see that the resulting document is an improvement. The DynamicDictionary type information is now being correctly picked up by RavenDB and whilst the dynamic property-names are correctly serialized, unfortunately the dynamic property-values are not.
The image below shows the new look DynamicDictionary in action. It all looks fine to me, the new Dictionary interface is clearly visible. The only thing I noticed was that the dynamic 'Results view' (as opposed to the 'Dynamic view') in the debugger, shows just the property-names and not their values. The 'Dynamic view' shows both as before (see image above).
Contents of DynamicDictionary at run time
biofractal,
The problem is the DynamicDictionary, in JSON, types can be either objects or lists ,they can't be both.
And for dynamic object serialization, we rely on the implementation of GetDynamicMemberNames() to get the properties, and I assume that is isn't there.