Opentelemetry 1.4.0 context propagation - instrumentation

I was running Opentelemetry 0.18rc1 and my application was working perfectly.
I'm using the W3C Trace Context specification for context propagation. For injection and extraction i used TraceContextTextMapPropagator
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from opentelemetry.context import get_current
prop = TraceContextTextMapPropagator()
carrier = {}
prop.inject(set_in_carrier=dict.__setitem__, carrier=carrier, context=get_current())
and for extraction in the next micro-service, i used:
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from opentelemetry.propagators import textmap
prop = TraceContextTextMapPropagator()
carrier_getter = textmap.DictGetter()
context = prop.extract(carrier_getter, request.headers)
When I tried to upgrade to the latest opentelemetry 1.4.0 my injection and extraction methods stopped working. It seems the DictGetter() class was removed from the new version, so i don't know how to set the getter parameter in the extract method. Also, the set_in_carrier was replaced with a setter parameter in the inject method and I'm not sure how to to set this one either.
How do I implement the Inject and Extract methods in opentelemetry 1.4.0 for W3C Trace Context specification?

set_in_carrier was replaced with something called Setter and DefaultSetter implements that to set value in dict-like carriers (ex. HTTP headers). You don't need to explicitly pass the current context because If you don't pass any context it takes the current context. And Inject into carrier would be simplified to
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
prop = TraceContextTextMapPropagator()
carrier = {}
prop.inject(carrier=carrier)
And when you extract it would be
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
prop = TraceContextTextMapPropagator()
context = prop.extract(carrier=request.headers)
Here is a working example when injecting and extracting happens in the same file but it real world it is usually happens in different services.
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (BatchSpanProcessor,
ConsoleSpanExporter)
from opentelemetry.trace.propagation.tracecontext import \
TraceContextTextMapPropagator
trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))
tracer = trace.get_tracer(__name__)
prop = TraceContextTextMapPropagator()
carrier = {}
# Injecting
with tracer.start_as_current_span("first-span") as span:
prop.inject(carrier=carrier)
print("Carrier after injecting span context", carrier)
# Extracting
ctx = prop.extract(carrier=carrier)
with tracer.start_as_current_span("next-span", context=ctx):
pass

Related

Internationalization in Ktor based website

I'm new to Ktor and Kotlin in general, so please be patient.
I am currently trying to create a little website (mostly for learning) that uses key-value files for internationalization.
I already did something similar in PHP where I just decoded a JSON file and got the value related to the key I passed. This way, I could do something as <p><?php echo $langJson["presentation"][0];?></p> (with $langJson being my json key-value file) so that I would get the proper translation.
I'm trying to do an equivalent in Kotlin using Ktor, but I don't know how to do it. I found the aymanizz ktor-i18n plugin on GitHub that allows to use i18n for internationalization but I don't know if it is really adapted to what I want to do since it detects the language in the header instead of it being chose by the user (with _GET for instance).
Does anyone have any clue on how I could do that?
Briefly, what I want to do is having a single coded page where the content is dynamicly chosen from the accurate language file.
Thank you all! :)
The basic idea is to get a language code from a request (a query parameter, a header, etc), generate a path to an i18n resource file, read it and then deserialize JSON into a map. The resulting map could be used as-is or passed as a model to a template.
Here is an example where I use kotlinx.serialization to transform a JSON string to get a map and FreeMarker template engine to render HTML. To switch a language just use the lang GET parameter, e.g, http://localhost:8080/?lang=es.
import freemarker.cache.ClassTemplateLoader
import io.ktor.application.*
import io.ktor.freemarker.*
import io.ktor.request.*
import io.ktor.response.*
import io.ktor.routing.*
import io.ktor.server.engine.*
import io.ktor.server.netty.*
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json
fun main() {
embeddedServer(Netty, port = 8080) {
install(FreeMarker) {
templateLoader = ClassTemplateLoader(this::class.java.classLoader, "templates")
}
routing {
get("/") {
call.respond(FreeMarkerContent("index.ftl", mapOf("i18n" to loadI18n(call.request))))
}
}
}.start()
}
fun loadI18n(request: ApplicationRequest): Map<String, String> {
val language = request.queryParameters["lang"] ?: "en"
val filePath = "i18n/$language.json"
val data = object {}.javaClass.classLoader.getResource(filePath)?.readText() ?: error("Cannot load i18n from $filePath")
return Json.decodeFromString(data)
}
resources/templates/index.ftl
<html>
<body>
<h1>${i18n.greetings}</h1>
</body>
</html>
resources/i18n/en.json
{
"greetings": "Hello"
}
resources/i18n/es.json
{
"greetings": "Hola"
}
If you want to fully support of i18n, i recomand to use https://github.com/aymanizz/ktor-i18n. You will have ability to use plulars and other thinks from i18n standart.

How to test Apache HttpClient RequestConfig values are set correctly? No public getters present

I have this class to configure a HttpClient instance:
package com.company.fraud.preauth.service.feignaccertifyclient;
import com.company.fraud.preauth.config.ProviderClientConfig;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.ssl.SSLContextBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
#Slf4j
#Configuration
#RequiredArgsConstructor
public class FeignClientConfig {
private final ProviderClientConfig providerClientConfig;
public HttpClient buildHttpClient() throws NoSuchAlgorithmException, KeyStoreException, KeyManagementException {
RequestConfig.Builder requestBuilder = RequestConfig.custom();
requestBuilder.setConnectTimeout(providerClientConfig.getConnectionTimeout());
requestBuilder.setConnectionRequestTimeout(providerClientConfig.getConnectionRequestTimeout());
requestBuilder.setSocketTimeout(providerClientConfig.getSocketTimeout());
SSLContextBuilder builder = new SSLContextBuilder();
builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
return HttpClientBuilder.create()
.setMaxConnPerRoute(providerClientConfig.getMaxConnectionNumber())
.setDefaultRequestConfig(requestBuilder.build())
.setSSLContext(builder.loadTrustMaterial(null, new TrustSelfSignedStrategy()).build())
.build();
}
}
How to unit test this class, to see into the resulted HttpClient that these values are correctly set?
From the httpClient I cannot get access to its RequestConfig.
I am aware of these two posts:
How do I test a private function or a class that has private methods, fields or inner classes?
(the number of upvotes in this question shows that it is a concurrent and controversial topic in testing, and my situation may offer an example that why we should look into the inner state of an instance in testing, despite that it is private)
Unit test timeouts in Apache HttpClient
(it shows a way of adding an interceptor in code to check configure values, but I don't like it because I want to separate tests with functional codes)
Is there any way? I understand that this class should be tested, right? You cannot blindly trust it to work; and checking it "notNull" seems fragile to me.
This link may point me to the right direction:
https://dzone.com/articles/testing-objects-internal-state
It uses PowerMock.Whitebox to check internal state of an instance.
So I have checked into PowerMock.Whitebox source code, and it turns out reflection is used internally. And, as PowerMock is said to be not compatible with JUnit 5 yet(till now), and I don't want to add another dependency just for testing, so I will test with reflection.
package com.company.fraud.preauth.service.feignaccertifyclient;
import com.company.fraud.preauth.config.PreAuthConfiguration;
import com.company.fraud.preauth.config.ProviderClientConfig;
import com.company.fraud.preauth.config.StopConfiguration;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import java.lang.reflect.Field;
import static org.hamcrest.CoreMatchers.equalTo;
import static org.junit.Assert.assertThat;
import static org.mockito.Mockito.when;
#ExtendWith(SpringExtension.class)
#SpringBootTest(classes = {
PreAuthConfiguration.class,
StopConfiguration.class,
})
public class FeignClientConfigTest {
#Mock
private ProviderClientConfig providerClientConfig;
#Test
#DisplayName("should return HttpClient with defaultConfig field filled with values in providerClientConfig")
public void shouldReturnHttpClientWithConfiguredValues() throws Exception {
// given
when(providerClientConfig.getConnectionRequestTimeout()).thenReturn(30000);
when(providerClientConfig.getConnectionTimeout()).thenReturn(30);
when(providerClientConfig.getMaxConnNumPerRoute()).thenReturn(20);
when(providerClientConfig.getSocketTimeout()).thenReturn(10);
FeignClientConfig feignClientConfig = new FeignClientConfig(providerClientConfig);
// when
HttpClient httpClient = feignClientConfig.buildHttpClient();
// then
// I want to test internal state of built HttpClient and this should be checked
// I tried to use PowerMock.Whitebox, but then I found it uses reflection internally
// I don't want to introduce another dependency, and PowerMock is said not to be compatible with JUnit 5, so..
Field requestConfigField = httpClient.getClass().getDeclaredField("defaultConfig");
requestConfigField.setAccessible(true);
RequestConfig requestConfig = (RequestConfig)requestConfigField.get(httpClient);
assertThat(requestConfig.getConnectionRequestTimeout(), equalTo(30000));
assertThat(requestConfig.getConnectTimeout(), equalTo(30));
assertThat(requestConfig.getSocketTimeout(), equalTo(10));
}
}
Also, I answer the first question in OP about when to test private members in a class here
Whitebox was working for me. As it is not documented here I'm adding my version:
in my case wanted to test that the timeout is different from 0 to avoid deadlock
HttpClient httpClient = factory.getHttpClient();
RequestConfig sut = Whitebox.getInternalState(httpClient, "defaultConfig");
assertNotEquals(0, sut.getConnectionRequestTimeout());
assertNotEquals(0, sut.getConnectTimeout());
assertNotEquals(0, sut.getSocketTimeout());

Cannot retrieve inferences for named individual added via OWL API without reloading ontology

In my application I need to add named individuals to an ontology. At a later point I need to be able to retrieve these named individuals and determine their inferred types, but for some reason I am not able to retrieve their types. I get either an exception or an empty set depending on the OWL reasoner I am using.
Here is a self contained example illustrating the problem:
package owl.api.test.StandaloneOWLNamedIndividualRetrievalv5;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Set;
import java.util.stream.Collectors;
import org.semanticweb.HermiT.ReasonerFactory;
import org.semanticweb.owlapi.apibinding.OWLManager;
import org.semanticweb.owlapi.model.IRI;
import org.semanticweb.owlapi.model.OWLClass;
import org.semanticweb.owlapi.model.OWLClassAssertionAxiom;
import org.semanticweb.owlapi.model.OWLDataFactory;
import org.semanticweb.owlapi.model.OWLIndividual;
import org.semanticweb.owlapi.model.OWLOntology;
import org.semanticweb.owlapi.model.OWLOntologyManager;
import org.semanticweb.owlapi.model.OWLOntologyStorageException;
import org.semanticweb.owlapi.model.PrefixManager;
import org.semanticweb.owlapi.model.parameters.ChangeApplied;
import org.semanticweb.owlapi.reasoner.NodeSet;
import org.semanticweb.owlapi.reasoner.OWLReasoner;
import org.semanticweb.owlapi.reasoner.OWLReasonerFactory;
import org.semanticweb.owlapi.search.EntitySearcher;
import org.semanticweb.owlapi.util.DefaultPrefixManager;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.Marker;
import org.slf4j.MarkerFactory;
import openllet.owlapi.OpenlletReasonerFactory;
import uk.ac.manchester.cs.jfact.JFactFactory;
public class App {
private static Logger logger = LoggerFactory
.getLogger(owl.api.test.StandaloneOWLNamedIndividualRetrievalv5.App.class);
// Why This Failure marker
private static final Marker WTF_MARKER = MarkerFactory.getMarker("WTF");
public static void main(String[] args) {
try {
// Setup physical IRI for storing ontology
Path path = Paths.get(".").toAbsolutePath().normalize();
IRI loadDocumentIRI = IRI.create("file:" + path.toFile().getAbsolutePath() + "/SimpleOntology.owl");
logger.trace("documentIRI=" + loadDocumentIRI);
IRI saveDocumentIRI = IRI.create("file:" + path.toFile().getAbsolutePath() + "/SimpleOntologyUpdated.owl");
logger.trace("documentIRI=" + saveDocumentIRI);
// Initialize
OWLOntologyManager manager = OWLManager.createOWLOntologyManager();
OWLDataFactory dataFactory = manager.getOWLDataFactory();
OWLOntology ontology = manager.loadOntologyFromOntologyDocument(loadDocumentIRI);
// OWLReasonerFactory reasonerFactory = new JFactFactory();
OWLReasonerFactory reasonerFactory = new ReasonerFactory();
// OWLReasonerFactory reasonerFactory = OpenlletReasonerFactory.getInstance();
OWLReasoner reasoner = reasonerFactory.createReasoner(ontology);
PrefixManager pm = new DefaultPrefixManager(ontology.getOntologyID().getOntologyIRI().get().getIRIString());
// Get references to a new named individual and an existing class
OWLIndividual individual = dataFactory.getOWLNamedIndividual("#ind1", pm);
OWLClass owlClass = dataFactory.getOWLClass("#ClassB", pm);
// Create class assertion axiom
OWLClassAssertionAxiom classAssertionAxiom = dataFactory.getOWLClassAssertionAxiom(owlClass, individual);
// Add class assertion axiom to ontology
ChangeApplied changeApplied = manager.addAxiom(ontology, classAssertionAxiom);
logger.trace("ChangeApplied = " + changeApplied);
if (changeApplied.equals(ChangeApplied.SUCCESSFULLY)) {
try {
manager.saveOntology(ontology, saveDocumentIRI);
} catch (OWLOntologyStorageException e) {
logger.error(e.getMessage());
}
}
// Now try to retrieve the individual
logger.trace(
"Trying to retrieve individual = " + classAssertionAxiom.getIndividual().asOWLNamedIndividual());
Set<Object> classExpressionTypes = EntitySearcher.getTypes(classAssertionAxiom.getIndividual(), ontology)
.collect(Collectors.toSet());
logger.trace("Individual = " + classAssertionAxiom.getIndividual() + " has types based on EntitySearcher "
+ classExpressionTypes);
NodeSet<OWLClass> types = reasoner.getTypes(classAssertionAxiom.getIndividual().asOWLNamedIndividual(),
false);
logger.trace("Individual = " + classAssertionAxiom.getIndividual()
+ " has types based on reasoner.getTypes " + types);
} catch (Throwable t) {
logger.error(WTF_MARKER, t.getMessage(), t);
}
}
}
Here is the Simple ontology I use to test against:
<?xml version="1.0"?>
<rdf:RDF xmlns="http://www.semanticweb.org/2017/simple#"
xml:base="http://www.semanticweb.org/2017/simple"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:owl="http://www.w3.org/2002/07/owl#"
xmlns:xml="http://www.w3.org/XML/1998/namespace"
xmlns:xsd="http://www.w3.org/2001/XMLSchema#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
<owl:Ontology rdf:about="http://www.semanticweb.org/2017/simple"/>
<owl:Class rdf:about="http://www.semanticweb.org/2017/simple#ClassA"/>
<owl:Class rdf:about="http://www.semanticweb.org/2017/simple#ClassB">
<rdfs:subClassOf rdf:resource="http://www.semanticweb.org/2017/simple#ClassA"/>
</owl:Class>
</rdf:RDF>
After running this code I expect that it will determine that the types of individual ind1 are Thing, ClassA and ClassB.
Thinking that this problem is related to perhaps to a specific OWL reasoner, I have tried using JFact, HermiT and Openllet. JFact throws a NullPointerException, HermiT only returns owl:Thing and Openllet nothing. However, when I save the changes to the ontology to file and reload it, I can find the inferred types of the individual I have added using any of these reasoners.
I have tested this with versions 5.1.2 and 4.5.0 of the OWL API. I have also tried calling reasoner.precomputeInferences() even though the documentation states that this is not necessary, but it made no difference.
reasonerFactory.createReasoner(ontology) creates a buffering reasoner, i.e. it has to be synchronized manually after you changed the ontology.
More details from the Javadoc:
Ontology Change Management (Buffering and Non-Buffering Modes)
At creation time, an OWLReasoner will load the axioms in the root
ontology imports closure. It will attach itself as a listener to the
OWLOntologyManager that manages the root ontology. The reasoner will
listen to any OWLOntologyChanges and respond appropriately to them
before answering any queries. If the BufferingMode of the reasoner
(the answer to getBufferingMode() is BufferingMode.NON_BUFFERING) the
ontology changes are processed by the reasoner immediately so that any
queries asked after the changes are answered with respect to the
changed ontologies. If the BufferingMode of the reasoner is
BufferingMode.BUFFERING then ontology changes are stored in a buffer
and are only taken into consideration when the buffer is flushed with
the flush() method. When reasoning, axioms in the root ontology
imports closure, minus the axioms returned by the
getPendingAxiomAdditions() method, plus the axioms returned by the
getPendingAxiomRemovals() are taken into consideration. Note that
there is no guarantee that the reasoner implementation will respond to
changes in an incremental (and efficient manner) manner.
Two options:
Call reasoner.flush() before you're asking the reasoner for inferences.
Create a non-buffering reasoner, i.e. use reasonerFactory.createNonBufferingReasoner(ontology)
the problem is that the reasoner does use the ontology at it is when the reasoner is created. I don't now if you use Protege (Desktop), which uses the OWL API under the hood. I you do and also use a reasoner in Protege you should have noticed that you have to refresh the reasoner after you made changes to the ontology. In Protege this is also indicated in the status line at the bottom of the window.
You have to recreate the reasoner every time you do changes to the ontology. The solve the problem in your example add the following line before block where you retrieve the individuals:
reasoner = reasonerFactory.createReasoner(ontology);
Best regards
Jens

How to use scalajs-bundler with client only app

In another question I was advised to use ScalaJS bundler to import NPM dependencies.
I would like to use some Javascript NPM packages in a simple client-only web application. There is an example called static which shows this.
My changes to the example:
Add into build.sbt:
npmDependencies in Compile += "esprima" -> "3.1.3"
Add into Main.scala:
import Esprima._
import JsonToString._
val code = "answer = 42"
val tokens = tokenize(code)
val tokensStr = tokens.json
Change in Main.scala: "This is bold" into s"This is bold $tokensStr"
Facade (a bit simplified, for full a version see GitHub):
import scala.scalajs.js
import scala.scalajs.js.annotation.JSName
#JSName("esprima")
#js.native
object Esprima extends js.Object {
def tokenize(input: String, config: js.Any = js.native, delegate: String => String = js.native): js.Array[js.Any] = js.native
def parse(input: String, config: js.Any = js.native): js.Dynamic = js.native
}
When running the html generated with fastOptJS::webpack the error is:
Uncaught TypeError: Cannot read property 'tokenize' of undefined
Inspecting the static-fastopt-bundle.js shows esprima is used, but its js is not bundled.
What other steps are needed to add dependencies into a client-only web page?
As described in this part of the documentation, you have to use #JSImport in your facade definition:
#JSImport("esprima", JSImport.Namespace)
For reference, #JSName defines a facade bound to a global name, while #JSImport defines a facade bound to a required JavaScript module.

Left-hand side of assignment expression cannot be a constant or a read-only property

When I use this line on my Express server, it works well in TypeScript 1.x
mongoose.Promise = global.Promise;
( The usage of mongoose.Promise = global.Promise; is from the mongoose document )
After updating to TypeScript 2.x, it shows this error in the terminal, and won't let me start the server.
Left-hand side of assignment expression cannot be a constant or a
read-only property.
How can I solve this? Thanks
This is because in es6 all module's variables are considered constants.
https://github.com/Microsoft/TypeScript/issues/6751#issuecomment-177114001
In TypeScript 2.0 the bug (of not reporting this error) was fixed.
Since mongoose is still using the commonjs - var mongoose = require("mongoose") - not the es6 import syntax (which is used in the typings), you can suppress the error by assuming the module is of type any.
WORKAROUND:
(mongoose as any).Promise = global.Promise;
There is also a way to maintain type-checking and intellisense with this technique.
import * as mongoose from "mongoose"; // same as const mongoose = require("mongoose");
type mongooseType = typeof mongoose;
(mongoose as mongooseType).Promise = global.Promise;
// OR
(<mongooseType>mongoose).Promise = global.Promise;
This can be a helpful way to override only certain functions within a module with mock functions without needing a mock framework like jest.mock().