I'm trying to run a SPARQL query on Wikidata, but it times out. I'd like to download a dump and index it in some database, so I can run local SPARQL queries using HTTP requests. I also need to support Wikidata-specific extensions like SERVICE wikibase:label. I've downloaded an RDF dump. What are the next steps?
Wikimedia has documentation on how to run your own SPARQL endpoint from one of their dumps. They also have an updater that streams updates from their servers, to keep your endpoint up-to-date.
You won't need to do anything special to support their extensions, it is included by default.
I recommend using a reverse proxy (like nginx or apache) with http auth in production, the admin dashboard is accessible by default.
Related
We are using kong api gateway as a single gateway for all apis. we are facing latency issue with few of our api's (1500-2000ms). later when we checked, latency was being created because of the "rate limiting" plugin. When we disable the plugin, latency improves and the response is same as what we get directly from IP (close to 300ms approx).
I m trying to setup redis node to cache database queries, I m not sure how we can configure kong to read from redis itself. how we can cache the database queries to redis node.
We are using postgresql for kong.
I think maybe you are trying to do a couple different things at once.
First, rate-limiting: what is the value for your config.policy parameter? The Kong documentation has three values: "local (counters will be stored locally in-memory on the node), cluster (counters are stored in the datastore and shared across the nodes) and redis (counters are stored on a Redis server and will be shared across the nodes)."
If you are seeing high latency, and your config.policy is set to cluster or redis, it might be due to latency between Kong and postgres/redis (depending on what policy you're using). If you are using rate-limiting just to prevent abuse, using the 'local' policy is faster. (There's more about this at the Kong documentation.)
The other question is about caching: Kong Enterprise has a built-in caching plugin, but for Kong Community, since it's built on top of Nginx, you can do caching with Nginx. This link might help you.
There is a community custom plugin out there that enables the use of caching with redis without the need to use the Kong Enterprise -> https://github.com/globocom/kong-plugin-proxy-cache
Maybe you could combine that with rate limiting to achieve the desired latency performance or use this plugin as inspiration.
can anyone point me to a resource, how I can get spring-webflux and javamelody to work together?
Seems, that a servletcontext is neccessary for startup, which I don't have/need.
I'm aware of the coll metrics stuff, that comes with spring-boot 2.x, but I don't have anything to display the metrics with, and am locked to a company environment, where just installing something isn't a valid option.
Thanks,
Henning
javamelody is mainly based on monitoring of memory, cpu, http requests, sql requests and spring components among other things. See javamelody-spring-boot-starter for example.
But as far as I know, Spring webflux does not use the servlet api. So what do you want to monitor?
If you just want to have graphs in a browser, then start a http server for javamelody reports like in standalone. And if you also want to monitor sql requests and spring components, then add in your application all methods from this example, except monitoringSessionListener and monitoringFilter.
A new spring-boot-starter for javamelody in webflux could be created if it makes sense.
We are running multiple fuseki servers. We want to run Sparql queries using data from any number of them. That means using the SERVICE key word, no problem.
How do we set up authentication in Fuseki server A to access Fuseki server B?
Presumably it is done using a service .ttl file, but after waterboarding google for an hour it still won't give an straight answer.
I am looking for a simple HTTP server which accepts (only GET) commands, queries the redis DB with a key and sends the reply (value) back in text format. My only requirement is that the server is very lightweight and can access a backend DB. Thanks in advance.
Try Webdis.
Webdis is a simple HTTP server which forwards commands to Redis and sends the reply back using a format of your choice.
I recently looked for the same kind of HTTP server.
As mentioned by Evandro you can try Webdis or you can go for Nginx with some modules.
In your case, for GET requests only, you can install Nginx with the HttpRedis module.
If later your requirements evolve, you can always go for HttpRedis2Module which support all the Redis commands.
I personally use the HttpLuaModule with the lua-resty-redis module and lua-cjson.
Once you got the HttpLuaModule running it's really easy to add new lua-modules and to extend the capabilities of Nginx. The resty-redis module let you add some logic between the HTTP request handling and your Redis queries using lua. You also have a large number of examples on the module setup and usage on github.
Adding cjon let you return JSON instead of raw text.
Use Webdis as suggested or Mod_redis (module for Redis) with nginx or apache2 server as per your requirements.
I am working on a project which has following requirements:
Perform sticky based load balancing(based on SOAP session ID) onto multiple backend servers.
Possibility to plugin my own custom based load balancer.
Easy to write and deploy.
A central configuration file(Possibly an XML), to take care of all the backend servers.
Easy extraction of a node from this configuration file(Possibly with xpath).
I tried working with camel for a while but, wasn't able to do perform certain task with it.
So thought of giving a try to Akka.
Will akka be possibly able to satisfy the above requirements?
If so is there a load balancing example in akka or proxy example?
Would really appreciate some feedbacks.
You can do everything you've described with Akka.
You don't mention what language you're working with, Scala or Java. I've included links to the Scala documentation.
Before you do anything with Akka you HAVE TO read the documentation and understand how Akka works.
http://doc.akka.io/docs/akka/2.0.3/
Doing so, you'll find Akka is perfect for the project you've described with some minor caveats.
Once you read the documentation the following answers should make a lot of sense.
Perform sticky based load balancing(based on SOAP session ID) onto multiple backend servers.
Load balancing is already part of the framework (it's called Routing in Akka http://doc.akka.io/docs/akka/2.0.3/scala/routing.html) and Remoting (http://doc.akka.io/docs/akka/2.0.3/scala/remoting.html) will take care of the backend servers. You can easily combine the two.
To my knowledge the idea of sticky load balancing is not a part of Akka but I can envision this being accomplished with a Map using the session ID as the key and the Actor name (or path) as the value. A quick actorFor will take care of the rest. Not well thought out but should give you a good idea of where to start.
Possibility to plugin my own custom based load balancer.
Refer to the Routing documentation.
Easy to write and deploy.
This depends on your aptitude and effort but after you read certain parts of the documentation you should be build a proof of concept in a couple of hours.
Deployment can be a bit frustrating mostly because the documentation isn't really great with respect to deploying Akka networks with remote components. However, there are enough examples on the web that you can figure out how to get it done...eventually. Once you do it once it's no big deal.
A central configuration file(Possibly an XML), to take care of all the backend servers.
Akka uses Typesafe Config (https://github.com/typesafehub/config) which is a lot easier to work with than XML (but I hate XML so take that with a grain of salt). As far as a central configuration, I'm not sure what you're trying to accomplish but it sounds like something that can be solved using remote actor creation. Again, see the Remoting documentation.
Easy extraction of a node from this configuration file(Possibly with xpath).
Akka provides a lookup method .actorFor. There's no need to go to the configuration file once the system is up and running.
If so is there a load balancing example in akka or proxy example?
Google is your friend.