I use Ktor for a backend service and I'd like to log incoming requests. I have the feature installed and everything is great but how I can obtain the remote IP address?
call.request.origin.remoteHost
I use this line but I get a hostname, not the IP. I use the standard getByName method from the InetAddress class to get the IP.
Is there a better way?
You can't do this with Ktor as it is not Ktor's job to resolve IP addresses from domains.
What you can use however is Java's InetAddress:
val url = "http://google.com";
val ip = Inet4Address.getByName(url);
Depends on how accurate you want this information to be.
If you read through Ktor code, you'll see that the remoteHost is actually set from the HTTP X-Forwarded-Host header.
From API documentation:
* NEVER use it for user authentication as it can be easily falsified (user can simply set some HTTP headers
* such as X-Forwarded-Host so you should NEVER rely on it in any security checks.
* If you are going to use it to create a back-connection please do it with care as an offender can easily
* use it to force you to connect to some host that is not intended to be connected to so that may cause
* serious consequences.
A better way might be to get the IP from the application engine itself. Unfortunately, the ApplicationCall itself is private, so you'll have to resort to reflection, which isn't optimal:
class RoutingApplicationCall(private val call: ApplicationCall,
Still, this is possible:
// Getting the private field through reflection
val f = context::class.memberProperties.find { it.name == "call" }
f?.let {
// Making it accessible
it.isAccessible = true
val w = it.getter.call(context) as NettyApplicationCall
// Getting the remote address
val ip: SocketAddress? = w.request.context.pipeline().channel().remoteAddress()
println("IP: $ip")
}
Related
I have an error when starting SpringBootApplication:
Unexpected exception during bean creation; nested exception is java.lang.IllegalArgumentException: url values must be not be absolute.
I'm a beginner is SpringCloud, but I worked with Openshift (On first look it's basically the same things).
I have a cluster with GatewayApplication and some business microservices in it, wrote on Kotlin. Inside cluster microservices communicate by FeignClient without authentification. In consumer-service it looks like it:
#FeignClient(name = "producer-service")
#Headers("Content-Type: application/json")
interface MarketServiceFeign {
#GetMapping("https://somehost.ru/{id}/orders")
fun getUserDevices(
#PathVariable id: String,
): ResponseEntity<List<UserOrder>>
}
I tried find same case, but couldn't.
I tried to:
use #RequestLine from feign-core, but it doesn't work with #FeignClient
use feign.#Param for argument instead of org.springframework.web.bind.annotation.#PathVariable
use url with http instead of https
The thing I didn't take into account is that FeignClient goes to services into cluster by name, not by host. So I fixed it like that:
#FeignClient(name = "producer-service")
#Headers("Content-Type: application/json")
interface MarketServiceFeign {
#GetMapping("/{id}/orders")
fun getUserDevices(
#PathVariable id: String
): ResponseEntity<List<UserDevice>>
}
How I understand, instead of "https://somehost.ru" FeignClient uses service name "producer-service". Result url for FeignClient is "producer-service/{id}/orders".
I hope this helps someone.
If you want to use an absolute URL instead of using load-balancing, you need to pass it via the url attribute in the #FeignClient annotation. It's going to be a URL per Feign client, so you cannot pass it per-request via #RequestMapping annotations. You can only use them to provide the path segments that follow the host url. If you do not pass the url in #FeignClient, the name will be used as serviceId to fetch all the instances of that service (for example, from a service registry) and load-balancing will be performed under the hood to select an instance to send the request to.
Objective:
We want few API calls should go to mock-server(https://192.x.x.x:8001) and others should go to an actual downstream application server(https://dev.api.acme.com).
Setup :
On local, mock server is up with standalone jar on port 8001. e.g https://192.x.x.x:8001
In application config file (config.property)downstream system(which need to mock) defined with mockserver IP i.e https://192.x.x.x:8001
Testing scenario and problem:
1.
Scenario: pathMatches('/profile/v1/users/{id}/user')
* karate.proceed('https://dev.api.acme.com')
* def response = read ('findScope.json')
* def responseStatus = 200ˀˀ
* print 'created response is: ' + response
Now, when we hit API request via postman or feature file then it does karate.proceed properly to https://dev.api.acme.com/profile/v1/users/123/user instead of 192.x.x.x. However, in this request, host is referring to https://192.x.x.x:8001 instead of https://dev.api.acme.com which create a problem for us.
How can we override request header in this case? I did try with karate.set and also with header host=https://192.x.x.x:8001 but no luck.
Thanks!
Please see if the 1.0 version works: https://github.com/intuit/karate/wiki/1.0-upgrade-guide
Unfortunately https proxying may not work as mentioned. If you are depending on this, we may need your help (code contribution) to get this working
If the Host header is still not mutable, that also can be considered a feature request, and here also I'd request you to consider contributing code
I would like to access the docker API using the /var/lib/docker.sock unix domain socket. I've seen examples where you can use (modern versions of) curl to call the API as follows:
curl --unix-socket /var/run/docker.sock http:/containers/json
where the REST command is expressed in the /containers/json path. I was excited to see the Alpakka Unix Domain Socket adapter, but you only seem to be able to send and receive raw bytes. Is there any elegant way to do this? Or do I have to manually construct an HTTP header and manage all the difficult stuff manually?
Here's a working snippet (see also the rest of the discussion at akka/akka-http#2139):
build.sbt:
val scalaV = "2.12.6"
val akkaV = "2.5.14"
val akkaHttpV = "10.1.3"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-http" % akkaHttpV,
"com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpV,
"com.typesafe.akka" %% "akka-stream" % akkaV,
"com.lightbend.akka" %% "akka-stream-alpakka-unix-domain-socket" % "0.20",
)
DockerSockMain.scala:
import java.io.File
import java.net.InetSocketAddress
import akka.actor.ActorSystem
import akka.http.scaladsl.ClientTransport
import akka.http.scaladsl.Http
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import akka.http.scaladsl.model.HttpRequest
import akka.http.scaladsl.model.HttpResponse
import akka.http.scaladsl.settings.ClientConnectionSettings
import akka.http.scaladsl.settings.ConnectionPoolSettings
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.ActorMaterializer
import akka.stream.alpakka.unixdomainsocket.scaladsl.UnixDomainSocket
import akka.stream.scaladsl.Flow
import akka.util.ByteString
import spray.json.JsValue
import scala.concurrent.Future
object DockerSockMain extends App {
object DockerSockTransport extends ClientTransport {
override def connectTo(host: String, port: Int, settings: ClientConnectionSettings)(implicit system: ActorSystem): Flow[ByteString, ByteString, Future[Http.OutgoingConnection]] = {
// ignore everything for now
UnixDomainSocket().outgoingConnection(new File("/var/run/docker.sock"))
.mapMaterializedValue { _ =>
// Seems that the UnixDomainSocket.OutgoingConnection is never completed? It works anyway if we just assume it is completed
// instantly
Future.successful(Http.OutgoingConnection(InetSocketAddress.createUnresolved(host, port), InetSocketAddress.createUnresolved(host, port)))
}
}
}
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
import system.dispatcher
val settings = ConnectionPoolSettings(system).withTransport(DockerSockTransport)
import SprayJsonSupport._
def handleResponse(response: HttpResponse): Future[String] =
// TODO: create docker json model classes and directly marshal to them
Unmarshal(response).to[JsValue].map(_.prettyPrint)
Http().singleRequest(HttpRequest(uri = "http://localhost/images/json"), settings = settings)
.flatMap(handleResponse)
.onComplete { res =>
println(s"Got result: [$res]")
system.terminate()
}
}
Interesting use-case. You should be able to use Alpakka Unix Domain socket flow and put Akka Http ClientLayer on top of it.
The short answer to the question is "It Can't be Done"—at least not with the existing building blocks of Akka HTTP and the Alkappa Unix Domain Sockets. You would have to handle writing the HTTP GET request by manually sending the headers, i.e. (using the Docker API as an example)
GET /v1.24/containers/json HTTP/1.1\n
Host: localhost\n
\n\n
...and then reading the TCP response manually. Additionally, the Unix Domain Socket logic can't use the Alpakka code because it only currently provides a ServerBinding, and thus is designed to create a server that handles requests to a Unix socket, not to send data to a Unix socket and handle the response.
So everything has to be done manually. There's another StackOverflow question here that points out how the AFUNIXSocket github source code could be used to help with some of the low-level Unix Domain Socket logic that might be of help to others wanting to tackle this same problem.
The most elegant solution would also involve (as suggested by dvim's comment) writing an HTTP.ClientTransport to plug in the Unix Domain Socket communication layer and allow the HTTP library to expose the low-level functionality of writing request/response headers, etc. (One interesting note on that is that the API assumes a host/port parameter pair, which is tightly bound to the TCP paradigm.)
Ich have one assignment and I need a little help. I have infected.pcap and the following task:
Hardcoded IP addresses Sometimes, malware contains hardcoded IP addresses to download their payload or to communicate with their command and control (C&C) server. Find all such communication. Hint: Such IPs have no preceding DNS request.
I need to solve it with Bro script. This was my idea, but unfortunatelly all my connections have no DNS request:
#load base/protocols/dns/main.bro
event file_timeout(f: fa_file)
{
for ( cid in f$conns )
{
if(f$conns[cid]?$dns){
print f$conns[cid]$dns;
print "DNS";
}else {
print "No DNS";
}
}
}
Do you know maybe what is wrong with my code?
I would suggest that you're using the wrong event for this. The file_timeout only occurs if a file transfer was occurring and then stopped without completing. A much more interesting event correlation would be:
Track DNS address lookup responses (I would likely use event
dns_A_reply(c: connection, msg: dns_msg, ans: dns_answer, a:
addr)).
Record the addresses returned in a set; this will provide
you a set of all addresses that were discovered through a DNS query.
Examine outbound requests (where orig_h on the SYN is an internal
address)
Check to see if the address in id$resp_h is in the set of
addresses step 2. If it is, return, if it isn't,
generate a notice since you have an outbound connection attempt with
no corresponding DNS lookup.
I want to request some api and set the response as a nginx variable. But it says "set_by_lua_block" directive is not allowed here. How can I achieve this?
http {
set_by_lua_block $hostIp {
local http = require 'resty.http'
local httpc = http.new()
local res, err = httpc:request_uri('http://some-pai')
local body, err = res:read_body()
ngx.log(ngx.INFO, "Using ngx.INFO")
ngx.log(ngx.INFO, body)
return body
}
...
}
set_by_lua_block is not allowed in http context
https://github.com/openresty/lua-nginx-module#set_by_lua
set_by_lua_* may be used within server context.
But your code will not work anyway because resty.http uses cosocket API.
At least the following API functions are currently disabled within the
context of set_by_lua:
Output API functions (e.g., ngx.say and ngx.send_headers)
Control API functions (e.g., ngx.exit)
Subrequest API functions (e.g., ngx.location.capture and ngx.location.capture_multi)
Cosocket API functions (e.g., ngx.socket.tcp and ngx.req.socket).
Sleeping API function ngx.sleep.
If you really need to request something once before nginx start - write script and set environment variable. Then
set_by_lua $my_var 'return os.getenv("MY_VAR")';