Problem Description
- We are having problems with a JAX-WS Webservice that wants to connect to
a server using HTTPS in combination with a proxy server.
The setups is as follows:
- WebSphere 6.0.1.47 running on AIX Version: 5300-10-07-1119
- A JAX-WS Webservice application
What happens is as follows:
JAX-WS application in WAS tries to connect to
'https://target.example.domain/url' while using a proxy server
- When the transport chain is started, the following error appears (i have
included the corresponding ffdc's as attachments to this mail) :
java.io.IOException: Async IO operation failed, reason: RC: 76 A socket
must be already connected.;
When we:
1) Use a HTTP destination and DO NOT use a Proxy Server then the
application works
2) Use a HTTPS destination and DO NOT use a Proxy Server then the
application works
3) Use a HTTP destination and USE a Proxy Server then the
application works
4) Use a HTTPS destination and USE a Proxy Server then the application
displays the error described above.
ffdc logs
" ------Start of DE processing------ = [1/14/15 13:04:39:913 CET] , key = java.io.IOException com.ibm.ws.websvcs.transport.http.HTTPConnection.connect 213
Exception = java.io.IOException
Source = com.ibm.ws.websvcs.transport.http.HTTPConnection.connect
probeid = 213
Stack Dump = java.io.IOException: Async IO operation failed, reason: RC: 76 A socket must be already connected.
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:679)
at com.ibm.io.async.ResultHandler$CompletionProcessingRunnable.run(ResultHandler.ja va:910)
at java.lang.Thread.run(Thread.java:813)
Dump of callerThis =
Object type = com.ibm.ws.websvcs.transport.http.HTTPConnection
com.ibm.ws.websvcs.transport.http.HTTPConnection#db30db3
Exception = java.io.IOException
Source = com.ibm.ws.websvcs.transport.http.HTTPConnection.connect
probeid = 213
Dump of callerThis =
Object type = com.ibm.ws.websvcs.transport.http.HTTPConnection
_tc =
defaultMessageFile = com.ibm.ejs.resources.seriousMessages
EXTENSION_NAME_DPID = DiagnosticProvider
ivDumpEnabled = false
ivResourceBundleName = com.ibm.ws.websvcs.resources.websvcsMessages
ivLogger = null
ivDiagnosticProviderID = null
anyTracingEnabled = true
ivLevel = 1
ivName = com.ibm.ws.websvcs.transport.http.HTTPConnection
ivDebugEnabled = true
ivEventEnabled = true
ivEntryEnabled = true
ivDetailEnabled = true
ivConfigEnabled = true
ivInfoEnabled = true
ivServiceEnabled = true
ivWarningEnabled = true
ivErrorEnabled = true
ivFatalEnabled = true
chainname = HttpsOutboundChain:xx-proxy- xxxxx.xxx.xxxx.com:8080:1665256594:10.21.197.161:9443
............."
We have tried setting the properties (https.proxyHost, https.proxyPort) at System level and also in the SOAP header, nothing works.
We are using BindingProv
Any help is much appreciated
Related
In my centos7 server, I have set up Telegraf and InfluxDB. InfluxDB successfully receives data from Telegraf and stores them in the database. But when I reconfigure both services to use https, I see the following error in Telegraf's logs
Dec 29 15:13:11 localhost.localdomain telegraf[31779]: 2020-12-29T13:13:11Z E! [outputs.influxdb] When writing to [https://127.0.0.1:8086]: Post "https://127.0.0.1:8086/write?db=GRAFANA": dial tcp 127.0.0.1:8086: connect: connection refused
Dec 29 15:13:11 localhost.localdomain telegraf[31779]: 2020-12-29T13:13:11Z E! [agent] Error writing to outputs.influxdb: could not write any address
InfluxDB doesn't show any errors in it's logs.
Below is my telegraf.conf file:
[agent]
hostname = "local"
flush_interval = "15s"
interval = "15s"
# Input Plugins
[[inputs.cpu]]
percpu = true
totalcpu = true
collect_cpu_time = false
report_active = false
[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs", "devfs"]
[[inputs.io]]
[[inputs.mem]]
[[inputs.net]]
[[inputs.system]]
[[inputs.swap]]
[[inputs.netstat]]
[[inputs.processes]]
[[inputs.kernel]]
# Output Plugin InfluxDB
[[outputs.influxdb]]
database = "GRAFANA"
urls = [ "https://127.0.0.1:8086" ]
insecure_skip_verify = true
username = "telegrafuser"
password = "metricsmetricsmetricsmetrics"
And this is the uncommented [http] section of the influxdb.conf
# Determines whether HTTP endpoint is enabled.
enabled = false
# Determines whether the Flux query endpoint is enabled.
flux-enabled = true
# The bind address used by the HTTP service.
bind-address = ":8086"
# Determines whether user authentication is enabled over HTTP/HTTPS.
auth-enabled = false
# Determines whether HTTPS is enabled.
https-enabled = true
# The SSL certificate to use when HTTPS is enabled.
https-certificate = "/etc/ssl/server-cert.pem"
# Use a separate private key location.
https-private-key = "/etc/ssl/server-key.pem"
I am trying to create a function that can call REST with the http socket lua.
And I tried to set the timeout this way. But, when I run this function, the timeout is not running. How should I set the timeout?
local http = require "socket.http"
local socket = require "socket"
local respbody = {}
http.request {
method = req_method,
url = req_url,
source = ltn12.source.string(req_body),
headers =
{
["Content-Type"] = req_content_type,
["content-length"] = string.len(req_body),
["Host"] = host,
},
sink = ltn12.sink.table(respbody),
create = function()
local req_sock = socket.tcp()
req_sock:settimeout(3, 't')
return req_sock
end,
}
You may want to check lua-http. I use it to call REST and works like a charm. I am not an expert but, as far as I can tell, it is a good LUA http implementation.
You can set a two seconds timeout as simple as:
local http_client = require "http.client"
local myconnection = http_client.connect {
host = "myrestserver.domain.com";
timeout = 2;
}
Full documentation in here.
if I implement the example with my requirements, will it be like this? cmiiw
local http_client = require "http.client"
local req_body = "key1=value1&key2=value2"
local myconnection = http_client.connect {
method = "POST";
url = "myrestserver.domain.com/api/example";
host = "myrestserver.domain.com";
source = req_body
headers = {
["Content-Type"] = "application/x-www-form-urlencoded",
["content-length"] = string.len(req_body),
},
timeout = 2;
}
LuaSocket implicitly set http.TIMEOUT to the socket object.
Also you have to remember that socket timeout is not the same as request timeout.
Socket timeout means timeout for each operation independently. For simple case you can wait connection up to timeout seconds and then each read operation can take up to timeout seconds. And because of HTTP client read response line by line you get timeout seconds for each header plus for each body chunk. Also, there may be redirecions where each redirection is a separate HTTP request/response. If you use TLS there also will be hendshake after connection which also took several send/receive operation.
I did not use lua-http module and do not know how timeout implemented there.
But I prefer use modules like cURL if I really need to restrict request timeout.
Image of the console error
mobile apps are successfully connected using mfp server IP Address with port 9080 but using instead of IP with Domain the mfp8 server response error msg "The Request time out" and response text "undefined"
Using IP Address: mfpclient properties file:
wlServerProtocol = http
wlServerHost = **.**.**.78
wlServerPort = 9080
wlServerContext = /mfp/
testWebResourcesChecksum = false
ignoredFileExtensions = png, jpg, jpeg, gif, mp4, mp3
wlPlatformVersion = 8.0.0.00-20190910-142437
wlSecureDirectUpdatePublicKey =
languagePreferences = en
wlBuildId = 8.0.0.00-20190910-142437
Using Domain: mfpclient properties file:
wlServerProtocol = https
wlServerHost = www.domainname.com
wlServerPort = 443
wlServerContext = /mfp/
testWebResourcesChecksum = false
ignoredFileExtensions = png, jpg, jpeg, gif, mp4, mp3
wlPlatformVersion = 8.0.0.00-20190910-142437
wlSecureDirectUpdatePublicKey =
languagePreferences = en
wlBuildId = 8.0.0.00-20190910-142437
the output whenever im using domain is request timeout error
Please update with which MobileFirst API request is timing out.
REQUEST_TIMEOUT error will come if
i.If the server is not accessible on the IP address/port specified in mfpclient.properties file.
ii.When timeout is set in WLResourceRequest and there is a delay in adapter response that is more than timeout value.
iii.Having said that, the request timeout error occurs when a request made by the device does not get a response from the MobileFirst server within the stipulated timeout period. For OAuth calls, this timeout is 10 seconds.
iv. Server is taking more time to respond. Check your backend logic.
v. Possibilties that DNS resolution is not happening within 10 sec.
I've SSL proxy server and I want to scrape https site. I mean the connection between scrapy and the proxy is encrypted then the proxy will open a connection to the website.
after some debugging I found the following:-
currently scrapy handle the situation as follows:-
if the site is http it use ScrapyProxyAgent which send client hello then send a connect request for the website to the proxy
but if the site is https
it use a TunnelingAgent which does not send client hello to the proxy and hence the connection is terminated.
What I need is to tell scrapy to first establish a connection via ScrapyProxyAgent then use a TunnelingAgent not sure how to do that.
I tried to create a https DOWNLOAD_HANDLERS but I'm not that expert
class MyHTTPDownloader(HTTP11DownloadHandler):
def download_request(self, request, spider):
"""Return a deferred for the HTTP download"""
timeout = request.meta.get('download_timeout') or self._connectTimeout
bindaddress = request.meta.get('bindaddress')
proxy = request.meta.get('proxy')
agent = ScrapyProxyAgent(reactor,proxyURI=to_bytes(proxy, encoding='ascii'),
connectTimeout=timeout, bindAddress=bindaddress, pool=self._pool)
_, _, proxyHost, proxyPort, proxyParams = _parse(proxy)
proxyHost = to_unicode(proxyHost)
url = urldefrag(request.url)[0]
method = to_bytes(request.method)
headers = TxHeaders(request.headers)
omitConnectTunnel = b'noconnect' in proxyParams
proxyConf = (proxyHost, proxyPort,
request.headers.get(b'Proxy-Authorization', None))
if request.body:
bodyproducer = _RequestBodyProducer(request.body)
if request.body:
bodyproducer = _RequestBodyProducer(request.body)
elif method == b'POST':
bodyproducer = _RequestBodyProducer(b'')
else:
bodyproducer = None
start_time = time()
tunnelingAgent = TunnelingAgent(reactor, proxyConf,
contextFactory=self._contextFactory, connectTimeout=timeout,
bindAddress=bindaddress, pool=self._pool)
agent.request(method, to_bytes(url, encoding='ascii'), headers, bodyproducer)
I need to establish a tunnel after the proxy agent is connected.
is that even possible?
thanks in advance
I am trying to do the following:
$basicHttpBinding = New-Object System.ServiceModel.BasicHttpBinding([System.ServiceModel.BasicHttpSecurityMode]::TransportCredentialOnly);
$basicHttpBinding.Security.Transport.ClientCredentialType = [System.ServiceModel.HttpClientCredentialType]::Windows;
$basicHttpBinding.MaxBufferSize = 65536000;
$basicHttpBinding.MaxReceivedMessageSize = 65536000;
$basicHttpBinding.MaxBufferPoolSize = 524288;
$basicHttpBinding.CloseTimeout = "01:01:00";
$basicHttpBinding.OpenTimeout = "01:01:00";
$basicHttpBinding.ReceiveTimeout = "01:01:00";
$basicHttpBinding.SendTimeout = "01:01:00";
$mexClient = New-Object System.ServiceModel.Description.MetadataExchangeClient($basicHttpBinding);
$mexClient.MaximumResolvedReferences = [System.Int32]::MaxValue
$metadataSet = $mexClient.GetMetadata([Uri]$WsdlUrl, $mode)
It appears that for whatever reason, the MexClient is not passing the credentials. I say this because when I browse the $WsdlUrl in IE running as the same user as the script, I can see the wsdl just fine. When I run the code above, I get an exception:
Exception calling "GetMetadata" with "2" argument(s): "Metadata contains a reference that
cannot be resolved: 'http_://myServer:myPort/myVirtualDirectory/myWcfService.svc?wsdl'." ---> System.InvalidOperationException: Metadata contains a
reference that cannot be resolved: 'http_://myServer:myPort/myVirtualDirectory/myWcfService.svc?wsdl'. ---> System.Net.WebException: The remote
server returned an error: (401) Unauthorized.
Edit 01
It also appears the the MetadataExchangeClient is not transmitting the request through http. When I use fiddler there is not 'request' being sent, only an HTTP response of the failure. How can I get the MexClient to send and receive via http?
I was missing:
$mexClient.HttpCredentials = [System.Net.CredentialCache]::DefaultNetworkCredentials;
Before my GetMetaData call. I thought the binding requirements were enough to clue in the MexClient that it should pass the default credentials, but that kind of intelligence was too optimistic. Hope this helps someone.