HTMLunit browser automation causes SSL error - ssl

I am writing a simple piece of Java code, to check my external IP and update my DNS entries if it has changed.
I use NetBeans and HTMLunit. The first part of the code works OK.
My code gets the proper External IP address from WhatIsMyIp.com
The second part, which involves logging into my DNS provider webpage (fastname.no) is attached below.
>
> public void LoginToDNS(HtmlPage LogInPage) throws Exception {
> try (final WebClient webClient = new WebClient()) {
>
> //get the webpage data we're looking for
> // also get the buttons and textfields
> final List<?> forms = LogInPage.getForms();
> final HtmlForm form = LogInPage.getFormByName("form1");
> final HtmlTextInput UsernameField = form.getInputByName("username");
> final HtmlPasswordInput PasswordField = form.getInputByName("password");
> final HtmlButton submit_button = form.getButtonByName("submit");
>
> // Set the Username and pasword
> UsernameField.setValueAttribute("myusername");
> PasswordField.setValueAttribute("mypassword");
>
> // submit the form by clicking the button and get back the control panel page
> final HtmlPage DNSPage = submit_button.click();
> CheckDNSEntries(DNSPage);
>
> }
> }
Although I can see that I have logged in properly (I know, because the page title changes to the correct one), I see an SSL error and the resulting webpage is not what I want.
java.util.concurrent.ExecutionException: java.io.IOException: Cannot init SSL
....
Caused by: java.io.IOException: Cannot init SSL
......
at org.eclipse.jetty.websocket.client.io.WebSocketClientSelectorManager.newConnection(WebSocketClientSelectorManager.java:96)
... 3 more
Right when I press the Submit_button, I see a hell of a lot of errors on the debugger coming from htmlunit. (see below)
Apr 12, 2016 10:36:49 AM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[An invalid or illegal selector was specified (selector: '*,:x' error: Invalid selector: :x).] sourceName=[https://www.fastname.no/panel/js/custom/bootstrap.js?bust=?v20160408110428] line=[66] lineSource=[null] lineOffset=[0]
Apr 12, 2016 10:36:49 AM com.gargoylesoftware.htmlunit.javascript.StrictErrorReporter runtimeError
SEVERE: runtimeError: message=[An invalid or illegal selector was specified (selector: '*,:x' error: Invalid selector: :x).] sourceName=[https://www.fastname.no/panel/js/custom/bootstrap.js?bust=?v20160408110428] line=[66] lineSource=[null] lineOffset=[0]
Apr 12, 2016 10:36:51 AM com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl runSingleJob
SEVERE: Job run failed with unexpected RuntimeException: [object Object] (https://www.googletagmanager.com/gtm.js?id=GTM-TWH4RS&bust=?v20160408110428#65)
======= EXCEPTION START ========
Exception class=[net.sourceforge.htmlunit.corejs.javascript.JavaScriptException]
com.gargoylesoftware.htmlunit.ScriptException: [object Object] (https://www.googletagmanager.com/gtm.js?id=GTM-TWH4RS&bust=?v20160408110428#65)
Apr 12, 2016 10:36:52 AM com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl runSingleJob
SEVERE: Job run failed with unexpected RuntimeException: [object Object] (https://www.googletagmanager.com/gtm.js?id=GTM-TWH4RS&bust=?v20160408110428#65)
======= EXCEPTION START ========
Exception class=[net.sourceforge.htmlunit.corejs.javascript.JavaScriptException]
com.gargoylesoftware.htmlunit.ScriptException: [object Object] (https://www.googletagmanager.com/gtm.js?id=GTM-TWH4RS&bust=?v20160408110428#65)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.handleJavaScriptException(JavaScriptEngine.java:982)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:894)
2016-04-12 10:36:55.106:INFO::JS executor for com.gargoylesoftware.htmlunit.WebClient#d1a10ac: Logging initialized #10613ms
Apr 12, 2016 10:36:55 AM com.gargoylesoftware.htmlunit.javascript.host.WebSocket run
SEVERE: WS connect error
java.util.concurrent.ExecutionException: java.io.IOException: Cannot init SSL
at org.eclipse.jetty.util.FuturePromise.get(FuturePromise.java:123)
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket$1.run(WebSocket.java:130)
I am not quite sure whether this is a HTMLunit problem handling JavaScript or it's just my code. Is there a better solution that I can use instead of HTMLunit that will get me where I want to go faster? After all I think the task I want to accomplish is relatively simple.

Try to enable SSL by adding:
webClient.getOptions().setUseInsecureSSL(true);
You can find more documentation here.
EDIT:
In order to hide the HtmlUnit logs, you have to add this:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);

Related

Clicking element once gives error, clicking it twice works fine, why? (Protractor)

I'm running end-to-end tests on a website with Protractor, and one of the elements is having a strange behavior. When I first tried specifying the code to click the element once, I got the following error.
Problematic function:
fooField = element(by.css('#inputFoo'));
foosList = $$('[type=radio]');
async chooseFoo(id: number) {
const foo = this.foosList.get(id - 1);
// await browser.sleep(2000);
// await this.fooField.click();
await this.fooField.click();
await browser.sleep(this.sleepTime);
await foo.click();
await this.selectFooButton.click();
}
Error:
**************************************************
* Failures *
**************************************************
1) reimbursement request page should allow a user to register a request
- Failed: element not interactable
(Session info: chrome=72.0.3626.81)
(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.15.0-45-generic x86_64)
Executed 2 of 2 specs (1 FAILED) in 28 secs.
[16:50:44] I/launcher - 0 instance(s) of WebDriver still running
[16:50:44] I/launcher - chrome #01 failed 1 test(s)
[16:50:44] I/launcher - overall: 1 failed spec(s)
[16:50:44] E/launcher - Process exited with error code 1
An unexpected error occurred: undefined
After changing the code to click on the element twice in a row, and changing nothing else, the code ran successfully. I also tried removing the redundant click and adding a sleep right before the click, but I still got the same error. Why is this happening?
Not sure which element you're trying to click, but this way of clicking has helped me when having similar problems:
browser.executeScript('arguments[0].click()', elementToClick);
let fooField = element(by.css('#inputFoo'));
await browser.wait(ExpectedConditions.elementToBeClickable(fooField), 5000, 'Element is not clickable.');
await fooField.click();
If the element is not clickable after the certain time will throw error, there is a problem with the button. Something is blocking it, to be clicked.
Another way to bypass it (not the best way) is using javascript click.
let fooField = element(by.css('#inputFoo'));
browser.executeScript('arguments[0].scrollIntoView(true); arguments[0].click();', fooField);

TestCafe - Test fails onGetBposShellInfoNavBarData

I am running TestCafe version 0.22.0 on Win10. I am not sure if my issue has something with the Win10 or not but unfortunately, I donĀ“t have another computer to test on. The issue is that my tests fail with error message "GetBposShellInfoNavBarData failed: SyntaxError: Unexpected end of JSON input"
Here is the simple code I use:
Error on page "https://outlook.live.com/mail/inbox": GetBposShellInfoNavBarData failed: SyntaxError: Unexpected end of JSON input
Browser: Chrome 69.0.3497 / Windows 10.0.0
await t.click('body > section > div > div > nav > div > div > div > a');
await t.typeText('#i0116', login);
await t.click('#idSIButton9');
await t.typeText('#i0118', password);
await t.click('#idSIButton9');
await t.maximizeWindow();
Test fails on this t.maximizeWindow(). It also fails with same error on next "t.click(Selector...) if I comment out t.maximizeWindow() line.
As #ioseph properly mentioned, this error occurs on your web page and is not related to TestCafe.
I recommend that you address this error on your website and run your tests again.
Alternatively, you can use the --skip-js-errors argument to ignore such errors.

Unexpected server response (0) while retrieving pdf

We are specifically getting this error when using Amazon ec2 instance. Configuration on aws instance is Tomcat 7, Ubuntu 16.04 and memory is 8gb. It occurs when the user tries to view pdf file. In our application, we are having one functionality where the user can only view PDF file onto browser, but won't be able to download it. PDF file is on the same server. We are using cors minimal configuration. We have tried it locally with Ubuntu and it is working fine.
Code snippet:
var fileSplitContent = fileName.split(".");
if ($('#viewImageOnlyForm')[0] != undefined && $('#viewPdfOnlyForm')[0] != undefined) {
if (fileSplitContent[fileSplitContent.length - 1].toLowerCase() != "pdf") {
$('#imageSource').val(requestURL + $.param(inputData));
$('#viewImageOnlyForm').submit();
} else {
var requestURL = "rest/file/getCapitalRaiseFile?";
$('#pdFSource').val(requestURL + $.param(inputData));
$('#viewPdfOnlyForm').submit();
}
} else {
// pop up download attachment dialog box
downloadIFrame.attr("src", requestURL + $.param(inputData));
}
}
Jan 04, 2017 5:07:31 AM org.glassfish.jersey.server.ServerRuntime$Responder writeResponse
SEVERE: An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.apache.catalina.connector.ClientAbortException: java.net.SocketException: Connection reset
Caused by: org.apache.catalina.connector.ClientAbortException: java.net.SocketException: Broken pipe (Write failed)
Depending where you access the document this can be because of you have a download manager installed on your browser. This sometimes causes problems - maybe take a look at your extensions. You should try by disabling downloader manager extension in your browser.

How to use spiderable with a meteor app hosted on modulus.io

I'm trying to make spiderable works on my meteor app hosted on modulus with SSL.
I have Meteor 1.0, iron:router 1.0, spiderable and node package of phantomjs
All is working on localhost. But once I deploy on Modulus, first I had the error
spiderable: phantomjs failed: null
Then, I added the following environment variable in the modulus panel:
METEOR_PKG_SPIDERABLE_PHANTOMJS_ARGS = --ssl-protocol=tlsv1 --ignore-ssl-errors=yes --debug=true
This is still not working and the debug is outputting multiple times (like it's looping over an error) the following message in the modulus console:
2014-12-03T17:01:00 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {
if (typeof Meteor === 'undefined'
|| Meteor.status === undefined
|| !Meteor.status().connected) {
return false;
}
if (typeof Package === 'undefined'
|| Package.spiderable === undefined
|| Package.spiderable.Spiderable === undefined
|| !Package.spiderable.Spiderable._initialSubscriptionsStarted) {
return false;
}
Tracker.flush();
return DDP._allSubscriptionsReady();
})(); })()"
2014-12-03T17:01:00 [DEBUG] WebPage - evaluateJavaScript result QVariant(bool, false)
If anyone knows how to solve this or succeeded to deploy a meteor project on modulus.io with SSL and spiderable. Let's me know the good way to do it :)
Thank a lot !
I solved my problem as follows:
I installed phantomjs locally and run the test script available at http://www.meteorpedia.com/read/spiderable/
phantomjs phantomtest.js
This gave me more details about the error: Parse Error.
Then, it was a javascript file that once compiled/minified, rendered an error caused by select2. The js library that was using it was flat-ui.js (http://designmodo.github.io/Flat-UI/).
I discover this by testing many deploys on *.meteor.com and by adding/removing .js file.
I edit the flat-ui.js library to avoid Parsing Error.
I redeployed on both modulus.io and *.meteor.com. All was working fine on *.meteor.com but still didn't work on modulus.io. That let me thinking about an SSL error but I only saw "spiderable: phantomjs failed: null" in the modulus.io logs.
I add the following environment variable in the modulus panel:
METEOR_PKG_SPIDERABLE_PHANTOMJS_ARGS = --debug=true
and it appears that it was a "SSL Handshake error":
[DEBUG] Network - Resource request error: 6 ( "SSL handshake failed" )
I add another option to the METEOR_PKG_SPIDERABLE_PHANTOMJS_ARGS environment variable:
METEOR_PKG_SPIDERABLE_PHANTOMJS_ARGS = --ignore-ssl-errors=yes --debug=true
Now everything is working fine on modulus.io.
To sumup:
solve javascript errors
add METEOR_PKG_SPIDERABLE_PHANTOMJS_ARGS = --ignore-ssl-errors=yes
I hope this will help some dudes,

Selenium-rc window causes permission denied message in IE

I am testing with Selenium-rc 1.0.3 and I am getting a Permission denied error message in IE when I run my IDE script from the command line.
I am trying to run an IDE script in Internet explorer using the selenium control RC 1.0.3
from the command line:
java -jar selenium-server.jar -htmlsuite "*iexploreproxy" "url
address/where" "C:\Users\sat\Documents\selenium\suite.html"
"C:\Users\sat\Documents\selenium scripts\results.htm" at this point
The IE window pops up saying as below
I get a security warning saying "Do you want to view only the webpage content that was delivered securely?" I hit Yes and I see this error in the test runner window:
Webpage error details
Message: Access is denied.
Line: 177
Char: 9
Code: 0
URI: xx.xx.xx.xxx/selenium-server/core/scripts/selenium-testrunner.js
UPDATE:
I looked at the line 177 and char :9 in the script and it points to
var runInterval = 0;
/** SeleniumFrame encapsulates an iframe element */
var SeleniumFrame = classCreate();
objectExtend(SeleniumFrame.prototype, {
initialize : function(frame) {
this.frame = frame;
addLoadListener(this.frame, fnBind(this._handleLoad, this));
},
getWindow : function() {
return this.frame.contentWindow;
},
getDocument : function() {
return this.frame.contentWindow.document; - line 177 char 9
},
_handleLoad: function() {
this._attachStylesheet();
this._onLoad();
if (this.loadCallback) {
this.loadCallback();
}
Do you know what the error is about? Why do I get that? I see my test cases and everything in the test runner window, but I can't run them in the IE browser. I searched the web with no avail.
I do not have much experience with running the test from CLI, but have you tried starting Selenium RC with administrator permission?
Any particular reason for not using the new Selenium 2 IWebDriver and a test framework?
The error might be caused by the use of iframe/frameset and IE's security settings. Default settings are that, if a site uses iframe/frameset and the frame content originates from 2 different root domains, then this is a security risk. Try is add the sites to your list thrusted site for IE. Have you tried to ude the firefox driver instead (will not have this security restriction).