how to set in basic-regular controller not TestController which is extended from TestCase, set to input field on Web text value?
public function automatical_login_with_codes()
{
$browser = new HttpBrowser(HttpClient::create());
$client = Client::createChromeClient();
$jsLink = "document.querySelector('#ctl00_uxAuthenticationBlock_uxOpenLogin').click()";
$crawler = $client->request('GET', 'https://www.best.com/CategoryList.aspx?');
$client->executeScript($jsLink);
$client->waitFor('#ctl00_uxAuthenticationBlock_uxLoginText');// $this->assertSame
$crawler->filter('#ctl00_uxAuthenticationBlock_uxLoginText')->text('USERNAME'); <<-- NOT SET :(((
$check = $crawler->filter('#ctl00_uxAuthenticationBlock_uxLoginText')->text();
dd($check); //<- result nothing
Answer is very simple (I simulate typing in browser input field value - USERNAME)
$crawler->filter('#ctl00_uxAuthenticationBlock_uxLoginText')->sendKeys('USERNAME');
Hope it save your time :)
Another way to do it:
$client->findElement(WebDriverBy::cssSelector('#ctl00_uxAuthenticationBlock_uxLoginText'))->sendKeys('USERNAME');
Related
I am trying to scrape data from this link https://www.flatstats.co.uk/racing-system-builder.php using scrapy.
I want to automate the ajax call using scrapy.
When I click "Full SP" Button (inspect in Firebug) the post parameter has the sql string which is "strange"
race|2|eq|Ordinary|0|~tRIDER_TYPE
What dialect is this?
My code :
import scrapy
import urllib
class FlatStat(scrapy.Spider):
name= "flatstat"
allowed_domains = ["flatstats.co.uk"]
start_urls = ["https://www.flatstats.co.uk/racing-system-builder.php"]
def parse(self, response):
query_lst = response.xpath('//table[#id="system"]//tr/td[last()]/text()').extract()
query_str = ' '.join(query_lst)
url = 'https://www.flatstats.co.uk/ajax/sb_report.php'
body_dict = {'a_e_max': '9.99',
'a_e_min': '0',
'arch_min': '0',
'exp_min': '0',
'report_type':'S',
# copied from the Post parameters by inspecting. Actually I tried everything.
'sqlFullString' : u'''Type%20(Rider)%7C%3D%7COrdinary%20(Exclude%20Amatr%2C%20App%2C%20Lady%20Races
)%7CAND%7Crace%7C2%7C0%7COrdinary%7C0%7C~tRIDER_TYPE%7C-t%7Ceq''',
#I tried copying this from the post parameters as well but no success.
#I also tried sql from the table //td text() which is "normal" sql but no success
'sqlString': query_str}
#here i tried everything FormRequest as well though there is no form.
return scrapy.Request(url, method="POST", body=urllib.urlencode(body_dict), callback=self.parse_page)
def parse_page(self, response):
with open("response.html", "w") as f:
f.write(response.body)
So questions are:
What is this sql.
Why isn't it returning me the required page. How can I run the right query?
I tried Selenium as well to click the button and let it do the stuff it self but that is another unsuccessful story. :(
It's not easy to say what the website creator is doing with the submitted sqlString. It probably means something very specific to how the data is processed by their backend.
This is an extract of the page JavaScript in-HTML code:
...
function system_report(type) {
sqlString = '', sqlFullString = '', rowcount = 0;
$('#system tr').each(function() {
if(rowcount > 0) {
var editdata = this.cells[6].innerHTML.split("|");
sqlString += editdata[0] + '|' + editdata[1] + '|' + editdata[7] + '|' + editdata[3] + '|' + editdata[4] + '|' + editdata[5] + '^';
sqlFullString += this.cells[0].innerHTML + '|' + encodeURIComponent(this.cells[1].innerHTML) + '|' + this.cells[2].innerHTML + '|' + this.cells[3].innerHTML + '|' + this.cells[6].innerHTML + '^';
}
rowcount++;
});
sqlString = sqlString.slice(0, -1)
...
Looks non trivial to reverse-engineer.
Although it's not a solution to your "sql" question above, I suggest that you try using splash (an alternative to selenium in some cases).
You can launch it with docker (the easiest way):
$ sudo docker run -p 5023:5023 -p 8050:8050 -p 8051:8051 scrapinghub/splash
With the following script:
function main(splash)
local url = splash.args.url
assert(splash:go(url))
assert(splash:wait(0.5))
-- this clicks the "Full SP" button
assert(splash:runjs("$('#b-full-report').click()"))
-- loading the report takes some time
assert(splash:wait(5))
return {
html = splash:html()
}
end
you can get the page HTML with the popup of the report.
You can integrate Splash with Scrapy using scrapyjs (a.k.a scrapy-splash)
See https://stackoverflow.com/a/35851072/ with an example how to do so with a custom script.
How to get window Username in silverlight page.
I try with below link but its give blank.
http://rouslan.com/2009/03/12/20-steps-to-get-together-windows-authentication-silverlight-and-wcf-service/
http://blogs.msdn.com/b/keithmg/archive/2010/12/11/silverlight-get-user-or-windows-credentials.aspx
Get current Windows user name within Silverlight
http://www.codeproject.com/Articles/47520/Silverlight-Windows-User-Identity-Name
Any help will be highly Appreciated
Thanks in Advance,
Hitesh
Step 1 : Add below code in Page_Load() of default.aspx page
String WindowsUserName;
System.Security.Principal.IPrincipal User;
User = System.Web.HttpContext.Current.User;
WindowsUserName = User.Identity.Name;
Char[] Spliter = { '\\' };
String[] user = new string[2];
user = WindowsUserName.Split(Spliter);
//WindowsUserName = user[1]; //" + WindowsUserName;
InitParams += ",WindowsUserName=" + WindowsUserName;//",WindowsUserName=ram";
Step 2 : Add below code in Application_Startup() of App.xaml page in silverlight application
string msWindowsUserName = e.InitParams["WindowsUserName"];
hello
i have a simple file upload with just one extra post parameter which is imgtitle.
it's used to rename the image after it's been transferred to the upload location on the server.
PHP:
<?php
$imgtitle = $_POST['title'];
$current_image=$_FILES['file']['name'];
$extension = substr(strrchr($current_image, '.'), 1);
$new_image = $imgtitle. "." . $extension;
$destination="uploads/".$new_image;
$action = copy($_FILES['file']['tmp_name'], $destination);
echo "--".$imgtitle."--";
?>
the upload works fine here.
http://tnsatchat.heliohost.org/tnsatchat/upload/upload.html
/upload/uploads/abcd.jpg
what i want to do is make a simple vb.net app which would allow me to upload the file/image
i'm using WebClient to do so.
but i'm having some difficulties passing the imgtitle parameter.
here's my code
Dim ur = New Uri("http://tnsatchat.heliohost.org/tnsatchat/upload/upload.php?title=testname")
wc.UploadFileAsync(ur, "POST", "C:\testimage.jpg")
the upload is working i.e the file is there .. but with an empty name !!
/upload/uploads/.jpg
".jpg" is supposed to be "testname.jpg" but i guess the PHP is not getting that parameter.
any hints please !
edit:
i used the QueryString property
Dim ur = New Uri("http://tnsatchat.heliohost.org/tnsatchat/upload/upload.php")
Dim col = New Collections.Specialized.NameValueCollection
col.Add("title", "testname")
wc.QueryString = col
wc.UploadFileAsync(ur, "POST", "C:\testimage.jpg")
but with no better luck !!
I have an application that uses a flex form to capture user input. When the user has entered the form data (which includes a drawing area) the application creates a jpg image of the form and sends back to the server. Since the data is sensitive, it has to use https. Also, the client requires both jpg and pdf versions of the form to be stored on the server.
The application sends data back in three steps
1 - send the jpg snapshot with ordernumber
2 - send the form data fields as post data so it is not visible in the address bar
3 - send the pdf data
I am sending the jpg data first using urlloader and waiting for the server to respond before performing opperation 2 and 3 to ensure that the server has created the record associated with the new orderNumber.
This code works fine in IE over http. But If I try to use the application over https, IE blocks the page response from store jpg step and the complete event of the urlloader never fires. The application works fine in FireFox over http or https.
Here is the crossdomain.xml (I have replaced the domain with ""):
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*.<mydomain>.com" to-ports="*" secure="false"/>
<allow-http-request-headers-from domain="*.<mydomain>.com" headers="*">
</cross-domain-policy>
Here is the code that is executed when the user presses the submit button:
private function loaderCompleteHandler(event:Event):void {
sendPDF();
sendPatientData();
}
private function submitOrder(pEvt:MouseEvent):void
{
//disable submit form so the order can't be submitted twice
formIsValid = false;
waitVisible = true;
//submit the jpg image first with the order number, userID, provID
//and order type. The receiveing asp will create the new order record
//and save the jpg file. jpg MUST be sent first.
orderNum = userID + "." + provID + "." + Date().toString() + "." + orderType;
var jpgURL:String = "https://orders.mydomain.com/orderSubmit.asp?sub=jpg&userID=" + userID + "&provID=" + provID + "&oNum=" + orderNum + "&oType=" + orderType;
var jpgSource:BitmapData = new BitmapData (vbxPrint.width, vbxPrint.height);
jpgSource.draw(vbxPrint);
var jpgEncoder:JPEGEncoder = new JPEGEncoder(100);
var jpgStream:ByteArray = jpgEncoder.encode(jpgSource);
var header:URLRequestHeader = new URLRequestHeader ("content-type", "application/octet-stream");
//Make sure to use the correct path to jpg_encoder_download.php
var jpgURLRequest:URLRequest = new URLRequest (jpgURL);
jpgURLRequest.requestHeaders.push(header);
jpgURLRequest.method = URLRequestMethod.POST;
jpgURLRequest.data = jpgStream;
//navigateToURL(jpgURLRequest, "_blank");
var jpgURLLoader:URLLoader = new URLLoader();
try
{
jpgURLLoader.load(jpgURLRequest);
}
catch (error:ArgumentError)
{
trace("An ArgumentError has occurred.");
}
catch (error:SecurityError)
{
trace("A SecurityError has occurred.");
}
jpgURLLoader.addEventListener(Event.COMPLETE, loaderCompleteHandler);
}
private function sendPatientData ():void
{
var dataURL:String = "https://orders.mydomain.com/orderSubmit.asp?sub=data&oNum=" + orderNum + "&oType=" + orderType;
//Make sure to use the correct path to jpg_encoder_download.php
var dataURLRequest:URLRequest = new URLRequest (dataURL);
dataURLRequest.method = URLRequestMethod.POST;
var dataUrlVariables:URLVariables = new URLVariables();
dataUrlVariables.userID = userID
dataUrlVariables.provID = provID
dataUrlVariables.name = txtPatientName.text
dataUrlVariables.dob = txtDOB.text
dataUrlVariables.contact = txtPatientContact.text
dataUrlVariables.sex=txtSex.text
dataUrlVariables.ind=txtIndications.text
dataURLRequest.data = dataUrlVariables
navigateToURL(dataURLRequest, "_self");
}
private function sendPDF():void
{
var url:String = "https://orders.mydomain.com/pdfOrderForm.asp"
var fileName:String = "orderPDF.pdf&sub=pdf&oNum=" + orderNum + "&oType=" + orderType + "&f=2&t=1" + "&mid=" + ModuleID.toString()
var jpgSource:BitmapData = new BitmapData (vbxPrint.width, vbxPrint.height);
jpgSource.draw(vbxPrint);
var jpgEncoder:JPEGEncoder = new JPEGEncoder(100);
var jpgStream:ByteArray = jpgEncoder.encode(jpgSource);
myPDF = new PDF( Orientation.LANDSCAPE,Unit.INCHES,Size.LETTER);
myPDF.addPage();
myPDF.addImageStream(jpgStream,0,0, 0, 0, 1,ResizeMode.FIT_TO_PAGE );
myPDF.save(Method.REMOTE,url,Download.ATTACHMENT,fileName);
}
The target asp page is not sending back any data, except the basic site page template.
Can anyone help me figure out how to get around this IE crossdomain issue? I have turned off the XSS filter in IE tools security settings, but that still didn't solve the problem.
THANKS
Do everything over https. Load the swf from an https url. Send the initial form post via https. Send the images via https.
What is the most convenient way using Selenium WebDriver to check if an URL GET returns successfully (HTTP 200)?
In this particular case I'm most interested in verifying that no images of the current page are broken.
Try this:
List<WebElement> allImages = driver.findElements(By.tagName("img"));
for (WebElement image : allImages) {
boolean loaded = ((JavaScriptExecutor) driver).executeScript(
"return arguments[0].complete", image);
if (!loaded) {
// Your error handling here.
}
}
You could use the getEval command to verify the value returned from the following JavaScript for each image on the page.
#Test
public void checkForBrokenImages() {
selenium.open("http://www.example.com/");
int imageCount = selenium.getXpathCount("//img").intValue();
for (int i = 0; i < imageCount; i++) {
String currentImage = "this.browserbot.getUserWindow().document.images[" + i + "]";
assertEquals(selenium.getEval("(!" + currentImage + ".complete) ? false : !(typeof " + currentImage + ".naturalWidth != \"undefined\" && " + currentImage + ".naturalWidth == 0);"), "true", "Broken image: " + selenium.getEval(currentImage + ".src"));
}
}
Updated:
Added tested TestNG/Java example.
I don't think that first response will work. When I src a misnamed image, it throws a 404 error as expected. However, when I check the javascript in firebug, that (broken) image has .complete set to true. So, it was a completed 404, but still a broken image.
The second response seems to be more accurate in that it checks that it's complete and then checks that there is some width to the image.
I made a python version of the second response that works for me. Could be cleaned up a bit, but hopefully it will help.
def checkForBrokenImages(self):
sel = self.selenium
imgCount = int(sel.get_xpath_count("//img"))
for i in range(0,imgCount):
isComplete = sel.get_eval("selenium.browserbot.getCurrentWindow().document.images[" + str(i) + "].complete")
self.assertTrue(isComplete, "Bad Img (!complete): "+sel.get_eval("selenium.browserbot.getCurrentWindow().document.images[" + str(i) + "].src"))
typeOf = sel.get_eval("typeof selenium.browserbot.getCurrentWindow().document.images[" + str(i) + "].naturalWidth")
self.assertTrue(typeOf != 'undefined', "Bad Img (w=undef): "+sel.get_eval("selenium.browserbot.getCurrentWindow().document.images[" + str(i) + "].src"))
natWidth = int(sel.get_eval("selenium.browserbot.getCurrentWindow().document.images[" + str(i) + "].naturalWidth"))
self.assertTrue(natWidth > 0, "Bad Img (w=0): "+sel.get_eval("selenium.browserbot.getCurrentWindow().document.images[" + str(i) + "].src"))
Instead of traversing in Java, it may be faster to call javascript only once for all images.
boolean allImgLoaded = (Boolean)((JavascriptExecutor) driver).executeScript(
"return Array.prototype.slice.call(document.images).every("
+ "function (img) {return img.complete && img.naturalWidth > 0;});");
One of the alternative solutions is analyzing web server logs after test executing. This approach allows to catch not only missed images, but css, scripts and other resources.
Description of how to do it is here.
Funda for Checking 404:
Basically 404s can be checked via HTTP Response of the URL.
Step 1: Import the Library for HTTPTestAPI
Step 2: Create the HTTPRequest.
String URL="www.abc.com";
HTTPRequest request = new HTTPRequest(URL);
//Get the response code of the URL
int response_code = request.getResponseCode();
//Check for 404:
if(response_code == 404)
FAIL -- THE URL is leading to 404.
else
PASS