How to change Awesomium user agent? - vb.net

I have a simple form with an Awesomium webcontrol going to a website.
I found how to change the useragent in C++ but nothing for vb.net:
code for C++ http://wiki.awesomium.com/general-use/changing-the-user-agent.html
Could someone help me with this?

I realize this is an old thread and you've probably found the answer, but I added this code to my WPF VB.net app and it worked.
Dim config As New Awesomium.Core.WebConfig()
If Not Awesomium.Core.WebCore.IsInitialized Then
Awesomium.Core.WebCore.Initialize(New Awesomium.Core.WebConfig() With { _
.HomeURL = New Uri("https://www.awesomium.com"), _
.RemoteDebuggingPort = 2229, _
.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
})
End If
It didn't fix the problem I was trying to fix, however it did make it so Chrome doesn't report that I'm using an old version.

I note your answer would be here if the documentation was up to date :-(
The best we can do is see what they've said for "WPF WebControl Initialization" here, and in summary I believe what I said in my comment holds:
You need to use WebCore.Initialize(config As WebConfig).
Note also the commentary about when things occur.

Related

Scrapy - Javascript rendering

I would like to get some data from here:
https://www.drivy.com/location-voiture/liege/mitsubishi-colt-359699?address=Gare+de+Li%C3%A8ge-Guillemins&city_display_name=&country_scope=BE&distance=200&end_date=2019-05-27&end_time=06%3A00&latitude=50.6251&longitude=5.5659&start_date=2019-05-26&start_time=06%3A00
I'm searching for the ID of the owner of the car. This ID is within the aattribute of class car_owner_section. For the page above it is the numbers in the hrefattribute like this "/users/1228276". The issue is that this link is apparently rendered by javascript and I absolutely want to avoid scrapy-splash. Does anyone has an idea on how to find this ID ? It should be somewhere on a JSON I guess but I've searched for days now and found nothing.
I tested it on scrapy shell, and the response returns the link you are looking for, without using splash. You might want to check your settings.
USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0'
# Obey robots.txt rules
ROBOTSTXT_OBEY = False

Phantomas not render page directly which the page url contains hashmark

I'm testing a webpage by using phantomas,but I found the problem when I use the url contains hashmark such as 'http://bookstore2.shuqireader.com/route.php?sq_pg_param=bsbc&ver=151011#!/bid/3379630/'.
The screenshot of this page in Phantomas is all about blank,but it work perfectly by using PhantomJs alone.
I installed Phantomas by 'npm install'
phantoms http://bookstore2.shuqireader.com/route.php?sq_pg_param=bsbc&ver=151011#!/bid/3379630/ --screenshot=saveimg.png
saveimg.png is all blank
var webPage = require('webpage');
var page = webPage.create();
page.customHeaders = {
"User-Agent": "Mozilla/5.0 (Linux; U; Android 4.0; en-us; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"
};
page.open('http://bookstore2.shuqireader.com/route.php? sq_pg_param=bsbc&ver=151011#!/bid/3379630/', function (status) {
if(status=="success"){
page.render('saveimg.png');
}
phantom.exit();
});
in phantomJs way, saveimg.png is normal
Is it a bug?
Not the answer you want, but generally all these NPM wrappers of PhantomJs suck for various reasons, the authors generally only handle specifically their use case and the packages fail others who have slightly different needs.
Usually they fail for performance reasons (no problem if you are OK maxing out a CPU every request) but as you see, sometimes you'll be caught by situations the authors didn't code for.
You are much better off just writing your launching phantomjs.exe as a child process. Another alternative is to use the api at http://api.PhantomJsCloud.com (disclosure: I made it)

Mobile Site not giving correct Data - Beautiful Soup

I'm trying to get product details from the following website.
Baby Shampoo
Specifically the TCIN:# and product details.
But this information is not showing up in the page when I parse it.
A simple line like:
spans = soup.find_all("span", {"class" : "list-value"})
is turning up no results, and when do I go even more basic to:
print(soup.prettify)
I see the page print out but none of the details are in the page. I am not seeing any iframes on the page, and can't figure out why the data is not showing.
I even attempted to adjust my headers in the request:
headers = { 'User-Agent': 'Mozilla/5.0 (Linux; <Android Version>; <Build Tag etc.>) AppleWebKit/<WebKit Rev> (KHTML, like Gecko) Chrome/<Chrome Rev> Mobile Safari/<WebKit Rev>'}
and also:
headers = { 'User-Agent': 'Mozilla/5.0'}
but neither of these are changing the results. Any ideas what could be happening, and where this data could be located?
Thanks,
Mike
If you see all the Network Requests through Chrome Developer Options or Firefox Firebug, you can see all the http get and post requests made and then you have to find out which one contains the needed information. Make sure that you have Network toolbar enabled and Preserve Log checked before making the request in browser. In your case, the information is fetched by the GET request - http://tws.target.com/productservice/services/item_service/v1/by_itemid?id=13197674&callback=browseCallback

"wclient".DownloadFile in VB Doesnt Work?

I am working on a visual basic project to download files from the internet.
Well i have a Url textbox, directory textbox, and a download button.
And YES i have done some research and i know it is something like...
wclient.DownloadFile(Url.Text,Directory.text)
but for some reason "wclient" doesn't work it says its "not defined"..?
What else should i use or how should i declare it? Should i import something?
Can You help me out?
Thanks in advance!
As simple as that:
Dim wClient As New WebClient
wClient.DownloadFile("RemoteAdress", "LocalFilePath")
RemoteAdress something like "http://example.com/sample.gif"
LocalFilePath something like "C:\Users\Username\Desktop\sample.gif"
Just make sure RemoteAdress is correct, and be aware that your application must have write access to LocalFilePath.

Why do I have an error on some Flattr buttons?

I've integrated Flattr with my Dotclear blog, using the appropriate plugin, and it works fine.
Except that, from time to time, some older posts have errors and I don't understand why.
See:
http://ploum.net/post/224-gpager-03-scenes-from-a-memory-finally-free
http://ploum.net/post/86-le-repas-de-noel-en-famille
those posts have nothing special and, yet, cannot be Flattered.
Could it be an issue on Flattr side ? Do you see what can cause that problem ?
Thanks in advance,
Seems like you're both using the new 0.6 version of the Flattr JavaScript API and a quite old 0.2 version. Using two versions at the same page can probably result in some pretty strange things - I would suggest just using the newer version that is documented here: https://flattr.com/support/integrate/js
Regarding why the button show an error - here's a bit of a secret way to find out: Look at the HTML for the button iframe and look within the div-tag with a "button-wrapper"-class - there you should find an HTML-comment that states the reason for the button error.
In your case the HTML-comment in the button iframe says "ERRORS: User couldn't be found and no owner specified." - which isn't strange since the iframe's URL has "0" specified as the uid-parameter - it should be a username instead.
Your script is disrupted by doublequotes in "var flattr_tle=", you should encode them with '"'