i am using scrapy_splash to crawl pages that requires js to retrieve proper content.
Everything is fine when I use SplashRequest without proxy settings, but when I put in proxy settings, the javascript would not render, giving me prejavascript html content which do not have the data I need.
Does anyone know how to solve this? I am sure the proxy ip are not blacklisted.
I can't comment,
But the short answer is because the proxy that you use don't allow to run javascript codes.
Please try another proxy that allow you run javascript code
it turns out to be a slow proxy. I had to set wait to 10 to get result. Thanks for the help guys.
Related
I just started working with Nuxt JS, and I'm trying to pass a request from the client to my Express server, but Nuxt treats the routes like they were suppose to be pages.
Hence I get the errors page not found in pages folder. Help please.
I've gotten into using an express backend with nuxt and it's a great way to go about it. As there is quite a lot steps to take and files to set up it's perhaps beyond the scope of a single answer here to cover what's needed. I'd recommend you start with this tutorial on codeburst which covers all the steps you need and can be expanded upon easily once you understand it. Good luck.
I am having problem with chimp.js configuration. When I run tests the browser is not using provided host / post. I tried many things but without success. Does anyone know how to set up it properly? Please find my config in screenshot provided.
When the browser first starts, it defaults to data;. You need to navigate somewhere first.
The host and port settings are not related to browsing. They tell Chimp where the Selenium server is, which by default is chromedriver.
I had to setup beforeFeature hook to start using localhost:3000
I want to redirect to a maintenance page when my apache server is down.I tried through
htacess
and nothing worked.. Please direct me to some working blog.
Thanks...
The .htaccess file contains no executable program code to do some activities. If you want to show a webpage then you need a running webserver. If you have no one bad luck.
Take a look at this post about the technicel enviroment to reach your goal.
Edit: I found another post about this topic, though the same thing is the use of an additional webserver.
So I am currently working on an iPad App (iOS 6) that shows a webpage that is located on my company's intranet. The access from the Internet is achieved via a reverse proxy which works great.
The problem is that this webpage is referencing a JavaScript file that is apprently loaded after the webpage is rendered on the client which results in some jQuery scripts not being run on load time. What works is this:
open the website through the reverse proxy
wait for the webViewDidFinishLoad delegation method to be called
then do [webView reload];
After this, the javascript code is successfully executed. However the solution is not pretty and results in extended wait times.
Are there any solutions to ensure that all referenced files are loaded before rendering takes place?
Thank you!
Christoph
You'll have to fix the webpage so that it doesn't try to run the javascript until the file has actually loaded.
Look up the UIWebView property that was added in iOS 6 called suppressesIncrementalRendering -- I think that will do exactly what you're looking for.
The problem was the way I had the UIWebViewer interact with the Reverse Proxy.
Getting the HTML code through the proxy without involving my webviewer object resulted in a delayed javascript loading.
However, after doing one test connection through the reverse proxy authorization class, I could use this method to load all subsequent webpages w/o any problems:
[webView loadRequest:request];
Thanks everyone for your ideas and feedback!
Christoph
I am using Watin mostly to automate thing I do by hand. Sometimes, I need to change proxy. To do this, I have to set up IE to use a local proxy all the time (listens locally and forwards to a remote porxy), and then by hand change the settings of that program each time I need to use another proxy.
This is not an elegant solution! It may work, but it is not nice.
So, my question is, how do I change programmatically IE settings while using Watin? I code in VB.NET.
Thank you
try this
http://huddledmasses.org/setting-windows-internet-connection-proxy-from-c/