What I'm looking for is some sort of a proxy tool that will allow me to specify a local file to load instead of one specified in the web page that is being browsed. I have tried Burp Suite which is almost working - it allows us to intercept a file and replace it by pasting the contents of the file we are swapping in into an input field. The file content is compiled code (Flash content) so we are pasting in bytecode, but something isn't working.
The reason is we are a 3rd party software developer without access to our client's development or testing environments. Our content must interact correctly with the rest of the content on their webpage (there are elements on their page that communicate with our content) and to test any changes we make takes several hours turnaround to get our files uploaded to their servers. So what we need is some sort of hacking tool to let us test our work with their web pages, hence the requirement to specify a file in a webpage to swap with a local version.
The autoresponder feature in Fiddler Web Debugging Proxy might do what you need, if it's only static content.
I've been using HTTP::Proxy for a long time, and it has always helped me fiddle with things on the fly.
You might be able to do this with Greasemonkey but I'm not sure if the tests will be totally reliable.
http://diveintogreasemonkey.org/patterns/replace-element.html
And if Greasemonkey seems plain wrong for you I would take it as the perfect excuse to try out mouseHole. Now I have to admit that I've never tried it but since _why also made Hpricot I expect it to be fun, productive, and different.
Related
My searches have turned up nothing concrete. My extension uses Google's file picker to allow the user to browse their sheets and choose a desired file to write some data to, which manifest v3 breaks because of some GAPI limitations. Unless I've missed something obvious, there does not seem to be a simple workaround or method for this to migrate to v3 -- it just seems to be disallowed.
I'm not asking if there's a way to do something that they intend to not be possible (even though I doubt such a thing would exist with Google) but I'm optimistically hoping that maybe there is some hacky/annoying workaround that still fits within their rules. If I absolutely have to just allow them to set a sheet URL manually I will...I'm just trying to avoid it.
Any tips or suggestions would be appreciated.
You may have to test it yourself to make sure there are no weird behaviors, but Google has some recommendations regarding this in their migration guide:
In Manifest V3, all of your extension's logic must be included in the extension. You can no longer load and execute a remotely hosted file. A number of alternative approaches are available, depending on your use case and the reason for remote hosting. Here are approaches to consider:
Configuration-driven features and logic
In this approach, your extension loads a remote configuration (for example a JSON file) at runtime and caches the configuration locally. The extension then uses this cached configuration to decide which features to enable.
Externalize logic with a remote service
Consider migrating application logic from the extension to a remote web service that your extension can call. (Essentially a form of message passing.) This provides you the ability to keep code private and change the code on demand while avoiding the extra overhead of resubmitting to the Chrome Web Store.
Bundle third-party libraries
If you are using a popular framework like React or Bootstrap, you can download the minified files, add them to your project and import them locally.
For your case, option #3 seems like the easiest. Looking at the Google Picker API documentation it only uses two relatively small script files, https://apis.google.com/js/api.js and https://accounts.google.com/gsi/client. You could try to bundle these in your Chrome extension and call the methods locally.
Using MS Edge and apache w/ php, I just discovered via access.log that when I have the JavaScript debug panel (i.e. developer panel) open, it is making every http call twice. When I closed this panel, it has fixed the issue of all insert statements getting called twice.
Question: Does this doubling of http calls happen on every / most browsers that I need to look out for, or is this something special/unique with MS Edge?
I can't speak for all browsers and all developer tools. But, for IE and Edge the first time you open the tools and then open a JS file in the sources view it will try to request the file again. That request will be served from the local browser cache, sometimes not, depending on the cache settings for the file being requested.
The reason browser tools need to make this request is that browsers will often throw out the original source file as it doesn't need it to execute the page, as the source has been parsed it into something else that it can work with.
However, after you've opened the developer tools the browser will keep around sources in future navigations, either in the tools front end or elsewhere. Not keeping sources is an optimization for the first time use case, to save browsers keeping around source on the very low odds of the tool being used on any given navigation.
Of course some files are never cached by the browser and will need to be downloaded when requested by the tools, for example sourcemapped files.
In general any resources on your site that can be accessed by HTTP GET should be idempotent. That is, a GET shouldn't change the resource being requested (or generall the state of your site), so hopefully making additional requests shouldn't be an issue.
I'm wondering how I can find out where the culprit is, as to what is NOT being transmitted over SSL on my website. It's blowing my mind, because I use relative URLs or explicitly choose HTTPS:// for all links, images, etc...
Any ideas/tools to find out what the issue is?
Thanks.
If you mean that some resources are transferred over HTTP without encryption, you can check for this in Chrome's Developer tools in the tab Resources - that should tell you which parts come from where - look for those with address starting with http:// .
Alternately, use Fiddler: by default, it won't decrypt HTTPS connections, so you'll be seeing CONNECT requests for HTTPS, and GET/POST for HTTP - those are your culprits.
For those, like myself, who run into this issue i suggest a few tips while designing your website.
Always use relative paths when ever possible "images/someimage.png" instead of using domain paths like http://someDomainName/images/someimage.png so on. Any one of these and it will cause the browser to throw that warning at you.
When linking to external content, Google/other Ads, javascript sources(such as jquery, so on), or any other media... make sure you use a https:// link if they have one available. Myself, i had one tiny image for a link to an external site but they did not offer a https link to the image, so i simply downloaded it and put it in my images folder. Problem solved.
The Chrome resources list is a very helpful tool, not sure if Firefox has something similar in its tool box. Another method, if you have shell/command line access, is to use grep to search the files for "http:". This, most often, will show anything that is linking to non secure content.
I'm building an application with a self-contained HTTP server which can be either accessed directly, or put behind a reverse proxy (like Apache mod_proxy).
So, let's say my application is running on port 8080 and you set up your Apache like this:
ProxyPass /myapp http://localhost:8080
ProxyPassReverse /myapp http://localhost:8080
This will cause HTTP requests coming into the main Apache server that go to /myapp/* to be proxied to my application. If a request comes in like GET /myapp/bar, my application will see GET /bar. This is as it should be.
The problem that arises is in generating URIs that have to be translated from my application's URI-space in order to work correctly via the proxy (i.e. prepending /myapp/).
The ProxyPassReverse directive takes care of handling this for URIs in HTTP headers (redirects and so forth.) But that doesn't handle URIs in the HTML generated by my application, or in static files and templates.
I'm aware of filters like mod_proxy_html, but this is a non-standard Apache module, and in any case, such filters may not be available for other front-end web servers which are capable of acting as a reverse proxy.
So I've come up with a few possible strategies:
Require an environment variable be set somewhere that contains the proxy path, and prepend this to all generated URIs. This seems inelegant; it breaks the encapsulation provided by the reverse proxy.
Put the proxy path in a configuration file for my application. Same objection as above.
Use only relative URIs in my application. This can get somewhat tricky; I would have to calculate the path difference between the current resource and where the link is going and add the appropriate number of ../'es. Seems messy. Another problem is that some things must generate absolute URIs, like RSS feeds and generated emails.
Use some hacky Javascript on the front-end to mungle URIs in the document text. This seems like a really horrible idea from an interoperability standpoint.
Use a singe URI-generating function throughout my code, and require "static" files like Javascript, CSS, etc. to be run through my templating system. This is the idea I'm leaning towards now.
This must be a fairly common problem. How have you approached it in the past? What has worked and what has made things more difficult?
Yep, common problem. How to solve this depends on the kind of app you have and the server platform and web framework you're working with. But there's a general way I've approached these problems which has worked pretty well so far.
My preference is to handle problems like this in application code, rather than relying on web server modules like mod_proxy_html to do it, because there are often too many special cases (e.g. client-side-javascript assembling URLs on the fly) which the server module doesn't catch. That said, I've resorted to the server-module approach in a few cases, but I decided to revise the module code myself to handle the corner cases. Also keep perormance in mind; fixing up URLs in your code at the time they're generated is usually faster than shoving the entire HTML through another server module.
Here's my recommendation of how to handle this in your code:
First, you'll need to figure out what kind of URLs to generate. My preference is for relative URLs. You are correct above that "add the appropriate number of ../'es" is messy, but at least it's your (the programmer's) mess. If you go with the config-file/environment-variable approach, then you'll be dependent on whoever deploys your app (e.g. an underpaid and grumpy IT operations engineer) to always set things up correctly. It also complicates release of your code, even if you're doing deployment yourself, since you can't simply copy your development files into production but need to add a per-deployment-environment custom step. I've found in the past that eliminating potential deployment problems is worth a lot of pre-emptive coding.
Next, you'll need to get those URLs into your code. How you do this varies based on type of content/code:
For server-side code (e.g. PHP, RoR, etc.) you'll want to make sure that server-side URL generation happens in as few places as possible in your code (ideally, one method!). If you're using any of the mainstream MVC web frameworks (e.g. RoR, Django, etc.), this should be trivial since URL generation using an MVC framework already generally goes through a single codepath that you can override. If you're not using one of those frameworks, you likely have URL generation littered throughout your code. But the approach you'll want to take is to generate all URLs via code, and then override that method to support transforming non-relative URLs into relative URLs. You can usually search for patterns in your code (like "/, '/, "http://, 'http://) and do a manual search and replace (or if you're really nerdy and have more patience than I do, craft a regex to replace each common case in your source code).
The key to making this work reliably is that, instead of manually replacing all absolute URLs with relative ones in your server-side code (which, even if you get each of them right, is fragile if files are moved), you can leave the absolute URLs in place and simply wrap them with a call to your "relativizer" method. This is much more reliable and unbrittle.
For Javascript, I generally like to do the same thing as server code-- move all URL generation into a single method and ensure any URL generation calls this method. This can be hard on an app with lots of pre-existing javascript, but the search-and-replace method above seems to work well in JS too.
For CSS, URLs in CSS are relative to the location of the CSS file (not the calling HTML page) so using relative URLs is generally easy. Simply put your CSS into a folder and either put images into deeper folders beneath it, or put images into a parallel folder to your CSS and use a single ../ to get to the images relatively. This is a good best practice in general-- if you're not doing relative URLs in CSS already, you should consider doing it, regardless of reverse proxy.
Finally, you'll need to figure out what to do for other oddball static files (like legacy static HTML files sometimes creep in). In general, I recommend the same practice as CSS and images-- ideally, you'd put static files into predictable directories and rely on relative URLs. Or (depending on your server platform) it may be easier to remap the file extensions of those static files so that they're processed by your web framework-- and then run your server-side URL generator for all URLs. Or, barring that, you can leave the files in place and manually fix up URLs to be relative-- knowing that this is brittle.
Coming full circle, sometimes there are just too many places where URLs are generated, and it's more effective to use a server module like mod_proxy_html. But I consider this a last resort-- especially if you won't be comfortable editing the source code if needed.
BTW, I realize I didn't mention anyting about your idea #4 above (javascript-link-fixup). I wouldn't do that-- if the user has javascript turned off or (more common) some network problem prevents that javascript for some time after the rest of the page loads, then your links won't work. Too risky.
We have an Adobe AIR application which could be possibly downloaded from multiple domains. And when it's run, it should connect back to the site it was downloaded from to get data to show to the user.
So far we have a separate application build for each domain with a site URL hardcoded into it. And I wonder is there a way for AIR application to find out at runtime the URL (or at least domain) from which it was downloaded?
What we would like to have is a single downloadable binary served from all different domains, which still can know it's origin URL.
There's no function to retrieve such information, it would just make no sense if you think about it.
The most stable way is to include an external configuration file into the package.
Note that you can use ANT to automate this process for this final deployment.
There's no direct way to do it.
Here are some options which come in mind:
Build different versions for each site (this could be automated)
Let user choose the site at first launch
Try to guess it using using whatever resources you have (timezone, language, etc)
How should this work? The only solution i see (independent from AIR) is that you deliver an extra (properties) file with the application, containing the URL downloaded from. So you dont need to build a separate app for each domain, but only package a different domain-file with it. The app then reads this file and executes some context sensitive stuff.
I am trying to address the exact same issue right now.
It looks like you can modify the install badge to pass parameters to the air app.
From what I gather the values are only passed down on install or launch-from-badge.
Something I plan on researching is that one of the parameters in "AIRBadge.as" is _appURL which is the URL of the page the badge is on. I don't yet know if that value makes it down to the installed AIR app in some way; but it could be a useful property. I'm ultimately hoping that the AIR install process injects that into the application descriptor xml, but I'm not holding my breath.
Check this page out: http://archive.davidtucker.net/2008/01/10/air-tip-5-passing-arguments-to-an-application-on-install/#
When the user downloads, you could store their IP address in your central DB. Then when the app is installed and runs the first time, the app could hit your central DB to match up their IP address with the server they downloaded from.
A cookie with a specific name being stored on a download page, and the AIR app looking for that? Though that might not work for direct downloads. It might also be hard to pull off since knowing the specific browser used to download it would be an issue.