I did comprehensive Google research but I cannot find any good solution, so any help (or just showing direction of next research) would by REALLY appreciated!
What I need is simple in idea, but looks like hard to implement:
I have data (say just picture) I want to show to all (even anonymous) users of website in the very same time. This data should change regularly (say once in 5 minutes), so the browsers of all users must refresh in given time.
The woflow is simple:
User will open page with countdown (which will show of course different time for each user depends of when the user has had connected).
At the end of countdown shall all browsers of connected users refresh to see new content.
The refresh should be ideally invoked by server to prevent prematured refresh when data doesn't changed yet.
I was thinking of "refresh" meta tag, but it is problematic for SEO and it rely on user computer's clock.
It can be done by javascript, but in that moment I rely on user computer's clock.
I have hearded it is possible "push" data from server to browser using e.g. Perl, it is correct (is there somewhere some example)?
And in which scripting language would you write script which would "tick". I cannot see way in PHP I am familiar with (use cron to execute counting script every minute looks really ugly)...
Thank you!
Michal
It's not possible to push data from a web server to a web brower, given the request-response architecture of HTTP. It is, however, possible to poll the webserver using JavaScript and window.setInterval(); combined with AJAX.
If not using local system time is an issue, why not respond to periodic AJAX requests with the number of microseconds until the next reload of data should commence? I would suggest you use AJAX for all of this instead of refreshing the browser with META REFRESH, or window.location.
The server-side code could be anything really, you simply need a page that will return the number of microseconds until the next schedule refresh (And perhaps an error if no refresh is scheduled yet, telling the client JavaScript to poll again in a few seconds).
Related
I want to create an app for faster payment of parking.
This question is more about logic of my app, and what tools I need to use about creating it.
At this point, I use a parking place every day and I pay for it through the web page.
I do it like this.
Login to page.
click on the menu and it redirects me to www.parkingexample.page/payments
there is a search menu and I enter my car plate number if my car is found it returns me how much I need to pay, and "Pay" Button appears.
I click "Pay" buttons and then it's all done.
So my goal is to create an app that when I start it will automatically connect to the page and will search for my plate and if found and payment is needed there would be just one button "Pay"
So I think I should do it like this, but as I haven't created any web app(I'm 100% back-end developer) I ask you is my thought process is correct.
And also I don't want to use WebView as I think it's not necessary for me.
When I start my app it sends "POST" request to page to login.
Then I send 'GET' request to www.parkingexample.page/payments with params = 'mycarspaltenumber'
Somehow I need to click on PAY button on page when it appears so I think it's probably again 'POST' request, but at this point, I'm not sure.
So a QUESTION is, is my logic valid? or it can be done in some other way?
UPDATE. ADDED SCREENSHOTS
First Screen shoot this is the menu after I logged in with the search bar where I need to enter my card plate.
Second screen is where I found my car(Entered plate number and clicked search)
and now the page is updated with sum I have to pay and there is a button "PAID" in the bottom right corner I need to click.
And that's all i need.
To validate whether your suggested sequence is correct I would start by capturing your typical browser session between yourself and your parking provider with something like Fiddler. Then I would use HTTP client library of choice (for C# it would be something like HttpClient) and emulate the same flow with correct headers, query parameters and such like.
Looknig at your screenshots it seems the application is ASP.NET Web Forms, which can get a bit painful to emulate due to way its state management works: you will likely need to decode View state object (to ensure you're passing it back correctly) and locate all dynamic field ids that it uses for postbacks. This however is very doable.
If you discover that the above is too hard to emulate (or there's javascript involved) it might be easier to explore Remote Selenium WebDriver coupled with a headless browser like PhantomJS. You'd then have your PhantomJS interact with the page on your server, and you'll drive it with your mobile app. Basically you'll reduce the complexity of your parking provider page to a well documented API.
Hopefully that gives you a starting point
In your application, all that you will need is services call and the security part of logging a new user everytime to check for payment.
So It will be a simple spring-boot application, where you can use the security part for logging, and you can exactly use the simple way , for example you don't need to have a database, just to redirect your page, and if you are not familiar to front-end framework, you can use a basic html-css pages for client side.
Another important point, you should start by designing your application, before coding, because it's very important to know all the ideas behind your application.
Enjoy your doing time!
This might sound like a question that gets asked frequently but I am not looking for solutions to handle duplicate requests. I just want to know what could cause Apache to receive duplicate requests in the first place.
I have been running into a rather sporadic problem. I have a form that does a POST request on submit but the request can somehow get duplicated just a second later (according to access logs). This used to be a more frequent problem because we were not handling it as gracefully so I put in some client side code to disable the submit button during the form submit event. This prevents double submission (obviously as long as javascript is enabled), but the problem still persists in a very randomly manner. One thing I noticed from logs is clients that cause the issue are android phones running Chrome. Does mobile Chrome do funky things like retry POST requests on it's own? When testing it on my own, I could never get duplicate POST requests to occur, unless I remove the javascript code that disables the submit button.
Web server setup is super simple. No load balancing or anything, just a single server running Apache 2.2.15. We use PHP 5.6 but that probably has nothing to do with this.
I guess it is users doubleclicking rather than clicking, and the application they use transforms every click into a new POST request. Here I'd look into the application design.
Usually I use frameworks that totally cover this and thus can only guess. Clicking the button should not only trigger the POST request but also disable the button while the action is in progress. So JavaScript code could look like
disable button
post the data
enable button
If, due to the POST, the browser navigates to another page this would not be harmful at all.
EDIT: Seeing you did exactly what I suggested, maybe there is another cause.
Suppose users POST their data, and then the screen goes dark, or they switch applications. When they reactivate the browser, is it possible the browser reloads the page by repeating the last request?
I think frameworks cover such situations by responding with a redirect as response to POST, and the redirect would retrieve the data via GET. Since GET is idempotent, it can be run repeatedly without further damage.
I was wondering how the services like http://www.inspectlet.com/ does store the video sessions. By the looks I don't think it's a webRTC implementation. What i was able to figure out that there is active express socket which is making communication but in that case they will have to store the page and track all the events from DOM. Just wanted to confirm that this is the approach they are following.
Looking at the event listeners on the page, it looks like there are a lot of bindings. For example, the <body> has scroll, keyup, and change events bound to a function. I'm sure it also has mousemove, mouseclick, etc. All of this is likely stored in a Javascript variable (object, probably) and then AJAXed every so often to http://hn.inspectlet.com/adddata with the data parameters. Here is a sample of what is being sent:
http://hn.inspectlet.com/adddata?d=mr,212941,46,337,46,1277)mr,213248,163,498,163,1438)mr,213560,144,567,144,1507)mr,213873,138,240,138,1180)mr,214188,169,184,169,1124)mr,214504,158,520,158,1460)mr,214816,231,487,231,1427)mr,215130,329,197,329,1137)mr,215444,894,289,894,1229)mr,215755,903,295,903,735)s,215755,440,0)&w=259769975&r=494850609&sd=1707&sid=1660474937&pad=3&dn=dn&fadd=false&oid=99731212&lpt=212629
As suggested in Adam's answer, they track many events in the page and send them either via a websocket or post/get request (XHR) to their servers.
I am not sure what inspectlet does specifically, but in general, such a solution will need to follow the below general steps:
When the page is fully loaded (hook on DOMContentLoaded probably) they will send the page data to the server. Then they also hook on MutationObserver in order to track all changes to the DOM in the page, so when something changes dynamically, the tracking script can 'record' it and send it to the server.
The SaaS application in turn, will have a player that will parse all this raw data and then play it back, this will usually require using some virtual file system for assests (images, css, scripts) and handling js scripts to not post back again (replay xhr will have bad results for tracked websites) but play back the mutations as they were captured (recorded)
In regards of data HTML pages compress really well, especially when you can make assumptions about the data (since you know its html) - so yes, they actually cache a lot (similar to what google does in that regards, but google does it for the entire web, not just 'customers')
The DOM Mutations will need to be stored as well. This is up to the algorithm, it can either be stored as plain text or using a smarter data model, storing them in plain text is obviously not the cost effective solution.
The above is an abstraction, there are many edge cases to handle in order to implement such a solution as well as a lot of mathematical and algorithmic thinking in regards of heatmaps to make them accurate.
So after a long search was able to find a new promising solution on the block, which solves all the complex parts in building such service. It is still under development but it solves the problem. Below is the link to it,
https://www.rrweb.io/
https://github.com/rrweb-io/rrweb
I am somehow familiar with benchmarking/stress testing traditional web application and I find it relatively easy to start estimating maximum load for it. With tools I am familiar (Apache ab, Apache JMeter) I can get a rough estimate of the number of request/second a server with a standard app can handle. I can come up with user story, create a list of page I would like to check and benchmark them separately. A lot of information can be found on the internet how to go from novice like me to a master.
But in my opinion a lot of things is different when benchmarking single page application. The main entry point is the most expensive request, because the user loads majority of things needed for proper app experience (or at least in my app it is this way). After it navigation to other places is just ajax request, waiting for json, templating. So time to window load is not important anymore.
To add problems to this, I was not able to find any resources how people do it properly.
In my particular case I have a SPA written with knockout, and sitting on apache server (most probably this is irrelevant). I would like to have rough estimate how many users can my app on the particular server handle. I am not looking for a tool recommendation (also it would be nice), I am looking for experienced person to share his insight about benchmarking process.
I suggest you to test this application just like you would test any other web application, as you said - identify the common use cases, prepare scripts for them, run in the appropriate mix and analyze the results.
Web-applications can break in many different ways for different reasons. You are speculating that the first page load is heavy and the rest is just small ajax. From experience I can tell you that this is sometimes misleading - for example, you can find that the heavy page is coming from cache and the server is not working hard for it, but a small ajax response requires a lot of computing power or long running database query or has some locking in the code that cause it to break or be slow under load - that's why we do load testing.
You can do this with any load testing tool, ideally one that can handle those types of script with many dynamic values. My personal preference is WebLOAD by RadView
I am dealing with similar scenario, SPA application where first page loads and there after everything is done by just requesting for other html pages and/or web service calls to get the data.
My goal is to stress test the web server and DB server.
My solution is to just create request for those html pages(very low performance issue, IMO, since they are static and they can be cached for a very long time in the browser) and web service call requests. Biggest load will come from the request for data/processing via the web service call requests.
Capture all the requests for html and web service calls using a tool like fiddler, and use any load test tools(like JMeter) to run these requests using as many virtual users as you want to test your application with.
I am trying to figure out just how Google Analytics logs the actual time of an event such as a visit or a click. Does it use the time being passed by the HTTP request, the date in the HTTP Header, or the date it actually receives the request ?
The reason why I am asking is because I am developing an app where there MAY be some times where the internet connection is lost. So, I am considering developing a buffered proxy to hold these requests until the connection is back up. If the data/time is based on the header or the time of the request, then it will not be a viable solution for me.
If anyone knows, it would be greatly appreciated.
The Google Analytics server uses the actual time of the event. This means that any event notification buffering that your app performs will be timeshifted in your reports.
Only you can decide if that's OK for your reporting requirements: which is the lesser evil, to drop events, or misreport the times that some events occur?