I built a WSGI page that contains both tabulated data and a scatterplot of the same data from a database. (I'm using flask and matplotlib but that doesn't matter). This generates two separate requests: One for the HTML page and one for the dynamically generated image called from the tag. Since the database is rather slow and since both requests need exactly the same data I'd like to make this work with just one SQL query. Two approaches come to mind:
After querying the DB in the HTML view function, generate the scatterplot and save that in a PNG file somewhere. Then pass the tabulated data on to the template and serve up the cached PNG once the browser requests the image.
Somehow embed the image data in the HTML itself and have the browser render it using Javascript.
Approach 1. is simple and straightforward, but I also need a way to get rid of the cached images when they are not needed any more. This is prone to get messy. Since the app is purely http-request driven I would have to scan my cache dir on each request and decide which file is old enough to be deleted. Alternatively I could have an "onload" javascript function call my app a third time to trigger deletion of the image. Maybe clever, but robust?
I have no idea how to do this, let alone in a browser-compatible way.
Any suggestions?
I've been on Usenet for 25 years and still posting a question is the best method to find the answer yourself after a few minutes:
<img src="data:image/png;base64, {{imgdata}}">
and in the view function:
return flask.render_template('chart_page.html', imgdata=base64.b64encode(pixbuf))
End of story. No javascript.
Related
So, I'm trying to come up with a better way to do a dynamic help module that displays a distinct help page for each page of a website. Currently there is a help.aspx that has a title and div section that is filled by methods that grab a database record. Each DB record is stored html withy the specific help content. Now, this works but it is an utter pain to maintain when, say an image, changes or the text has to be edited, you have to find and updated 1 or more DB records. I was thinking instead, I could build a single html page that basically shows/hides panels and inside each panel is the appropriate help content. As long as you follow a proper naming convention (name the panels ID to the page/content it represents) using ctrl + f will get you where you need to go and make it easier to find the content you need. What I'm curious of is would this have an impact on performance? The html page would be a fairly large file and would be hosted/ran at the server but it would also remove the need for Database calls. Would the work even be worth the benefit here or am I reinventing the wheel already in place?
Dynamic anything should be stored in the database. A truly usable web application should NEVER need code modified to change content. Hiding content is usually not a good idea, imagine if you expanded your application to 100 different pages that need their own help page. Then when someone clicks help their browser has to load 99 hidden pages to get 1 that it will show. You need to break your help page down into sections and just store the plain text in the database. I would need to know more about what language you're using as well as the architecture you're using to elaborate further but take a look below.
The need your describing is pretty much what MVC (web application architecture type) was built for.
If you're already using ASP.net and you aren't too far into your project I would consider switching to MVC. It's an architecture built specifically with dynamic page content in mind. You build different 'Views' (the V in MVC) that will dynamically build the HTML based on the content it receives from the Controller (The C in MVC) which pulls it's data from the database/Model (The M) and modifies it for the View. Also once you get into MVC you can couple it with Razor and half of your code get's written for you. It's a wonderful thing.
http://www.asp.net/mvc
I want to exclude a number of items on my website when viewing it on a screen with a small screen resolution so I'm using CSS media queries to hide the items. But a Facebook Like box, for example, is still loaded in the background - a number of javascripts and css files - how do I prevent them from being loaded in order to decrease the page load time?
Right now I'm using PHP to check user agent and then simply exclude the code, but I wonder if that is the best way really. Over time I will have to change the PHP when new devices are introduced for instance. Is a script like Modernizr an option here?
Using a back-end and front-end mix would be best. You can use something like WURFL or even Apache Mobile Filter to make it easier to work on the back-end, and loading certain content only when is_tablet() or is_mobile() are true.
On the other hand, I'd recommend checking jQuery image lazy-loading and share-button on demand loading (like techcrunch.com does). Those two fron-end techniques can really improve page load times (they cut almost in half the initial pageload time on my site).
Depending on the situation, you could also use Modernizr (a front-end javascript solution) to detect something truthy (such as [if width > 320]) and then conditionally load some scripts etc based on that.
I have a Rails 3 application that uses jQuery DataTable to display a list of products that a user has drilled down to. However, in the case of a generic type of product, there may be over 3,000 different products to show. It takes about 30 seconds to render the page for such result sets, even though there are only 25 visible in the datatable (I understand that they ALL render, then datatable pages through the results).
My question is, I would like to only render the first page worth of results, and only call the others if the user clicks the next or previous buttons. Is that even possible?
All the processing i.e. pagination, sorting, etc. can be done serverside with this plugin. This example from the plugin docs shows how to handle the server side using php (sorry, I know you are using Rails) but it might give you an idea. They key is to set
"bServerSide": true
in the initialization of your datatable. Once you do that, you should be able to do whatever you like on the server side, instead of returning the entire dataset in one shot.
Note: Correct me if I am wrong but I assume you are using DataTables.
Much better than doing this server side is to set bDeferRender to true. This does just what it sounds like and only renders the rows it is displaying. We found that every browser later than IE6 has a good enough JS engine to handle many thousands of rows with that option enabled.
I'm writing an obj-c app and would like to upload a binary file a few megs in size to my appengine server (python). I'm guessing I need to use the blob entity for this, but am unsure how to go about doing this. I've been using http requests and responses to send and receive data up to now, but they've been encoded in strings. Can someone advise how I'd go about doing the same with blobs from an obj-c app? I see some examples that involve http requests but they seem geared toward web page and I'm not terribly familiar with it. Are there any decent tutorials or walkthroughs perhaps?
I'm basically not completely sure, if I'm supposed to encode it into the http request and send it back through the response, how to get the binary data into the http string from the client and how to send it back properly from the server when downloading my binary data. I'm thinking perhaps the approach has to be totally different from what I'm used to with encoding values into my request in the param1=val¶m2=val2 style format but uncertain.
Should I be using the blobstore service for this? One important note is that I've heard there is a 1 meg limit on blobs, but I have audio files 2-3 megs in size that I need to store (at the very least 1.8 megs).
I recently had to do something similar, though it was binary data over a socket connection. To the client using XML, to the server as a data stream. I ended up base64 encoding the binary data when sending it back and forth. It's a bit wordy but especially on the client side it made things easier to deal with, no special characters to worry about in my XML. I then translated it with NSData into a real binary format. I used this code to do the encoding and decoding, search for "cyrus" to find the snippet I used, there are a few that would work here.
In your case I would change your http request to a post data call rather than putting it all in the URL. If you're not sure what the difference is, have a look here.
I'm not as familiar with python, but you could try here for help on that end.
Hope that helps.
Edit - it looks like blobs are the way to go. Have a look at this link for the string/blob type as well as this link for more info on working with the blob.
There are three questions in one here:
Should you use a BLOB for binary data?
How do you post binary data, and use it from app engine
How do you retrieve binary data from app engine
I can't answer if you "should" use blobs, only you would know the answer to that, and it greatly depends upon the type of data you are trying to store, and how it will be used. Let's take an image for example (which is probably the most popular use case for this). You want users to take a photo with their phone, upload it, and then share it with other users. That's a good use of blobs, but as #slycrel suggests you'll run into limitations on record size. This can be workable, for example you could use the python image library (pil) to downsize the image.
To post binary data, see this question. It would be best to cache 2 copies, a thumbnail and a full size. This way the resizing only has to happen once, on upload. If you want to go one better, you can use the new background jobs feature of app engine to queue up the image processing for later. Either way, you'll want to return the ID of the newly created blob so you can reference it from the device without an additional http request.
To retrieve data, I think the best approach would be to treat the BLOB as it's own resource. Adjust your routes such that any given blob has a unique URL:
http://myweb/images/(thumbnail|fullsize)/<blobid>.(jpg|png|gif)
Where BLOBID is dynamic, and JPG, PNG or GIF could be used to get the particular type of image. Thumbnail or fullsize could be used to retrieve the smaller or larger version you saved when they posted it.
I need to display pages in a tutorial fashion. I looked in to netsupport, beamyourscreen and other possibilities but, I do not want the viewers to download anything. I cannot use gd / send screenshots due to audio / video instructions embedded in some of the pages.
Basically, I need the ability to "refresh" a users browser window to a different page via an interface on my end. Whether via a form submission, javascript or any other type of "controller" that allows me to change the page on the viewers browser. PERL preferred but, PHP / javascript whatever works and is cross browser. I set up a simple javascript page forward timer that "works" but, page load times and conversation interruptions are a huge factor.
The entire tutorial website will be developed around this ability.
I was looking in to curl / cron / wget methods but, found little information.
I have seen forum and chat scripts that basically perform a similar task but, there must be a simple(ish) solution in leau of hacking up another script to suit my needs.
I do not want others to control the pages either. The site really, only needs to be accessable during the tutorial however, It "could" remain web accessable as long as user interaction was normal unless (being controlled).
The initial site concept is based on instructing people how to properly introduce new pets into a home. Will be operated by a veteranarian that saved my pets life. I wanted to give something back.
Possible? I really appreciate simple examples etc...
You have no other way but to keep polling the server for "instructions" using javascript. No, you can't send nothing to the end user browser, neither curl nor wget.
Mainly, you'll have to set up a simple request/response protocol between the browser and the server.
If you want to go deeper, you can use something like cometd/meteord/etc. If not, a hidden iframe that reloads himself and receives pages with javascript code for the needed actions can do the trick.
Another alternative.
With javascript dopolling and single character flatfile. Have a simple one character flatfile with a single var. Write it in perl (it is faster and uses less resources than php). The parent script calls a javascript variable in a flatfile. It hits the flatfile and goes wherever the var sets it. The flatfile is written to by the controller. Done.
I guess you could also rename an empty flatfile and use that as the controller. I am usure which is faster, open and read a specific file or hit the directory and return the file name. On the controller side, opening and writing to a file vs renaming a file. Maybe they counter each other in resources and time?
This way the site can act as a normal site. When you want to have remote users see a "presentation" (automatically being shown the site pages at the controllers pace), the controller activates polling and tells the viewers to push a start button. This allows a remote instructor to load pages for the viewers at his leisure.
It is a simple solution that works with nothing really sophisticated going on. No frames are needed either. Just need javascript enabled.
Any better suggestions are welcome!
It occurred to me that what you might want to use is HTML Push technology. Check out the wiki, they have several links. I have never used it myself