Our legacy ASP.NET 2.0 website, running in IIS6 on Windows 2003 Server, accepts lots of user uploaded and user downloadable files. Not page content; asset files like PDFs and PNGs mostly. However it is of course advisable not to let the naughty buggers upload ASPX files and suddenly be able to execute code on our server.
So ... you'd think we could just set "Execute permissions: none" in the appropriate folder where user uploaded files end up, and that'd be the end of it. Alas.
User uploaded files end up in folders named like these patterns: (nnnn, kkkk and llll represent digit strings of varying length)
/sites/nnnn/files/ftp
/sites/nnnn/files/orders/kkkk/llll
/sites/nnnn/files/proofs
/sites/nnnn/files/webFiles
BUT ... there are also CSS files stored in folders like these:
/sites/nnnn/files/skins/kkkk/*.css
And the CSS files are actually interpreted as ASPX code, so that color palette substitutions can be passed in the querystring, like this:
/sites/2132/skins/83/layout.css?palette=cc33dd,aa22bb,1155ff
So, "Execute permissions: none" is a no-no at the /sites level. And I really would rather not do it at /sites/nnnn/files, individually for a few thousand nnnn.
So I've been told URLScan 3.1 is the ticket, but after an hour or two of staring at the documentation I'm more confused than ever.
Any ideas?
Related
I have a database file that looks like the following:
**Name Number1 Number2 File**
Henk 123 456 c:\henk.pdf
Piet 345 789 c:\piet.pdf
When I put this in a SQL Datasource (Visual Studio), the file should be clickable text or image.
Now it's just plain text.
<asp:BoundField DataField="Relatienummer" HeaderText="Relatienummer" SortExpression="Relatienummer" />
<asp:BoundField DataField="nummer" HeaderText="nummer" SortExpression="nummer" />
<asp:BoundField DataField="company" HeaderText="company" SortExpression="company" />
<asp:BoundField DataField="Link" HeaderText="Link" SortExpression="Link" />
What I currently have is:
<asp:HyperLinkField DataNavigateUrlFields="Link" DataNavigateUrlFormatString="Link" DataTextField="Link" HeaderText="Link" />
But this links to link instead of the content of the sql.
Thanks
Remember, code behind can use full file system - like any desktop code.
But if you going to use that file in the asp.net (web side), then you MUST be able to produce a valid URL path name.
So ALWAYS keep in mind the above:
Codebehind - path names are STANDARD windows file path names (and even "\" etc.).
Web pages - markup = valid URL's mapped by the web site.
If you confuse the above two concpets above? you be in for a world of pain by ignoring the above VERY simple idea.
So, you need TWO things to convert those path names to valid WEB links:
First up: The web server "site" needs rights and a "means" to point to a given folder in question.
So, you might have some say SAN or big network storage device - and you toss/place/have/put your files there.
Say like this:
\\BIGSERVER1\PdfFolder\Customers
Now in your code behind? You can use
\\BIGSERVER1\PdfFolder\Customers\Policy.pdf
But for the web server? Well, there is no WAY that you could and would allow just any old server path names on your network to be used by VALID WEB URL's (again, that concept of code behind vs VALID web links MUCH be in your brain thoughts at ALL times).
Ok, so how would we use the above file path on the asp.net + web side? (now we NOT talking about code behind - right?).
Well, in 99% of cases, you thus fire up IIS management, and add what is called a Virtual folder. This thus mapps from the web site to the above folder.
It will look like this:
So in above, the folder UpLoadFiles will and can be mapped say this:
\\BIGSERVER1\PdfFolder\Customers
So, codebehind (your vb/c# code will use this:
\\BIGSERVER1\PdfFolder\Customers\myfile.pdf
But, a web link will be this
http://myCoolWebSite/UpLoadFiles/myfile.pdf
So any hyper link etc. HAS to be a valid URL as per above. And since we ONLY mapped the ONE folder to the web site, then users can't just type in any old email and grab/get/mess with files on that server, or your whole netowrk for that matter. But KEEP in mind your code behind does not have such restrictions - that just plan jane code with plane jane valid full server or simple full valid windows pathnames.
So, in your data table, lets assume the file saved was
myfile.pdf
So, I do suggest that you separate out the valid windows path name from the file.
Ok, so to provide a valid web link, you need this expression:
"~/UpLoadFiles/" + "myfile.pdf"
So this can be a data bind expression, or even part of the SQL query that drives the grid/list or whoever it is displaying.
Last but not least:
Often in your code, you need (want/have to) translate a given valid URL to a correct good old plane jane file path name (after all ANY code behind will only work with those plane jane full valid windows path names.
So, your code can do this:
dim strFileName as string
' assume say you have the database and a row in a "data row"
strFileName = Server.MapPath("~/UpLoadFiles/" & MyDataRow("FileName"))
So above code will of course translate the virtual folder that points
to the server holding the files and return a full valid windows path name
for your code. (the same as the full server pathname I exampled above).
The other approach?
Well, don't setup a virtual folder, and don't allow valid URL paths directly to those files. If you dealing with say customer A files, and customer B files, then allowing valid URL's to those files (that they COULD just type in), then you resort to ONLY allowing and use code behind to get those files.
So, now your hyper link becomes a button. When they click on that button, code behind reads the file and STREAMS (pumps) out the file to the web browser. They will thus get a standard file download when you do this, but NEVER does any actual valid URL exist for them to type in (and thus the files are secure - since then you and your code will fully control who can get what file since no valid URL's that map to that folder question exist. Thus ONLY code behind can continue to use and enjoy plane jane valid windows file names, but the web side of things will not have nor allow valid URL's to the mapped virtual folder, since you did not create one.
Of course if you create a "files folder" as a sub folder in the web site, then yes in most cases valid URL's can be typed in. But if you use server or file folder OUTSIDE of the root web site and folders, then you have to use that virtual folder mapping to create and produce a value URL mapping to the web site.
So, if you want a hyper link? Then want a link button or some such.
iFile='<%# "~/UpLoadFiles/" + Eval("InternalFileName")%>'>
or say this for a hyper link:
src ='<%# "~/UpLoadFiles/" + Eval("InternalFileName")%>'>
So you have to setup a Virtual folder to map out the windows land plane jane folder to that of a valid web based url.
So you could even place that in the SQL query
SELECT ID, [Name], Number1, Number2, File, '~/UpLoadFiles/' + File as FileUrl
FROM tblFiles
So, you now have a valid URL for that hyper link or link button.
You NOT shared how you setup file mapping for the web server, but just keep VERY clear in your mind that code behind STILL WILL ALWAYS use full standard path names. But the web site URL's are not - they are based on the path name of your web site. As noted, you could just place the files in a sub-folder of the root of the web site you have, and then you don't need a virtual folder. But for files outside of the site, or for security reasons, you often have the files and folders for these files NOT in the base web site folders. But adding a virtual (mapped) folder quite much means you can point/expose any valid folder system on the same network that the web server is running on.
I doubt that you allow URL's to drive c:\myfile.pdf.
I'm a bit confused about where the application should send a bunch of files. Let me suppose that an application accepts a number of images with NSOpenPanel at a time from the user. The application applies graphic filters to them. And it's now ready to save processed files. Before they forced us to sandbox applications, we were allowed to export processed files to application folder in Application Support without NSSavePanel. If you wanted to save files elsewhere, then you had to use NSSavePanel. If the application is sandboxed, it cannot send files to NSApplicationSupportDirectory/{app name}(which points to the containers folder assigned to this application)? My first sandboxed application was rejected a few days ago merely because a text field showed a path to container's application support folder. So if you have a bunch of files to export, you have to prompt the user to ask where to save each file? AppSandboxDesignedGuide, which Apple, Inc. has issued, has nothing to say exactly about where to save files except that it says "Your app has unrestricted read/write access to the container and its subdirectories." I think this PDF guide is a printed version of this web site. I'm asking this question here because I have some doubts and reviewers were often wrong at least when I submitted applications to them two years ago.
Thank you for your advice.
If the files are only for the application itself to use you can save the files in "Application Support/", which under the sandbox is under your container, just as before - just use the APIs to construct the path to that folder (and create it, it doesn't exist automatically, just as before).
If you are outputting files for the user to access then you don't put them in the container - that folder is meant to be hidden from ordinary users, though yours is the first time I've heard that even showing the path got you a rejection, but Apple are pretty random.
Here are three choices of where to put your files:
First is to ask the user. This is what you would normally do anyway, you shouldn't just dump files somewhere.
Second is a situation that the sandbox makes harder - when where the file should is is implicit, e.g. a graphic conversion program might sensibly output the converted file with the same name but different extension in the same folder as the original. This was finally addressed by Apple around 10.8.3 or something with "Related Items" - Apple's docs for this are here. Essentially in the Document Types in the Info.plist you must list all the extensions you handle - both in and out - and add a NSIsRelatedItemType key with the value of YES to all those you might convert between. E.g. For TextEdit .rtf, .rtfd and .txt are flagged in this way so TextEdit can open as one format and save as another.
Third, if you wish to put all your files in one location, say in a "Converted Items" folder. Then you ask the user once to specify this folder and then save a security-scoped bookmark to that folder in your applications defaults or elsewhere in your app's container. On subsequent executions you can access this bookmark and regain access to the folder. For an introduction to this start with Apple's Security-Scoped Bookmarks and Persistent Resource Access. This is really no harder than pre-sandbox as any decent app would always ask for the location of the folder from the user, the difference is the need to save the security-scoped bookmark so the user doesn't need to give permission every time.
HTH.
For an ios app I try to develop, I need to get a listing of the content of an online directory. (My app works with local directories but I'm trying to edit it so it works with an online directory)
I've been looking into this a lot but I can't manage to find the best solution to do this. (I learned you can't read the contents of an online directory and subdirectories with objective-c and print them to an array to display them in, for example, a tableview).
I did learn how to create a connection and output the html of a certain page (or an xml file). That's why I was wondering... Is there a way (webservice?) to generate an xml/JSON/html file that prints the content of a directory (if possible also subdirectories)? The generation of this xml/JSON/html has to be done at runtime, the moment my app asks for the file, since people will be able to add files (pdf's, video's,...) to the directory via FTP. (editing the xml/JSON file everytime is not an option).
Any help would be very much appreciated.
You could create a simple webservice in many ways. For example, with PHP you could write something like this:
<?php
$dir = '/myDir';
$files = scandir($dir);
echo json_encode($files);
?>
Then just point your app to get the contents of that page and parse the JSON.
scandir: http://php.net/manual/en/function.scandir.php
json_encode: http://php.net/manual/en/function.json-encode.php
I have a Search web reference (from a Search Server 2010 express install) in a vb.net application that is utilizing the QueryService Class to search a production Sharepoint foundation 2010 site.
At a previous point in time, we had created a proof of concept on an entirely test system that has since been turfed. From my recollection on this test system when documents were uploaded as a specific site content type (that inherits from document) and metadata was provided, we could search for specific metadata by making managed properties for each, and search results would be returned as documents (with the isdocument flag set to true). Viewing the document then became simple, as we could simply use the filename and path to display the stored file.
Now we are developing a production system and we have encountered a new behavior, where these results now are returned as aspx results such as
http://digitizaton/Company/Client Documents/Forms/DispForm.aspx?ID=1703
This of course makes it terribly difficult to locate and view the document, we can extract the Title which will then give us the name of the file with no extension, but that hardly helps, as the FileExtension data is aspx, not the documents file extension, so we don't have a full filename. We could display the page returned as a result, but would much prefer the document itself.
I've made a test document library, with just bare bones setup, (not using the site content type, or site columns) and uploaded some documents on the same site, and they are returned in the same fashion, so I don't believe the document library, or content type are the issue.
With a fairly limited understanding of both Sharepoint and Search Server, I don't know if this is a setup issue with the search service itself, with the site configuration, or with the querypacket I am sending. We also have a third party application (Knowledgelake) installed on the server that ties into sharepoint which could have changed configuration somewhere as well?
I don't think the query packet has changed since it was working in the proof of concept, other than the custom data column names. I will provide it here in case there is something glaringly obvious to an external reader.
<QueryPacket xmlns='urn:Microsoft.Search.Query.Document'>"
<Query>
<SupportedFormats>
<Format>urn:Microsoft.Search.Response</Format>
</SupportedFormats>
<Range>
<Count>0</Count>
</Range>
<Context>
<QueryText type='MSSQLFT'>
SELECT Filename, Title, FileExtension, IsDocument, Path from Scope() WHERE ""Scope"" = 'Department1' AND CustomData = 'X' --
</QueryText>
</Context>
Any guidance would be incredibly appreciated. If I have not provided some relevant information, please let me know and I can track it down.
Thanks everyone
So now I feel like an idiot, I've searched for hours with no luck, and literally seconds after composing this post, I find the nugget of gold I've been searching for.
It appears that our primary file type, PDF, has a known issue with Sharepoint 2010, as shown at the following site.
http://www.sharepointsharon.com/2010/03/sharepoint-2010-and-adobe-pdf/
and further to that, this registry entry setting is required to link it all together
http://www.mossgurus.com/adnan/Lists/Categories/Category.aspx?Name=SharePoint%202010%20--%20Configuration
I have a client that's been struggling with slow loading pdf files on the web.
My client has some very large pdf files that are almost 10 Mb. They take upwards of 3-4 minutes to download. The files will not display until the whole file is loaded.
We and they have seen other's sites where the pdfs load one page at a time, so the end user can start looking at the file as the rest of the page is still loading in the background. Gives the illusion that the page has loaded faster.
According to the documentation they see, IIS 6 should automatically do this if the pdf file is created with “Optimized for fast web view” checked. It is checked, and the file will still not load a page at a time.
They have searched and found nothing other than IIS will do this automatically if the file is saved correctly.
How can they "stream" the pdf? Is this because the pdf's were saved in a special way? Is this a java script that handles the download? Or is there a change that needs to happen in IIS?
Thanks
Update:
The file starts out like this:
%PDF-1.4
%âãÏÓ
171 0 obj << 0/Linearized 1
Linearized?
The PDF document isn't being served up from an aspx/asp page. (It's just posted directly to the site and linked to).
You need to lineraize the PDF and not trust IIS to do this for you.
There are a number of apps that will do this for you. I have used CVision (thier compression is 2nd to none, but the licensing and SDK are a pain), there is also some cheaper alternatives here, but I dont know how well they work.
To clarify Tony's point... (I think)
If you have actually used these tools and your pdf is linearized, try converting the PDF to a byte array and Response.Write() the byte array (with content headers, etc) to the client (in a new browser window or frame)
Would it be possible to use a third party service, like Scribd? If you go this route you can embed their streaming viewer onto your client's website. Just a thought, although I know it's not really suitable for every type of business.
This might happen if you are serving the PDF from an aspx page, to get the byte-serving that linearized pdf's need the page needs to be served directly or you need to provide the byte serving from the aspx code.
Save one of the files and open it up in a text editor. If you don't see something like
<< /Linearized 1.0 /L <number> /H [<number> <number>] /O <number> /E <number> ...
in the first couple hundred bytes or so, then you're not getting a linearized (ie, fast web) PDF.
First, the document needs to be "linearized", as others have explained; you can linearize it in Acrobat or using pdfopt from Ghostscript. Second, the web server must be able to serve byte ranges (i.e., support the Range header); I have no idea how to configure IIS for this, but even if the document is linearized, the client has to be able to read particular byte ranges.