What is the best practice approach to local user generated content when using Microsoft WebDeploy and Team City to deploy fixes to a site?
Using the deployment process described by Troy Hunt:
http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity_26.html
When changes are made to a site the WebDeploy agent updates the site including removing old files that are no longer needed - which is great. However in the case where a site contains user generated data (say users can upload an image which is stored as a file on disk or a simple CMS where page content files can be updated by the user), what is the best practice to prevent these files being deleted by the deployment agent?
Is there an ignore flag for certain folders?
Should the user files be stored outside the root of the deployed website (Is this a security risk)?
You basically need to use MSDeploy's skip rules. This will tell MSDeploy to ignore certain files, folder, or subfolders etc.
It depends on where you implement these to what the syntax will look like. But you have the following options:
If your publishing through VS.Net using a publishing profile you can include skip rules here (I've taken this approach and seen it work fine). This SO question should point you in the right direction - MSDeploy skip rules when using MSBuild PublishProfile with Visual Studio 2012
If your using a vs.net web solution (website / web application) I later found out you can also implement skip rules in the web.config. Although the following article is a bit old the approach may still be viable - How to write skip and replace rules for MSDeploy (I havent used, or tested this approach)
Last, but not least, you could use MSDeploy skip rule on the command line itself. So assuming you execute msdeply directly (as opposed to via msbuild) you would need to append a skip parameter with the relevant attributes you require. Further information can be found at: Demystifying MSDeploy skip rules or Web Deploy Operation Settings (Look for the skip command reference, about 2/3 down the page) (Using publishing profiles with MSBuild ultimately makes this call for you, i've seen it in action working by using the first approach above).
Hope that helps!
Related
We have a product which is developed using SharePoint foundation 2010. We are hosting on port 80 and which was stand alone installation of the MSF 2010. Now i need to deploy the same product on different ports for different clients in the farm environment.Right now i am facing some challenges as follows:
we have css, images, jquery files which are mapped to 14 hive folders. Since we were deploying in standalone environment, entire 14 hive folders were serving for one client. Now challenge is, how to manage these css, jquery, images folders in 14 hive folder for different clients(different web applications). If i do small change in one js file(test.js) for one particular file, it should affect for that client itself. It should not affect on other client's js(test.js) file.
Since we are having only one wsp solution, and we need to add this to solution repository. Since sharepoint allows only solutions whose names and GUID are unique. So, if i do some change in one solution for particular client , when i upload it to solution repository , it will not allow, since old wsp is already existing. So, how to manage wsp to serve for multiple clients ? . if i do changes for one client, it should not affect on other clients.
please let me know the best practice for this.
Given that a feature can be activated and deactivated at a SPSite or SPWeb level, there could be a more elegant solution to this situation.
Instead of packaging your images, CSS and JS files to be deployed to the 14 Hive, you could actually have them deploy to a new Document Library that your feature creates in the SPSite/SPWeb when your feature is activated. You can also configure your WSP such that the JS, CSS and Image files get uploaded to the document library so created.
This way one of your problems is solved wherein you can modify the files for one client manually directly through the document library and it won't affect other websites/clients.
Also, in this configuration, your single WSP will be sufficient and should not conflict since you won't be deploying a whole new WSP when you change some code for just one client.
I found this blogpost that shows you how to achieve the above: http://jcapka.blogspot.com/2011/05/deploying-document-library-including.html
I also would recommend this series to know more about different options of packaging things to your WSP: http://blogit.create.pt/blogs/andrevala/archive/2007/12/02/SharePoint-2007-Deployment_3A00_-Overview.aspx
Its written for SharePoint 2007 but most of it hasn't changed in 2010.
If there are any challenges in the above approach, you can probably find workarounds for it.
When you deploy solutions to SharePoint 2010, they are generally packaged as WSP, which are generally named as business area.
If you are using Visual Studio, Open "View -> Other Windows -> Package Explorer" to explore contents of WSP.
So if your clients have different html, css, javascript, consider packaging them in different WSPs.
One solution per client
This way under 14 hive, each solution (WSP) items will be contained in their respective folders, which means changes in css of one solution won't affect other solutions.
If you have application pages, they can go:
14\TEMPLATE\LAYOUTS\Client_1\
14\TEMPLATE\LAYOUTS\Client_1\css
14\TEMPLATE\LAYOUTS\Client_1\javascript
14\TEMPLATE\LAYOUTS\Client_2\
14\TEMPLATE\LAYOUTS\Client_2\css
14\TEMPLATE\LAYOUTS\Client_2\javascript
14\TEMPLATE\LAYOUTS\Client_3\
14\TEMPLATE\LAYOUTS\Client_3\css
14\TEMPLATE\LAYOUTS\Client_3\javascript
If you have client specific webparts, they can go:
14\TEMPLATE\CONTROLTEMPLATES\Client_1\Webpart1
14\TEMPLATE\CONTROLTEMPLATES\Client_2\Webpart1
14\TEMPLATE\CONTROLTEMPLATES\Client_3\Webpart1
All you need is to create well defined directory structure in your WSPs.
I have followed the imageresizer's troubleshooting for when images do not appear as they are listed here:
Potential causes:
You are using the .jpg.ashx syntax, and you did not register the HttpModule properly in both places of your Web.config file.
You are using the .jpg.ashx syntax, but you're not using a query string. You should drop the '.ashx' unless you actually want to process the file.
You are using ASP.NET MVC, but do not have the MvcRoutingShim plugin installed.
You have Precompilation enabled, and are using an image provider. This is caused by a long-standing bug in the .NET framework.
However, everything works perfectly in my local development environment. I have the MvcRoutingShim plugin installed and all the correct web.config references. On the live site, all images referenced by either just a querystring OR the RemoteReaderPlugin.Current.CreateSignedUrl procedure are not found.
All images are resulting in a 404 error.
Any ideas?
The answer: DiskCache(ConfigurationError): Not working: Your NTFS Security permissions are preventing the application from writing to the disk cache
Please give user read and write access to directory "C:\inetpub\wwwroot{site}\imagecache" to correct the problem. You can access NTFS security settings by right-clicking the aformentioned folder and choosing Properties, then Security.
In Visual Studio 2012, using publish profiles along with web deploy simplifies the deployments quite a bit. However it still is missing few things or may be I don't know how to use it yet.
I prefer to use the NTLM authentication without storing the username and password (especially) in the publish profiles. How can this be done? If I leave the username and password empty, I am prompted for it. Is there a way like manually modifying the .pubxml files?
Why is the username/password stored in PublishProfileName.pubxml that I have checked in the source control and not in PublishProfileName.pubxml.user that is local to each user? I could at least save the username but obviously don't want that to be checked in.
The Configuration itself is not part of PublishProfileName.pubxml but is stored in PublishProfileName.pubxml.user as LastUsedBuildConfiguration.
Same for the Platform as last point.
I am also missing support for multi-server deployments. I am currently forced to use batch files in addition to Publish Profiles.
EDIT
The command line that works fine for publishing is
MSBuild.Exe MyProject.sln /p:Configuration=QA /p:DeployOnBuild=true;PublishProfile=PublishToQA;AllowUntrustedCertificate=true /p:authType=NTLM /p:UserName=
In this I would like to omit the /p:Configuration=QA if the configuration becomes part of the publish profile itself.
Some answers to your questions.
I prefer to use the NTLM authentication without storing the username and password (especially) in the publish profiles. How can
this be done? If I leave the username and password empty, I am
prompted for it. Is there a way like manually modifying the .pubxml
files?
Your authentication is typically driven by how Web Deploy is hosted. By default if you are using the Web Management Service then you are using IIS users for auth. With IIS users you can control which users have permissions to specific sites/apps. You can configure WMSVC to use windows auth as well though. If you have issues using VS for those scenarios let me know.
If you are using the Remote Agent service to host Web Deploy then in this case you'll be using windows auth.
Why is the username/password stored in PublishProfileName.pubxml that I have checked in the source control and not in
PublishProfileName.pubxml.user that is local to each user? I could
at least save the username but obviously don't want that to be checked
in.
We have another mechanism for you to determine what information is private/shared. With the exception of the password all publish info is shared (and checked in by default). In order to simplify the design you can either have a publish profile which is shared, or one which is not shared at all. There is no in-between in which you have a profile that some fields are shared and other not. Password is special cased here and encrypted on a per-user/per-machine basis in the .pubxml.user file.
If you'd like to have a private publish profile then you can simply not check in the .pubxml file which corresponds to the publish profile. These are stored in the Properties\PublishProfiles (or My Project\PublishProfiles for VB) and just exclude them from the project and don't check the files in. The publish dialog looks for the profiles on disk, not just the ones which are in the project. Everything should continue to work.
We don't support the concept of selectively storing values in the .pubxml.user file. The publish dialog will only store a set number of values in that file. Instead of
The Configuration itself is not part of PublishProfileName.pubxml but is stored in
PublishProfileName.pubxml.user as LastUsedBuildConfiguration.
Same for the Platform as last point.
This was a mistake it should have been stored in the .pubxml file, not the .pubxml.user file. We have since fixed this, but haven't had a chance to release the update yet.
The Configuration property cannot be set in the publish profile. The Configuration property is a core part of the build process. To be more specific, the reason why we didn't call this property Configuration is because the .pubxml file is imported into the definition of the .csproj/.vbproj during a build & publish. Since other properties are defined based on Configuration you cannot change the value once it's been set. I just blogged with way too much detail on this subject at http://sedodream.com/2012/10/27/MSBuildHowToSetTheConfigurationProperty.aspx. This limitation is an MSBuild thing not a publish limitation. For command line you should specify Configuration in the following way:
msbuild.exe myproj.csproj /p:...(other properties)... /p:Configuration=
I am also missing support for multi-server deployments. I am currently forced to use batch files in addition to Publish Profiles.
We don't have direct support for this, but if you expand on your needs I may be able to help. FYI I have an extension which you may be interested in. I have posted a 5 min video to http://sedodream.com/2012/03/14/PackageWebUpdatedAndVideoBelow.aspx.
You are free (and encouraged) to manually edit your pubxml files, so feel free to remove the password.
To switch to NTLM, change AuthType to NTLM in the first PropertyGroup.
Platform and Configuration remain build configuration, the user file just stores them so Visual Studio knows what the last configuration you deployed was.
By multi-server, do you mean a web farm? If so, you might try looking at the Web Farm Framework which basically performs MSDeploy syncs from the primary server to the others.
Alternatively, you could switch to the command line and use postSync to upload and execute a batch file on the remote server that triggers the other deployments from there.
When I deploy my web site I need to:
Take the site offline.
run sql to update the database.
update the web site.
Put the site back online
I would like to script as much of this as possible without have to go the server and make manual changes.
I am currently using WebDeploy to push my website changes. It works great, and I don't have to know exactly where my site is installed on the server.
So what is the best way to put up/take down the app_offline file? If have seen some solutions where you rename an existing file ( app_offline.htm_ ) using an msbuild script. But it seems like that would require me to know the location of this file.
Is there a way to do this through web_deploy?
Thanks
Yes, you can enable AppOffline during a WebDeploy deployment you can even customize the AppOffline template:
msdeploy.exe -verb:sync -source:iisApp=sourceApp -dest:iisApp=destApp,appOfflineTemplate="offlineTemplate.htm" -enablerule:AppOffline
https://blogs.iis.net/msdeploy/webdeploy-3-5-rtw
Where would you write an error log file, say ErrorLog.txt, in Windows? Keep in mind the path would need to be open to basic users for file write permissions.
I know the eventlog is a possible location for writing errors, but does it work for "user" level permissions?
EDIT: I am targeting Windows 2003, but I was posing the question in such a way as to have a "General Guideline" for where to write error logs.
As for the EventLog, I have had issues before in an ASP.NET application where I wanted to log to the Windows event log, but I had security issues causing me heartache. (I do not recall the issues I had, but remember having them.)
Have you considered logging the event viewer instead? If you want to write your own log, I suggest the users local app setting directory. Make a product directory under there. It's different on different version of Windows.
On Vista, you cannot put files like this under c:\program files. You will run into a lot of problems with it.
In .NET, you can find out this folder with this:
Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData)
And the Event Log is fairly simple to use too:
http://msdn.microsoft.com/en-us/library/system.diagnostics.eventlog.aspx
Text files are great for a server application (you did say Windows 2003). You should have a separate log file for each server application, the location is really a matter of convention to agree with administrators. E.g. for ASP.NET apps I've often seen them placed on a separate disk from the application under a folder structure that mimics the virtual directory structure.
For client apps, one disadvantage of text files is that a user may start multiple copies of your application (unless you've taken specific steps to prevent this). So you have the problem of contention if multiple instances attempt to write to the same log file. For this reason I would always prefer the Windows Event Log for client apps. One caveat is that you need to be an administrator to create an event log - this can be done e.g. by the setup package.
If you do use a file, I'd suggest using the folder Environment.SpecialFolder.LocalApplicationData rather than SpecialFolder.ApplicationData as suggested by others. LocalApplicationData is on the local disk: you don't want network problems to stop you from logging when the user has a roaming profile. For a WinForms application, use Application.LocalUserAppDataPath.
In either case, I would use a configuration file to decide where to log, so that you can easily change it. E.g. if you use Log4Net or a similar framework, you can easily configure whether to log to a text file, event log, both or elsewhere (e.g. a database) without changing your app.
The standard location(s) are:
C:\Documents and Settings\All Users\Application Data\MyApp
or
C:\Documents and Settings\%Username%\Application Data\MyApp
(aka %UserProfile%\Application Data\MyApp) which would match your user level permission requirement. It also separates logs created by different users.
Using .NET runtime, these can be built as:
AppDir=
System.Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData)
or
AppDir=
System.Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)
followed by:
MyAppDir = IO.Path.Combine(AppDir,'MyApp')
(Which, hopefully, maps Vista profiles too).
Personally, I would suggest using the Windows event log, it's great. If you can't, then write the file to the ApplicationData directory or the ProgramData (Application Data for all users on Windows XP) directory.
The Windows event log is definitely the way to go for logging of errors. You're not limited to the "Application" log as it's possible to create a new log target (e.g. "My Application"). That may need to be done as part of setup as I'm not sure if it requires administrative privileges or not. There's a Microsoft example in C# at http://support.microsoft.com/kb/307024.
Windows 2008 also has Event Log Forwarding which can be quite handy with server applications.
I agree with Lou on this, but I prefer to set this up in a configuration file like Joe said. You can use
file value="${APPDATA}/Test/log-file.txt"
("Test" could be whatever you want, or removed entirely) in the configuration file, which causes the log file to be written to "/Documents and Settings/LoginUser/Application
Data/Test" on Windows XP and to "/Users/LoginUser/AppData/Roaming/Test on Windows Vista.
I am just adding this as I just spent way too much time figuring how to make this work on Windows Vista...
This works as-is with Windows applications. To use logging in web applications, I found Phil Haack's blog entry on this to be a great resource:
http://haacked.com/archive/2005/03/07/ConfiguringLog4NetForWebApplications.aspx
%TEMP% is always a good location for logs I find.
Going against the grain here - it depends on what you need to do. Sometimes you need to manipulate the results, so log.txt is the way to go. It's simple, mutable, and easy to search.
Take an example from Joel. Fogbugz will send a log / dump of error messages via http to their server. You could do the same and not have to worry about the user's access rights on their drive.
I personally don't like to use the Windows Event Log where I am right now because we do not have access to the production servers, so that would mean that we would need to request access every time we wanted to look at the errors. It is not a speedy process unfortunately, so your troubleshooting is completely haulted by waiting for someone else. I also don't like that they kind of get lost within the ones from other applications. Sure you can sort, but it's just a bit of a nucance scrolling down. What you use will end up being a combination of personal preference coupled along with limitations of the enviroment you are working in. (log file, event log, or database)
Put it in the directory of the application. The users will need access to the folder to run and execute the application, and you can check write access on application startup.
The event log is a pain to use for troubleshooting, but you should still post significant errors there.
EDIT - You should look into the MS Application Blocks for logging if you are using .NET. They really make life easy.
Jeez Karma-killers. Next time I won't even offer a suggestion when the poster puts up an incomplete post.