SQL server 2008 running large query - out of memory - sql

I've got a 140mb SQL query to import data to a new database. I file > open the query, and try and execute it but I get:
Exception of type'System.outofmemoryexception' was thrown. (mscorlib)
I got 8gb ram, using <3gb so what can I do?
at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
at System.Text.StringBuilder..ctor(String value, Int32 startIndex, Int32 length, Int32 capacity)
at System.Runtime.InteropServices.Marshal.PtrToStringUni(IntPtr ptr)
at Microsoft.SqlServer.Management.UI.VSIntegration.ShellTextBuffer.GetText(Int32 startPosition, Int32 chars)
at Microsoft.SqlServer.Management.UI.VSIntegration.ShellTextBuffer.get_Text()
at Microsoft.SqlServer.Management.UI.VSIntegration.ShellCodeWindowControl.GetSelectedTextSpan()
at Microsoft.SqlServer.Management.UI.VSIntegration.Editors.ScriptEditorControl.GetSelectedTextSpan()
at Microsoft.SqlServer.Management.UI.VSIntegration.Editors.ScriptAndResultsEditorControl.OnExecScript(Object sender, EventArgs a)
Microsoft SQL Server Management Studio 10.50.1600.1
Microsoft Data Access Components (MDAC) 6.1.7600.16385
Microsoft MSXML 3.0 6.0
Microsoft Internet Explorer 9.0.8112.16421
Microsoft .NET Framework 2.0.50727.4952
Operating System 6.1.7600

This is a known issue reported on Microsoft Connect:
http://connect.microsoft.com/SQLServer/feedback/details/269566/sql-server-management-studio-cant-handle-large-files
If you take a look you can see some workarounds such as using SQL CMD to execute and editing in Notepad++.
It does not appear they will be fixing this however.

Related

Unable to deploy .dacpac to SQL'16. Method Not found: set_Encryption

I have a dacpac file that is used from SQL 2005 to SQL 2016 databases. Our developers have been using SQL 2016 for the past few months, so we are now moving to production with it.
When attempting to upgrade a database (no previous dacpackage) using our .dacpac file, it will give this error below
===================================
Method not found: 'Void Microsoft.SqlServer.TransactSql.ScriptDom.AlterTableAlterColumnStatement.set_Encryption(Microsoft.SqlServer.TransactSql.ScriptDom.ColumnEncryptionDefinition)'. (Microsoft.Data.Tools.Schema.Sql)
------------------------------
Program Location:
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.AlterStatementGenerator.GenerateForSimpleColumn(SqlSimpleColumn oldColumn, SqlSimpleColumn newColumn)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.AlterStatementGenerator.GenerateAlterTableForColumn(IModelElement oldElement, IModelElement newElement)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.AlterStatementGenerator.GenerateStatement(IModelElement oldElement, IModelElement newElement)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.Sql90AlterStatementGenerator.GenerateScriptDom(IModelElement oldElement, IModelElement newElement)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.DeploymentScriptDomGenerator.GenerateTableAlter(SqlTable sourceTable, SqlTable targetTable)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.DeploymentScriptDomGenerator.GenerateFragment(Int32 operation, IModelElement element)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.DeploymentScriptDomGenerator.GenerateSteps(Int32 operation, IModelElement element)
at Microsoft.Data.Tools.Schema.Sql.Deployment.OrderedStepGenerator.Add(DeploymentScriptDomGenerator scriptDom, IModelElement element, Int32 operation)
at Microsoft.Data.Tools.Schema.Sql.Deployment.Analyzers.PlanMediator.BuildDependencyOrderedSteps(Int32 operation, List`1 classOrder, List`1 operationOrder, Dictionary`2 changes, Boolean preserveGraphs, Dictionary`2& relating, Dictionary`2& related)
at Microsoft.Data.Tools.Schema.Sql.Deployment.Analyzers.PlanMediator.BuildOperations()
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.OnGeneratePlan()
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeploymentPlanGenerator.GeneratePlan(List`1 drops)
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeployment.BuildPlan()
at Microsoft.Data.Tools.Schema.Sql.Deployment.SqlDeployment.Controller.CreatePlan()
at Microsoft.SqlServer.Dac.DacServices.<>c__DisplayClass25.<CreateDeploymentArtifactGenerationOperation>b__23(Object operation, CancellationToken token)
at Microsoft.SqlServer.Dac.Operation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.ReportMessageOperation.Microsoft.SqlServer.Dac.IOperation.Run(OperationContext context)
at Microsoft.SqlServer.Dac.OperationExtension.Execute(IOperation operation, DacLoggingContext loggingContext, CancellationToken cancellationToken)
at Microsoft.SqlServer.Dac.DacServices.GenerateDeployReport(DacPackage package, String targetDatabaseName, DacDeployOptions options, Nullable`1 cancellationToken)
at Microsoft.SqlServer.Management.Dac.DacWizard.ReviewPlanPage.ActionStepCalculator_DoWork(Object sender, DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)
As far as I can tell, we don't have Encryption enabled at all on any of the tables. This is a fresh database. We've even tried recreating it from scratch, but we get the same error once we try to apply the dacpac file.
Does anyone know what I can do to get the server to recognize the set_Encryption method that it seems to be missing? Any suggestions are appreciated.
Edit: I've further determined that it is not a SQL 2016 issue, but rather a SQL Server Management Studio 2016 issue.
I was able to get around this error, by first running the dacpac through SqlPackage.exe. After that- I was able to publish through management studio.

Team Foundation Server 2013 Error TF400129: Verifying SQL connection

I'm trying to install TFS2013 (en_visual_studio_team_foundation_server_2013_with_update_4_x86_x64) on my machine (windows 8.1 x64)
I had a existing SQL instance installed and tried to use that one during my TFS installation but I kept on getting error messages about TFS not being able to connect to my SQL instance.
I started googling te error message and found guides about how you needed to config the SQL to be put on port 1433, add Firewall Rules,.. I tried literally everything, nothing worked.
So I decided to completely remove my SQL instance and install the SQL SERVER 2012 Express edition that comes with the TFS installer. The SQL installation completes successfully, but when configuring TFS again, the same error popped up.
So I went another step further and removed the SQL Server instance again and downloaded the SQL SERVER 2014 Express edition. Yet, same problem.
I've been looking around the web for the error but can't seem to find anything useful. Note that using a database on the SQL SERVER instance from code or connecting using the SQL Management Studio works like charm!
I started inspecting the full blown error log TFS provided me with and found this:
TF400129: TF400129: Error from readiness check: Verifying SQL
connection [Info #18:58:47.025] +-+-+-+-+-| Verifying SQL connection
|+-+-+-+-+- [Info #18:58:47.025] Starting Node: VSQLCONNECT [Info
#18:58:47.025] NodePath :
VINPUTS/Progress/Conditional/VSQLNOTLOCALDB/VSQLISRUNNING/VSQLCONNECT
> [Error #18:58:47.025] System.OverflowException: Arithmetic operation
resulted in an overflow. at
SNIOpenSyncExWrapper(SNI_CLIENT_CONSUMER_INFO* , SNI_ConnWrapper** )
at SNINativeMethodWrapper.SNIOpenSyncEx(ConsumerInfo consumerInfo,
String constring, IntPtr& pConn, Byte[] spnBuffer, Byte[]
instanceName, Boolean fOverrideCache, Boolean fSync, Int32 timeout,
Boolean fParallel) at
System.Data.SqlClient.SNIHandle..ctor(ConsumerInfo myInfo, String
serverName, Byte[] spnBuffer, Boolean ignoreSniOpenTimeout, Int32
timeout, Byte[]& instanceName, Boolean flushCache, Boolean fSync,
Boolean fParallel) at
System.Data.SqlClient.TdsParserStateObject.CreatePhysicalSNIHandle(String
serverName, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Byte[]&
instanceName, Byte[] spnBuffer, Boolean flushCache, Boolean async,
Boolean fParallel) at
System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo,
SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout,
Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean
integratedSecurity, Boolean withFailover)
...
For completeness, this is the error screen I can't pass:
I'm really at a dead end here. Ive been trying to get this fixed for days now and I come here as my last resort. Thanks in advance.

Loading a flat file into SQL Server 2008 R2 via BizTalk 2010

I am trying to find my way around BizTalk 2010. The scenario is simple: I need to take a flat file (delimited, positional, segmented etc...) and load it into a table in SQL Server 2008 R2.
I am trying to do this with a WCF-SQL adapter.
At this point I do not care for speed (i.e., it does not matter if there will be multiple inserts or one bulk insert, but in the long run I would like to be able to do a bulk insert), I just want to get something working.
I have the following done so far:
TestTable1 in table in SQL Server 2008 R2
InsertIntoTestTable stored procedure in SQL Server 2008 R2
Visual Studio Project with (a) a schema for the incoming Flat File, (b) Generated Schema for the SQL table, (c) a Map from the incoming flat file schema to the SQL file schema and (d) a pipeline to disassemble the flat file.
I have deployed this project to BizTalk and created a receive port for the flat file specifying the receive folder and inbound map. When I drop a test file into the receive folder, it disappears but TestTable1 is not loaded with any values.
What am I missing? Where does BizTalk place the test file? How can I set this up?
I have looked over a number of tutorials (https://msdn.microsoft.com/en-us/biztalk/gg153515, https://sandroaspbiztalkblog.wordpress.com/2012/08/20/biztalk-server-transform-text-files-flat-files-into-xml-a-practical-example-part-3/, ) but just don't understand what I am missing.
Any help will be greatly appreciated.
So I got it to work. Not sure how yet, but I tweaked the adapter settings and changed the value of Quantity in the input file from 1.000 to 1. I also now able to write to an xml file. The error that I was getting is as follows looks like it was having problems with "Quantity" data type:
The adapter failed to transmit message going to send port "ToSQL" with URL "mssql://.//TestDB?". It will be retransmitted after the retry interval specified for this Send Port. Details:"Microsoft.ServiceModel.Channels.Common.XmlReaderParsingException: The input data for the field/parameter "Quantity" is invalid according to the expected SqlDbType Int. ---> System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
at Microsoft.Adapters.Sql.MetadataHelper.ConvertXmlValueToDotNetObject(String xmlString, String fieldParameterName, SqlDbType sqlDbType, Int32 maxLength, Int32 precision)
--- End of inner exception stack trace ---
Server stack trace:
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.EndRequest(IAsyncResult result)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at System.ServiceModel.Channels.IRequestChannel.EndRequest(IAsyncResult result)
at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfClient`2.RequestCallback(IAsyncResult result)".
Based on your error:
Quantity" is invalid according to the expected SqlDbType Int.
'1.000' is not a valid int type.
you should convert it to a valid format such as "1"

VB.NET issue connecting to Oracle 11g

OS: Windows 7 64bit
VB: Visual Studio 2010
Oracle Client: 11g
I am developing an application that connects to an Oracle 11g server. I am able to connect to the Oracle server via Oracle SQL Developer, ODBC (in SYSWOW64), and in VB Server Explorer.
I am using the .NET reference Oracle.DataAccess located at: C:\Oracle\product\11.2.0\client_32\ODP.NET\bin\2.x\Oracle.DataAccess.dll
I have tried adding the .DLL files to my bin folder as suggested by other questions and that does not work.
Does anyone have any other suggestions?
When I run my application, I get the following:
Here are the details:
System.TypeInitializationException was unhandled
Message=The type initializer for 'Oracle.DataAccess.Client.OracleConnection' threw an exception.
Source=Oracle.DataAccess
TypeName=Oracle.DataAccess.Client.OracleConnection
StackTrace:
at Oracle.DataAccess.Client.OracleConnection..ctor(String connectionString)
at Orc_Test_1.Form1.Button1_Click(Object sender, EventArgs e) in c:\Visual Studio 2010\Projects\Orc_Test_1\Orc_Test_1\Form1.vb:line 9
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(IntPtr dwComponentID, Int32 reason, Int32 pvLoopData)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.OnRun()
at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.DoApplicationModel()
at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(String[] commandLine)
at Orc_Test_1.My.MyApplication.Main(String[] Args) in 17d14f5c-a337-4978-8281-53493378c1071.vb:line 81
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException: Oracle.DataAccess.Client.OracleException
DataSource=""
ErrorCode=-2147467259
Message=The provider is not compatible with the version of Oracle client
Number=-11
Procedure=""
Source=Oracle Data Provider for .NET
StackTrace:
at Oracle.DataAccess.Client.OracleInit.Initialize()
at Oracle.DataAccess.Client.OracleConnection..cctor()
InnerException:
VB.NET Code:
Imports System.Data
Imports Oracle.DataAccess.Client ' ODP.NET Oracle managed provider
Imports Oracle.DataAccess.Types
Public Class Form1
Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
Dim oradb As String = "Data Source=SERVER1;Persist Security Info=True;User ID=username;Password=password;Unicode=True" 'From Server Explorer
'Dim oradb As String = "Data Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=Server1.host.net)(PORT=1522))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=Server1))); USER ID = username;Password = password;" 'From TNSnames.ora
Dim conn As New OracleConnection(oradb) '<--- Error happens on this line
conn.Open()
Dim cmd As New OracleCommand
cmd.Connection = conn
cmd.CommandText = "select system_id from schema1.system_impacted where system_name = AWESOME"
cmd.CommandType = CommandType.Text
Dim dr As OracleDataReader = cmd.ExecuteReader()
dr.Read()
Label1.Text = dr.Item("system_id")
conn.Dispose()
End Sub
End Class
I feel your pain, just went through something similar in a deployment situation. You probably have multiple clients installed, and your environment is pulling dlls for older releases (even if you have a latest oracle.dataaccess.dll correctly referenced in your project). Fixing this on your dev environment is one thing, a prod deployment server is another. Not sure what your deployment situation is, but here's what worked for me.
After struggling with trying to upgrade odp.net in existing oracle home, adding new oracle home, etc., I found the easiest way to fix everything is to download the latest odac with xcopy deployment from Oracle, and follow the readme (and see here for an older article on this also). Basically you'll run an install.bat file to setup locally (in separate folder, mine was c:\oracle_odac), then change your project reference to point to the oracle.dataaccess.dll in this new folder (I used 4 instead of 2.x), and add the new folder's bin dirs to front of your path (c:\oracle_odac\bin and c:\oracle_odac\odp.net\bin\4). On your deployment server, you'll just need to copy the entire c:\oracle_odac folder over (via xcopy or however), and setup the path.
That said, I anxiously await the production release of the fully managed odp.net from Oracle (in beta now).
EDIT: Just to add that you can avoid messing with PATHs if you setup in your app or web config file the dllpath. For example:
<configuration>
...
<configSections>
<section name="oracle.dataaccess.client" type="System.Data.Common.DbProviderConfigurationHandler, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"/>
...
<oracle.dataaccess.client>
<settings>
<add name="DllPath" value="c:\oracle_odac\bin"/>
</settings>
</oracle.dataaccess.client>
...
This will override other settings such as registry or machine.config. And it will allow multiple odp.net configurations to exists peacefully, and allow each app to point to the version it needs on the same server.
Usually happens when you're building x64 application which is trying to pick up x32 drivers that doesn't really work.
What you do in this case (and it's quite painful):
Install x32 oracle client to oraclehome1
Install x64 oracle client to oraclehome2
Make sure your application build (x64, x32) is pointed to correct oraclehome. I don't have that code handy, but I believe it is something along the lines of "Home=oraclehome2" in the connection string or similar.
I removed all of the oracle clients that were available in the 'Programs' section of 'Control Panel'. Then I went into the registry searched for 'Oracle' and removed all instances/references of Oracle client(s). After 3 hours of registry removal and a reboot, I installed the 11g client and everything worked as expected.
Note: when searching the registry for "Oracle", there may be references to the VB version of Oracle, or a reference to Crytsal Reports driver for 'Oracle'. I did not remove these. So pay attention to the file location and/or the syntax of the registry item you are deleting.
/Reading. It works.
download ODAC 32/ ODAC 64 bit ,depending upon on the compatibility of OS, VS and oracle 11g .
Use oracle 11g 32,os 32 bit,oracle 32 bit or oracle 64 bit,os 64 bit and oracle 11g 64 bit.
In my experience, after trying tbone's suggestion, playing with PATH env variable and other unsuccessful plays, the solution that worked immediately was to save a copy of the 8 DLLs files listed below RIGHT ON THE SAME FOLDER of the vb.net executable, and then add a reference to Oracle.DataAccess.dll in the same folder.
C:\Users\Legarcia\Documents\Visual Studio 2010\Projects\LabRetention\LabRetention\bin\Debug>dir *.dll
Volume in drive C is OS
Volume Serial Number is 10EC-8C5D
Directory of C:\Users\Legarcia\Documents\Visual Studio 2010\Projects\LabRetention\LabRetention\bin\Debug
09/30/2008 07:57 PM 348,160 msvcr71.dll
10/01/2008 02:22 AM 520,192 oci.dll
10/01/2008 01:56 AM 352,256 ociw32.dll
08/12/2009 07:13 AM 1,011,712 Oracle.DataAccess.dll
09/18/2008 08:47 PM 1,130,496 orannzsbb11.dll
10/01/2008 01:48 AM 868,352 oraocci11.dll
10/01/2008 02:45 AM 117,604,352 oraociei11.dll
08/12/2009 07:21 AM 413,696 OraOps11w.dll
8 File(s) 122,249,216 bytes
0 Dir(s) 238,127,943,680 bytes free
When developing/testing/debuging, the set of DLLs is copied to ..\projects\{Your project}\bin\Debug
e.g.: [C:\Users\Legarcia\Documents\Visual Studio 2010\Projects\LabRetention\LabRetention\bin\Debug]
And when releasing the application for the users just copy the DLLs to the same dir of your EXE
This seems to isolate all other Ora client installations.
The connection string used in the VB.net app is created from the info in TNSNAMES.ORA file found on some other client directory.
Regards.
LE+

Castle Active Record NHibernate Sql Server Ce 3.5 (SqlServerCe) Problem Updating Schema

I am trying to Update Schema on SQL Server CE 3.5 using Active Record by Castle. I have encountered an error - exception on the bottom.
I believe it is not a new issue, but I could not find a walk around for this issue. When trying to Update Schema using Active Record I get an exception. It seems that there is a wrong GetSchema call. It should be System.Data.SqlServerCe.SqlCeConnection.GetSchema() instead of DbConnection.GetSchema(). I got this information from here.
I am looking for resolution, not just information that it is MS problem. It is my problem right now, and possibly other people as well.
Exception:
System.NotSupportedException: The method is not supported.
in System.Data.Common.DbConnection.GetSchema(String collectionName, String[] restrictionValues)
in NHibernate.Dialect.Schema.AbstractDataBaseSchema.GetTables(String catalog, String schemaPattern, String tableNamePattern, String[] types)
in NHibernate.Tool.hbm2ddl.DatabaseMetadata.GetTableMetadata(String name, String schema, String catalog, Boolean isQuoted)
in NHibernate.Cfg.Configuration.GenerateSchemaUpdateScript(Dialect dialect, DatabaseMetadata databaseMetadata)
in NHibernate.Tool.hbm2ddl.SchemaUpdate.Execute(Action`1 scriptAction, Boolean doUpdate)
SQL Server CE 4.0 implements GetSchema(). If you can't upgrade I guess you're SOL...