Managing trace files on Sql Server 2005 - sql-server-2005

I need to manage the trace files for a database on Sql Server 2005 Express Edition. The C2 audit logging is turned on for the database, and the files that it's creating are eating up a lot of space.
Can this be done from within Sql Server, or do I need to write a service to monitor these files and take the appropriate actions?
I found the [master].[sys].[trace] table with the trace file properties. Does anyone know the meaning of the fields in this table?

Here's what I came up with that is working pretty good from a console application:
static void Main(string[] args)
{
try
{
Console.WriteLine("CcmLogManager v1.0");
Console.WriteLine();
// How long should we keep the files around (in months) 12 is the PCI requirement?
var months = Convert.ToInt32(ConfigurationManager.AppSettings.Get("RemoveMonths") ?? "12");
var currentFilePath = GetCurrentAuditFilePath();
Console.WriteLine("Path: {0}", new FileInfo(currentFilePath).DirectoryName);
Console.WriteLine();
Console.WriteLine("------- Removing Files --------------------");
var fileInfo = new FileInfo(currentFilePath);
if (fileInfo.DirectoryName != null)
{
var purgeBefore = DateTime.Now.AddMonths(-months);
var files = Directory.GetFiles(fileInfo.DirectoryName, "audittrace*.trc.zip");
foreach (var file in files)
{
try
{
var fi = new FileInfo(file);
if (PurgeLogFile(fi, purgeBefore))
{
Console.WriteLine("Deleting: {0}", fi.Name);
try
{
fi.Delete();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
Console.WriteLine("------- Files Removed ---------------------");
Console.WriteLine();
Console.WriteLine("------- Compressing Files -----------------");
if (fileInfo.DirectoryName != null)
{
var files = Directory.GetFiles(fileInfo.DirectoryName, "audittrace*.trc");
foreach (var file in files)
{
// Don't attempt to compress the current log file.
if (file.ToLower() == fileInfo.FullName.ToLower())
continue;
var zipFileName = file + ".zip";
var fi = new FileInfo(file);
var zipEntryName = fi.Name;
Console.WriteLine("Zipping: \"{0}\"", fi.Name);
try
{
using (var fileStream = File.Create(zipFileName))
{
var zipFile = new ZipOutputStream(fileStream);
zipFile.SetLevel(9);
var zipEntry = new ZipEntry(zipEntryName);
zipFile.PutNextEntry(zipEntry);
using (var ostream = File.OpenRead(file))
{
int bytesRead;
var obuffer = new byte[2048];
while ((bytesRead = ostream.Read(obuffer, 0, 2048)) > 0)
zipFile.Write(obuffer, 0, bytesRead);
}
zipFile.Finish();
zipFile.Close();
}
fi.Delete();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
Console.WriteLine("------- Files Compressed ------------------");
Console.WriteLine();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
Console.WriteLine("Press any key...");
Console.ReadKey();
}
public static bool PurgeLogFile(FileInfo fi, DateTime purgeBefore)
{
try
{
var filename = fi.Name;
if (filename.StartsWith("audittrace"))
{
filename = filename.Substring(10, 8);
var year = Convert.ToInt32(filename.Substring(0, 4));
var month = Convert.ToInt32(filename.Substring(4, 2));
var day = Convert.ToInt32(filename.Substring(6, 2));
var logDate = new DateTime(year, month, day);
return logDate.Date <= purgeBefore.Date;
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
return false;
}
public static string GetCurrentAuditFilePath()
{
const string connStr = "Data Source=.\\SERVER;Persist Security Info=True;User ID=;Password=";
var dt = new DataTable();
var adapter =
new SqlDataAdapter(
"SELECT path FROM [master].[sys].[traces] WHERE path like '%audittrace%'", connStr);
try
{
adapter.Fill(dt);
if (dt.Rows.Count >= 1)
{
if (dt.Rows.Count > 1)
Console.WriteLine("More than one audit trace file defined! Count: {0}", dt.Rows.Count);
var path = dt.Rows[0]["path"].ToString();
return path.StartsWith("\\\\?\\") ? path.Substring(4) : path;
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
throw new Exception("No Audit Trace File in sys.traces!");
}

You can also set up SQL Trace to log to a SQL table. Then you can set up a SQL Agent task to auto-truncate records.

sys.traces has a record for every trace started on the server. Since SQL Express does not have Agent and cannot set up jobs, you'll need an external process or service to monitor these. You'll have to roll your own everything (monitoring, archiving, trace retention policy etc). If you have C2 audit in place, I assume you have policies in place that determine the duration audit has to be retained.

Related

Downloading a file from Google cloud storage is corrupted randomly

I am trying to download data Bigquery data through Google cloud storage. Am able to send data from BigQuery to GCS but when downloading data from GCS to load the files are corrupted randomly.
getObject.getMediaHttpDownloader().setDirectDownloadEnabled(true);
out = fs.create(pathDir, true);
getObject.executeMediaAndDownloadTo(out);
boolean match= ismd5HashValid(o.getMd5Hash(), pathDir);
and to check md5 checksum
private boolean ismd5HashValid(String md5hash, String path) {
org.apache.hadoop.fs.Path pathDir = new org.apache.hadoop.fs.Path(path);
org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
InputStream is = null;
try {
FileSystem fs = FileSystem.get(conf);
MessageDigest md = MessageDigest.getInstance("MD5");
is = fs.open(pathDir);
byte[] bytes = new byte[1024];
int numBytes;
while ((numBytes = is.read(bytes)) != -1) {
md.update(bytes, 0, numBytes);
}
byte[] digest = md.digest();
String result = new String(Base64.encodeBase64(digest));
Log.info("Source file md5hash {} Downloaded file md5hash {}", md5hash, result);
if (md5hash.equals(result)) {
Log.info("md5hash check is valid");
return true;
}
} catch (IOException e) {
// TODO Auto-generated catch block
Log.warn(e.getMessage(), e);
} catch (NoSuchAlgorithmException e) {
// TODO Auto-generated catch block
Log.warn(e.getMessage(), e);
} finally {
IOUtils.closeQuietly(is);
}
return false;
}

JAX-RS 2.0 MULTIPART_FORM_DATA file upload not library specific

I need to create a JAX-RS 2.0 client that posts a file and a couple of parameters using MULTIPART_FORM_DATA content type. (Don't need the service, just the client) I’ve seen some examples that depend on an specific implementation, like Jersey or RESTEasy, but I’d like not to bind my code to any... in particular, to Apache CXF (I am using WAS Liberty Profile). Any ideas on how to do it? Do I have to stick to some specific classes? If so, how can I do it using Apache CXF 3.0 (Liberty uses CXF for JAX-RS 2.0)
Thanks
[I currently cannot comment under the already written answer]
If someone is searching for the maven dependency of IMultipartBody from the answer of Anatoly:
<dependency>
<groupId>com.ibm.websphere.appserver.api</groupId>
<artifactId>com.ibm.websphere.appserver.api.jaxrs20</artifactId>
<version>1.0.39</version>
<scope>provided</scope>
</dependency>
Thanks to andymc12 from https://github.com/OpenLiberty/open-liberty/issues/11942#issuecomment-619996093
You can use this example how to implement it by using jax-rs 2.0 feature: https://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.nd.doc/ae/twlp_jaxrs_multipart_formdata_from_html.html this is almost working example (some statements should be wrapped in try-catch block, but you'll see when'll post it to IDE.
package com.example.jaxrs;
#POST
#Consumes("multipart/form-data")
#Produces("multipart/form-data")
public Response postFormData(IMultipartBody multipartBody) {
List <IAttachment> attachments = multipartBody.getAllAttachments();
String formElementValue = null;
InputStream stream = null;
for (Iterator<IAttachment> it = attachments.iterator(); it.hasNext();) {
IAttachment attachment = it.next();
if (attachment == null) {
continue;
}
DataHandler dataHandler = attachment.getDataHandler();
stream = dataHandler.getInputStream();
MultivaluedMap<String, String> map = attachment.getHeaders();
String fileName = null;
String formElementName = null;
String[] contentDisposition = map.getFirst("Content-Disposition").split(";");
for (String tempName : contentDisposition) {
String[] names = tempName.split("=");
formElementName = names[1].trim().replaceAll("\"", "");
if ((tempName.trim().startsWith("filename"))) {
fileName = formElementName;
}
}
if (fileName == null) {
StringBuffer sb = new StringBuffer();
BufferedReader br = new BufferedReader(new InputStreamReader(stream));
String line = null;
try {
while ((line = br.readLine()) != null) {
sb.append(line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
formElementValue = sb.toString();
System.out.println(formElementName + ":" + formElementValue);
} else {
//handle the file as you want
File tempFile = new File(fileName);
...
}
}
if (stream != null) {
stream.close();
}
return Response.ok("test").build();
}

get stream from multipartFileData, file in use

Hello im trying to access an image from a post request with MultipartFormDataStreamProvider.
So far it seems successful other than when i try to create a file steam from the local name, the file is then in use. How do i get the current open stream to read the file, or get the old stream to close?
*Note: Please ignore the not so great try catch.
What I have so far:
[ResponseType(typeof(AdminImage))]
public IHttpActionResult PostAdminImage([FromUri]AdminImage adminImage)
{
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
try
{
Request.Content.ReadAsMultipartAsync(provider);
foreach (MultipartFileData file in provider.FileData)
{
FileStream fs = new FileStream(file.LocalFileName, FileMode.Open);
adminImage.ImageContent = adminImage.ImageToByteArray(Image.FromStream(fs));
}
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
db.AdminImages.Add(adminImage);
db.SaveChanges();
}
catch (Exception ex)
{
return InternalServerError();
}
return CreatedAtRoute("DefaultApi", new { id = adminImage.Id }, adminImage);
}

Having Trouble with ObjectInputStream/OutputStream

I am having trouble with my programs ability to save my Maps to a file. Here are my two methods for writing and reading my maps and arraylist.
Here is my read method:
private void getData() throws IOException, ClassNotFoundException {
File f_Instructors = new File(PSLTrackerInfo.file + "instructors.brent");
File f_Students = new File(PSLTrackerInfo.file + "students.brent");
File f_Times = new File(PSLTrackerInfo.file + "times.brent");
if (f_Instructors.exists()) {
try (ObjectInputStream in = new ObjectInputStream(new
BufferedInputStream(new FileInputStream(f_Instructors)))) {
//Add theList back in
if (in.readObject() != null) {
TreeMap<Instructor, Set<Student>> read = null;
while(in.available() > 0) {
read = (TreeMap<Instructor, Set<Student>>)
in.readObject();
}
if (read != null) {
for (Instructor key : read.keySet()) {
System.out.println(key);
Set<Student> values = read.get(key);
PSLTrackerInfo.addInstructor(key, values);
}
System.out.println("Instructors Found! Reading...");
} else {
System.out.println("No instructor data saved.1");
}
} else {
System.out.println("No instructor data saved.2");
}
in.close();
}
}
//Add times back in
if (f_Times.exists()) {
try (ObjectInputStream in = new ObjectInputStream(new
BufferedInputStream(new FileInputStream(f_Times)))) {
if (in.readObject() != null) {
TreeMap<Student, ArrayList<Date>> readTimes = null;
while(in.available() > 0) {
readTimes = (TreeMap<Student, ArrayList<Date>>) in.readObject();
}
if (readTimes != null) {
for (Student key : readTimes.keySet()) {
System.out.println(key);
ArrayList<Date> values = readTimes.get(key);
PSLTrackerInfo.addTimes(key, values);
}
System.out.println("Dates Found! Reading...");
} else {
System.out.println("No dates saved.");
}
} else {
System.out.println("No dates saved.");
}
in.close();
}
}
//Add newStudents back in
if (f_Students.exists()) {
try (ObjectInputStream in = new ObjectInputStream(new
BufferedInputStream(new FileInputStream(f_Students)))) {
if (in.readObject() != null) {
ArrayList<Student> readStudents = null;
while (in.available() > 0) {
readStudents = (ArrayList<Student>) in.readObject();
}
if (readStudents != null) {
PSLTrackerInfo.setTheList(readStudents);
}
System.out.println("New students found! Reading...");
} else {
System.out.println("No new students data saved.");
}
in.close();
}
}
}
And Here is my Writing method:
private void saveData() {
System.out.println("Saving Data...");
File f_Instructors = new File(PSLTrackerInfo.file + "instructors.brent");
File f_Students = new File(PSLTrackerInfo.file + "students.brent");
File f_Times = new File(PSLTrackerInfo.file + "times.brent");
ObjectOutputStream out_Instructors = null;
ObjectOutputStream out_Students = null;
ObjectOutputStream out_Times = null;
try {
out_Instructors = new ObjectOutputStream(new
BufferedOutputStream(new FileOutputStream(f_Instructors)));
out_Students = new ObjectOutputStream(new
BufferedOutputStream(new FileOutputStream(f_Students)));
out_Times = new ObjectOutputStream(new
BufferedOutputStream(new FileOutputStream(f_Times)));
out_Instructors.writeObject(PSLTrackerInfo.getMap());
out_Times.writeObject(PSLTrackerInfo.getTimes());
out_Students.writeObject(PSLTrackerInfo.getList());
out_Instructors.flush();
out_Students.flush();
out_Times.flush();
out_Instructors.close();
out_Students.close();
out_Times.close();
} catch (IOException ex) {
Logger.getLogger(PrivateLessonsTrackerGUI.class.getName())
.log(Level.SEVERE, null, ex);
}
System.exit(0);
}
Sorry if it is a little confusing I have 3 files to save 3 different objects, if there is a way to save it into one file let me know but I just was getting a lot of errors that I couldn't figure out how to solve so this is what I ended up doing. Thanks for any help given.
To EJP: I tried this
TreeMap<Instructor, Set<Student>> read = null;
try {
read = (TreeMap<Instructor, Set<Student>>)
in.readObject();
} catch (EOFException e) {
System.out.println("Caught EOFException!");
}
And even when there was data in it when it was written to the file, I got an EOFException everytime.
readObject() doesn't return null unless you wrote a null. If you're using that as a test for end of stream, it is invalid. The correct technique is to catch EOFException.
You are calling it and throwing away the result if it isn't null, and then calling it again. The second call will throw EOFException if there isn't another object in the file. It won't give you the same result as the first call. It's a stream.
available() is also not a valid test for end of stream. That's not what it's for. See the Javadoc. Again, the correct technique with readObject() is to catch EOFException.

Adding new revision for document in DropBox through android api

I want to add a new revision to the document(Test.doc) in Dropbox using android api. Can anyone share me any sample code or links. I tried
FileInputStream inputStream = null;
try {
DropboxInputStream temp = mDBApi.getFileStream("/Test.doc", null);
String revision = temp.getFileInfo().getMetadata().rev;
Log.d("REVISION : ",revision);
File file = new File("/sdcard0/renamed.doc");
inputStream = new FileInputStream(file);
Entry newEntry = mDBApi.putFile("/Test.doc", inputStream, file.length(), revision, new ProgressListener() {
#Override
public void onProgress(long arg0, long arg1) {
Log.d("","Uploading.. "+arg0+", Total : "+arg1);
}
});
} catch (Exception e) {
System.out.println("Something went wrong: " + e);
} finally {
if (inputStream != null) {
try {
inputStream.close();
} catch (IOException e) {}
}
}
New revision is created for first time. When i execute again, another new revision is not getting created.