Our application calls out-of-the-box Office 365 Management API to retrieve activities and events on files stored in SharePoint Online. However per our experiment, the application can’t seem to retrieve not enough logs.
Example: We upload 1000 files to document library in Sharepoint Online. We receive 8 subscriptiona. Each subscription, we only get maximum 100 logs. Total call API get logs to retrieve 600 logs. Not enough!
Here my code to get subscription
List<SubscriptionsContent> GetSubscriptionsContents(AuthenticationResult authenticationResult, ManagementAPI m, DateTime startDate, DateTime endDate, bool proxyRequired = false)
{
try
{
string jsonSubscription = string.Empty;
string url = string.Empty;
string logType = "Audit.SharePoint";
if (authenticationResult != null)
{
url = string.Format(UrlFormat, m.TenantId, string.Format("subscriptions/content?contentType={0}&startTime={1}&endTime={2}", logType, startDate.ToUniversalTime().ToString(DateFormat), endDate.ToUniversalTime().ToString(DateFormat)));
jsonSubscription = ExecuteRequest(url, HttpMethod.Get, authenticationResult);
//Log.Info("jsonSubscription:");
//Log.Info(jsonSubscription);
}
var listContent = Common.GetListSubscriptionsContent(jsonSubscription);
Log.Info("Common.GetListSubscriptionsContent(jsonSubscription); Count: " + (listContent != null ? listContent.Count.ToString() : "IS NULL"));
return listContent;
}
catch (Exception ex)
{
Log.Error(ex);
return new List<SubscriptionsContent>();
}
}
Here my code to execute Request
public string ExecuteRequest(string url, HttpMethod method, AuthenticationResult token)
{
var responseStr = "";
try
{
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpRequestMessage request = new HttpRequestMessage(method, url);
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token.AccessToken);
HttpResponseMessage response = client.SendAsync(request).Result;
Log.Info("ExecuteRequest(string url, HttpMethod method, AuthenticationResult token): response.StatusCode: " + response.StatusCode + " ; response.ReasonPhrase: " + response.ReasonPhrase + " ; response.RequestMessage: " + response.RequestMessage);
if (response.IsSuccessStatusCode)
{
responseStr = response.Content.ReadAsStringAsync().Result;
}
}
catch (Exception ex)
{
Log.Error(ex);
}
return responseStr;
}
Here my code to get audit log from each subscription
List<AuditLog> listAudit = new List<AuditLog>();
foreach (var item in listSubscription)
{
var jsonAudit = ExecuteRequest(item.ContentUri.ToString(), HttpMethod.Get, authenticationResult);
if (string.IsNullOrEmpty(jsonAudit))
continue;
var listAuditLog = Common.GetListAuditLog(jsonAudit);
}
Here my code to parser JsonString
public static List<AuditLog> GetListAuditLog(string jsonString)
{
try
{
return JsonConvert.DeserializeObject<List<AuditLog>>(jsonString);
}
catch (Exception ex)
{
Log.Error("public static List<AuditLog> GetListAuditLog(string jsonString)", ex.InnerException);
return new List<AuditLog>();
}
}
I think that you need to use the pagination header.
If the amount of data is too big, the API will return a header entry named NextPageUrl containing an address to be used to request the next page of results. This link (representing the query) will be available for 24 hours.
Ex.
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
NextPageUrl:https://manage.office.com/api/v1/{tenant_id}/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2015-10-01&endTime=2015-10-02&nextPage=2015101900R022885001761
So, if the response contains this header entry, just use the value of NextPageUrl to request more data.
Repeat the process until this header entry doesn't exists anymore.
You can find more information in the Office 365 Management API reference
I am using the below c# code to connect the FTP server. Sometimes while sending the files it is getting the Error Code 551 with Description: Exception caught in sending FTP: The remote server returned an error: (551) Page type unknown.
I don't why this is happening.
Can anyone tell me what the issue is?
private bool sendFTP(string sDestPath, string sFileName, string sUserName, string sPassword, string sDomain, bool isProxyUsed, string sProxy, int nProxyPort, byte[] sData)
{
try
{
NetworkCredential nCred;
if (!sDomain.Equals(String.Empty))
{
nCred = new System.Net.NetworkCredential(sUserName, sPassword, sDomain);
}
else
{
nCred = new System.Net.NetworkCredential(sUserName, sPassword);
}
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(sDestPath + "//" + sFileName);
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = nCred;
request.Proxy = new WebProxy();
if (isProxyUsed)
{
WebProxy p = new WebProxy(sProxy, nProxyPort);
p.Credentials = nCred;
WebRequest.DefaultWebProxy = p;
}
request.ContentLength = sData.Length;
Stream reqStream = request.GetRequestStream();
reqStream.Write(sData, 0, sData.Length);
reqStream.Close();
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
return true;
}
catch (Exception objEx)
{
// Error
EventLog.WriteEntry("STCCommon", "Exception Caught:" + objEx.Message, EventLogEntryType.Error);
throw new Exception("Exception caught in sending FTP: " + objEx.Message);
}
}
This err msg is the next one I get after resolving “NotSupportedException” as noted here
I don't even reach the break point in the server code (set on the first line of the method that should be getting called).
This is the relevant server code:
[Route("api/PlatypusItems/PostArgsAndXMLFileAsStr")]
public async void PostArgsAndXMLFileAsStr([FromBody] string stringifiedXML, string serialNum, string siteNum)
{
string beginningInvoiceNum = string.Empty; // <= Breakpoint on this line
string endingInvoiceNum = string.Empty;
XDocument doc = XDocument.Load(await Request.Content.ReadAsStreamAsync());
. . .
And the client (handheld, Compact Framework) code:
private void menuItem4_Click(object sender, EventArgs e)
{
GetAndSendXMLFiles("LocateNLaunch"); // There is a "LocateNLaunch.xml" file
}
private void GetAndSendXMLFiles(string fileType)
{
string serNum = User.getSerialNo();
string siteNum = User.getSiteNo();
if (serNum.Length == 0)
{
serNum = "8675309";
}
if (siteNum.Length == 0)
{
siteNum = "03";
}
string uri = string.Format("http://localhost:28642/api/PlatypusItems/PostArgsAndXMLFileAsStr?serialNum={0}&siteNum={1}", serNum, siteNum);
List<String> XMLFiles = HHSUtils.GetXMLFiles(fileType, #"\");
MessageBox.Show(XMLFiles.Count.ToString());
foreach (string fullXMLFilePath in XMLFiles)
{
MessageBox.Show(fullXMLFilePath);
RESTfulMethods.SendXMLFile(fullXMLFilePath, uri, 500);
}
}
public static string SendXMLFile(string xmlFilepath, string uri, int timeout) // timeout should be 500
{
MessageBox.Show(string.Format("In SendXMLFile() - xmlFilepath == {0}", xmlFilepath));
MessageBox.Show(string.Format("In SendXMLFile() - uri == {0}", uri));
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.KeepAlive = false;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
StringBuilder sb = new StringBuilder();
using (StreamReader sr = new StreamReader(xmlFilepath))
{
String line;
while ((line = sr.ReadLine()) != null)
{
sb.AppendLine(line);
}
byte[] postBytes = Encoding.UTF8.GetBytes(sb.ToString());
if (timeout < 0)
{
request.ReadWriteTimeout = timeout;
request.Timeout = timeout;
}
request.ContentLength = postBytes.Length;
request.KeepAlive = false;
request.ContentType = "application/xml";
try
{
Stream requestStream = request.GetRequestStream();
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
using (var response = (HttpWebResponse)request.GetResponse())
{
return response.ToString();
}
}
catch (Exception ex)
{
MessageBox.Show("SendXMLFile exception " + ex.Message);
request.Abort();
return string.Empty;
}
}
}
Running this code, I see from the client the following "debug strings":
0) "1" (from MessageBox.Show(XMLFiles.Count.ToString());)
1) "\Program Files\LocateNLaunch\LocateNLaunch.xml" (from MessageBox.Show(fullXMLFilePath);)
2) "In SendXMLFile() - xmlFilePath == \Program Files\LocateNLaunch\LocateNLaunch.xml" (from MessageBox.Show(string.Format("In SendXMLFile() - xmlFilepath == {0}", xmlFilepath));)
3) "In SendXMLFile() - uri == http://localhost:28642/api/PlatypusItems/PostArgsAndXMLFileAsStr?serialNum=8675309&siteNum=03" (from MessageBox.Show(string.Format("In SendXMLFile() - uri == {0}", uri));)
- and then this one from somewhere:
4) "SendXMLFile exception Unable to connect to the remote server"...
So what could be causing this inability to connect?
UPDATE
The same thing ("Unable to Connect to the Remote Server") happens with this code (different operation, but also from the WindowsCE/Compact Framework/handheld app that tries to connect to the Web API server app):
private void menuItem3_Click(object sender, EventArgs e)
{
string serNum = User.getSerialNo();
if (serNum.Length == 0)
{
serNum = "8675309";
}
string clientVer =
HHSUtils.GetFileVersion(#"\Application\sscs\vsd_setup.dll");
if (clientVer.Contains("Win32Exception"))
{
clientVer = "0.0.0.0";
}
MessageBox.Show(string.Format("After call to GetFileVersion(), serial num == {0};
clientVer == {1}", serNum, clientVer));
string uri =
string.Format("http://localhost:28642/api/FileTransfer/GetHHSetupUpdate?
serialNum={0}&clientVersion={1}", serNum, clientVer);
RESTfulMethods.DownloadNewerVersionOfHHSetup(uri);
}
public static void DownloadNewerVersionOfHHSetup(string uri)
{
string dateElements = DateTime.Now.ToString("yyyyMMddHHmmssfff",
CultureInfo.InvariantCulture);
var outputFileName = string.Format("HHSetup_{0}.exe", dateElements);
try
{
var webRequest = (HttpWebRequest)WebRequest.Create(uri);
var webResponse = (HttpWebResponse)webRequest.GetResponse();
string statusCode = webResponse.StatusCode.ToString();
if (statusCode == "NoContent")
{
MessageBox.Show("You already have the newest available version.");
}
else
{
var responseStream = webResponse.GetResponseStream();
using (Stream file = File.Create(outputFileName))
{
CopyStream(responseStream, file);
MessageBox.Show(string.Format("New version downloaded to {0}",
outputFileName));
}
}
}
catch (WebException webex)
{
MessageBox.Show("DownloadNewerVersionOfHHSetup: " + webex.Message);
}
}
// I see the "After call to GetFileVersion()" message in menuItem3_Click() handler, but then "DownloadNewerVersionOfHHSetup: Unable to Connect to the Remote Server" in DownloadNewerVersionOfHHSetup()
And yes, the server app is running.
UPDATE 2
Here is the code that I tested prior to "dumbing it down" (retrofitting it, making it as similar as possible to this working test code, yet that may not be saying much) for Compact Framework:
Client code:
DownloadTheFile(textBoxFinalURI.Text); // with textBoxFinalURI.Text being
"http://localhost:28642/api/FileTransfer/GetUpdatedHHSetup?
serialNum=8675309&clientVersion=1.3.3.3" and the file on the server being
version 1.4.0.15
private void DownloadTheFile(string uri)
{
var outputFileName = "Whatever.exe";
try
{
var webRequest = (HttpWebRequest)WebRequest.Create(uri);
var webResponse = (HttpWebResponse)webRequest.GetResponse();
string statusCode = webResponse.StatusCode.ToString();
if (statusCode == "NoContent")
{
MessageBox.Show("You already have the newest available version.");
}
else
{
var responseStream = webResponse.GetResponseStream();
using (Stream file = File.Create(outputFileName))
{
CopyStream(responseStream, file);
MessageBox.Show(string.Format("New version downloaded to {0}",
outputFileName));
}
}
}
catch (WebException webex)
{
MessageBox.Show(webex.Message);
}
}
Server code:
public HttpResponseMessage GetHHSetupUpdate(string serialNum, string clientVersion)
{
HttpResponseMessage result;
string filePath = GetAvailableUpdateForCustomer(serialNum);
FileVersionInfo currentVersion = FileVersionInfo.GetVersionInfo(filePath);
if (!ServerFileIsNewer(clientVersion, currentVersion))
{
result = new HttpResponseMessage(HttpStatusCode.NoContent);
}
else
{
result = new HttpResponseMessage(HttpStatusCode.OK);
var stream = new FileStream(filePath, FileMode.Open);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType =
new MediaTypeHeaderValue("application/octet-stream");
}
return result;
}
private string GetAvailableUpdateForCustomer(string serialNum)
{
if (serialNum == "8675309")
{
return HostingEnvironment.MapPath(#"~\App_Data\HHSetup.exe");
}
else
{
return HostingEnvironment.MapPath(#"~\App_Data\HDP.exe");
}
}
// clientFileVersion is expected to be something like "1.4.0.15"
private bool ServerFileIsNewer(string clientFileVersion, FileVersionInfo serverFile)
{
Version client = new Version(clientFileVersion);
Version server = new Version(string.Format("{0}.{1}.{2}.{3}",
serverFile.FileMajorPart, serverFile.FileMinorPart,
serverFile.FileBuildPart, serverFile.FilePrivatePart));
return server > client;
}
... This code works fine (server code is the same; the client code has been "retrofied")
I can't use the code as-is because of the limitations of Compact Framework / Windows CE. As the title of this post makes clear, I'm not even able to connect to the server from there yet. Is it possible? If so, what needs to change in my client code (not the client code in Update 2, which works in newer versions of .NET, but the client code shown prior to there)?
It's a similar story with the other method that is also returning "Unable to connect to the remote server" - it works fine in "modern" code running in a test app, but once it's retrofitted (better word than refactored when "dumbing down" to Compact Frameworkerize the code).
UPDATE 3
I tried to get more info from the err msg with the code below (old line commented out), but this "rewards" me instead with a NullReferenceException:
catch (WebException webex)
{
//MessageBox.Show("DownloadNewerVersionOfHHSetup: " + webex.Message);
string msg = webex.Message;
string innerEx = webex.InnerException.ToString();
string resp = webex.Response.ToString();
string stackTrace = webex.StackTrace;
string status = webex.Status.ToString();
MessageBox.Show(
string.Format("Message: {0}; Inner Exception: {1}; Response: {2}; Stack Trace: {3}; Status: {4}", msg, innerEx, resp, stackTrace, status));
}
UPDATE 4
As I continued to get NREs, I commented out each subsequent line, one-by-one, until I now have this that runs:
//string innerEx = webex.InnerException.ToString();
//string resp = webex.Response.ToString();
//string stackTrace = webex.StackTrace;
string status = webex.Status.ToString();
MessageBox.Show(
//string.Format("Message: {0}; Inner Exception: {1}; Response: {2}; Stack Trace: {3}; Status: {4}", msg, innerEx, resp, stackTrace, status));
//string.Format("Message: {0}; Response: {1}; Stack Trace: {2}; Status: {3}", msg, resp, stackTrace, status));
//string.Format("Message: {0}; Stack Trace: {1}; Status: {2}", msg, stackTrace, status));
string.Format("Message: {0}; Status: {1}", msg, status));
...but all I get from it is Status of "ConnectFailure" (I already knew that).
UPDATE 5
This runs without an NRE:
string msg = webex.Message;
string innerEx = webex.InnerException.ToString();
string status = webex.Status.ToString();
MessageBox.Show(string.Format("Message: {0}; Status: {1}; inner Ex: {2}", msg, status, innerEx));
And this is what I see:
So why would the server actively refuse the connection?
BTW, ASAP I'm going to bountify this question, or will bountify the answerer after the fact*, with a bounty that would make even Long John Silver and Perro-Negro's eyes glimmer and gleam (cared they for geekCoin, that is).
For facts leading to the arrest and eviction of this bug.
PSYCHE! I changed my mind/there's been a mutiny on the bounty => the bountification will happen here instead.
UPDATE 6
This also (using the "raw" IP Address of the server machine) gives me an NRE:
string uri = string.Format("http://192.168.125.50:28642/api/FileTransfer/GetHHSetupUpdate?serialNum={0}&clientVersion={1}", serNum, clientVer);
...as does using the "friendly name" ("Platypus") of the machine in place of the IP Address.
The large problem I see here is the fact that you have localhost as your address. That's absolutely wrong. localhost means, effectively, "on the same machine as I am running" so unless you've somehow managed to get a async .NET 4.0 web service to run on your Windows CE device and your server code is running there, then this is most certainly not what you want.
If you're running on an emulator, it's still wrong. The emulator is, for all intents and purposes, a separate machine.
You must use the address of the server/PC where that web service is running. It must be a routable address, meaning if you're connected over USB then it's probably ppp_peer and not an IP address (well it resolves to a private address, but the name is easier to remember).
I am writing an MVC Web API the make async HttpWebRequest calls. I am getting 2 different exceptions. Below is the method I am using.
The 1st exception is: "This stream does not support seek operations." and it is happening on the responseStream.
The 2nd exception is: "timeouts are not supported on this stream" and that is happening on the MemoryStream content.
What am I doing wrong? I have been Googling but not really finding any solution.
Thanks,
Rhonda
private async Task GetHtmlContentAsync(string requestUri, string userAgent, string referrer, bool keepAlive, TimeSpan timeout, bool forceTimeoutWhileReading, string proxy, string requestMethod, string type)
{
//string to hold Response
string output = null;
//create request object
var request = (HttpWebRequest)WebRequest.Create(requestUri);
var content = new MemoryStream();
request.Method = requestMethod;
request.KeepAlive = keepAlive;
request.Headers.Set("Pragma", "no-cache");
request.Timeout = (Int32)timeout.TotalMilliseconds;
request.ReadWriteTimeout = (Int32)timeout.TotalMilliseconds;
request.Referer = referrer;
request.Proxy = new WebProxy(proxy);
request.UserAgent = userAgent;
try
{
using (WebResponse response = await request.GetResponseAsync().ConfigureAwait(false))
{
using (Stream responseStream = response.GetResponseStream())
{
if (responseStream != null)
{
await responseStream.CopyToAsync(content);
}
}
var sr = new StreamReader(content);
output = sr.ReadToEnd();
sr.Close();
}
}
catch (Exception ex)
{
output = string.Empty;
var message = ("The API caused an exception in the " + type + ".\r\n " + requestUri + "\r\n" + ex);
Logger.Write(message);
}
return output;
}
I fixed the issue by adding
content.Position = 0
before new StreamReader line. Now I just need to get it work with GZip compression.
Rhonda
I have a page containing links to some files.
I basically need to access the source of the page for parsing it then and obtaining all the hyperlinks to the files.
My code is something like this (some piece of code I've found in many places on the net ..):
"private static byte[] ReadImageFromUrl(string url)
{
var myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.Timeout = 10000;
WebResponse myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
List<byte> bytesList = new List<byte>();
using (var br = new BinaryReader(stream))
{
try
{
while (true)
{
var b = br.ReadByte();
bytesList.Add(b);
}
}
catch (Exception)
{}
br.Close();
}
myResp.Close();
return bytesList.ToArray();
}"
Now the problem is I get "System.Net.WebException: The remote server returned an error: (500) Internal Server Error." when calling "myReq.GetResponse()" - examining the error I see that the status is 'ProtocolError'.
The response property of the WebException object contains some server error ..(although when opening it from the browser it opens correctly) ...also when I call this function with the url of one of my files I get the same ProtocolError status, but the 404 error ...
Please give any hint how could I solve it... or any other possibility of accomplishing this task.
Thanks !
My new code after using Fiddler is:
private static byte[] ReadFileFromUrl(string url)
{
var myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.Accept = const_AcceptHeader;
myReq.Headers.Set(const_AcceptLanguageHeaderName, const_AcceptLanguageHeader);
myReq.UserAgent = const_AcceptUserAgentHeader;
myReq.CookieContainer = new CookieContainer();
myReq.KeepAlive = true;
myReq.Timeout = Int32.Parse(ConfigSettings.RequestPageTimeout) * 1000;
WebResponse myResp = null;
List<byte> bytesList = null;
myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
bytesList = new List<byte>();
using (var br = new BinaryReader(stream))
{
try
{
while (true)
{
var b = br.ReadByte();
bytesList.Add(b);
}
}
catch (Exception ex)
{
throw;
}
br.Close();
}
return bytesList.ToArray();
}
All variables that start with const_ are taken from Fiddler.
Well, I solved that using Fiddler ... I passed to my request object the headers as I have seen them in Fiddler ...& it worked, no error