I'm using VB.NET to communicate with Magento via the API SOAP v1. I had it working fine until I got to a call that needed an associative array. I've tried dictionaries, hashtables, custom types, etc. I did read hashtables and dictionaries are not serializable. So I tried the custom type/object/class. Below is the error I received:
The type XXXX was not expected. Use the Xmlinclude or SoapInclude attribute to specify types that are not known statically.
So I've read a lot of posts in regards to the above error and I can't get anything to work. The error seems at least to say what I'm trying is possible if I do it right, but maybe that is not the case. I don't know much about SOAP, but I can see this never working since the web service might not know how to handle the object.
So my question is if it's possible to send a custom object to the Magento API. Or more broadly is it possible to get SOAP v1 to work with VB.NET. I know v2 is an option but I'm pretty familiar with v1 and already having it working in VB.NET other than this scenario.
Answering a question you didn't ask — but have you considered using the XML-RPC adapter for the V1 API? It exposes the same exact methods as the SOAP API, but since XML-RPC doesn't have the same strong concept of types that SOAP does you may be able to get a generic object through from .NET (said without being that familiar with .NET SOAP or XML-RPC libraries)
It took a lot of digging, but I have it all working. To start I used the PHP SoapClient and noted how it formed associative arrays in the soap request. You can trace requests and responses, pretty handy. From there I wrote my own soap client in VB.NET using a WebRequest object. In doing so I have full control over the xml being sent to the API.
Again the reason I went down this road is ultimately V2 was not working for me. For some reason not all parameters were making it to the API. That and the fact I'm pretty comfortable with V1 too. I've written several custom APIs.
I apologize for the brevity, but there is a lot that went into this. Probably most of my time was hitting multiple dead ends. If anyone wants specifics feel free email me.
EDIT:
Here is the php code I used to see how I needed to format the requests:
$client = new SoapClient('http://www.site.com/index.php/api/soap/?wsdl',array('trace' => TRUE));
$session = $client->login('user','api-key');
echo $client->__getLastRequest() ."\n\n";
echo $client->__getLastRequestHeaders() ."\n\n";
echo $client->__getLastResponse() ."\n\n";
echo $client->__getLastResponseHeaders() ."\n\n";
$result = $client->call($session, 'cataloginventory_stock_item.list','393');
echo $client->__getLastRequest() ."\n\n";
echo $client->__getLastRequestHeaders() ."\n\n";
var_dump($result);
$client->endSession($session);
Below is how to send the request using VB.NET. You will need to construct the XML/SOAP body using the php above as a guide. I made a class per API call which output the needed XML. You will need System.Net, System.Xml & System.IO. I used getSoapHeader() because there is some common XML that goes into a request. See next code section for more details:
Private Function makeSoapRequest(ByVal soapBody As String) As String
Dim req As WebRequest = WebRequest.Create(_soap_url)
Dim xml As String
xml = getSoapHeader() & soapBody
Dim buffer() As Byte = System.Text.Encoding.UTF8.GetBytes(xml)
req.ContentType = "text/xml; charset=utf-8"
req.Method = "POST"
req.Headers.Add("SOAPAction", "urn:Mage_Api_Model_Server_HandlerAction")
req.ContentLength = buffer.Length
Dim st As System.IO.Stream = req.GetRequestStream
st.Write(buffer, 0, buffer.Length)
st.Close()
Dim response As WebResponse
Try
response = req.GetResponse
Catch ex As WebException
response = ex.Response
End Try
st = response.GetResponseStream()
Dim reader As New StreamReader(st)
Dim responseFromServer As String = reader.ReadToEnd()
makeSoapRequest = responseFromServer
response.Close()
st.Close()
End Function
Below is the getSoapHeader() function. As noted the ns2 portion is only needed if you are using type="ns2:Map" which is what I needed for associative arrays:
Private Function getSoapHeader() As String
'ns2 is not always needed
getSoapHeader = "<?xml version=""1.0"" encoding=""UTF-8""?><SOAP-ENV:Envelope xmlns:SOAP-ENV=""http://schemas.xmlsoap.org/soap/envelope/"" xmlns:ns1=""urn:Magento"" xmlns:xsd=""http://www.w3.org/2001/XMLSchema"" xmlns:xsi=""http://www.w3.org/2001/XMLSchema-instance"" xmlns:SOAP-ENC=""http://schemas.xmlsoap.org/soap/encoding/"" xmlns:ns2=""http://xml.apache.org/xml-soap"" SOAP-ENV:encodingStyle=""http://schemas.xmlsoap.org/soap/encoding/""> " & vbCrLf
End Function
Related
return webClient//
.post()//
.uri(whatever.com)
.header("Authorization", "Bearer " + authToken)//
.header("userId", CLIENT_ID)//
.header("clientRequestId", requestId)//
.bodyValue(bodyValue())//
.retrieve()//
.bodyToMono(Responseclass.class)//
.block();
The above is working. But let's say I'm debugging and I just want to dump the raw response json into a String. How would I do that? toString() after retrieve doesn't work, and bodyToMono(String.class) didn't seem to work either. Either way it just printed the default toString value of the address of the pointer.
With Spring Webflux WebClient you can get response as String like this:
WebClient client = WebClient.create("http://someurl.de/something");
String responseBody = client.get().retrieve().toEntity(String.class)
.block().getBody();
In my German blog I wrote an article about using WebClient, there you will find more details around the code snippet above:
https://agile-coding.blogspot.com/2021/01/reactive-webclient.html
I am very new to vb/.net and I'm trying to do something that I can do easily in classic vb. I want to get the source html for a webpage from the URL.
I'm using vb.net in Visual Studio Express for Windows 8.
I've read loads of stuff that talk about HttpWebRequest, but I can't get it to work properly.
I did at one point have it returning the html header, but I want to content of the page. Now, I can't even get it back to giving me the header. Ultimately, I want to process the html returned which I'll do (to begin with) the old-fashioned way and process the returned html as a string, but for now I'd like to just get the page.
The code I've got is:
Dim URL As String = "http://www.crayola.com/"
Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create(New Uri(URL))
txtHTML.Text = request.GetRequestStreamAsync().ToString()
Can anyone help me with an example to get me going please?
You're trying to use an Async method in a synchronous way, which won't make any sense. If you're using .NET 4.5, you can try marking the calling method with Async and then using the Await keyword when calling GetRequestStreamAsync.
Public Sub MyDownloaderMethod()
Dim URL As String = "http://www.crayola.com/"
Dim request As System.Net.HttpWebRequest
= System.Net.HttpWebRequest.Create(New Uri(URL))
' Use the Await keyword wait for the async task to complete.
Dim response = request.GetResponseAsync()
txtHTML.Text = response.GetResponseStream().ToString()
End Function
See the following MSDN article for more information on async programming with the Await keyword: http://msdn.microsoft.com/en-us/library/vstudio/hh191443.aspx
Edit
You are receiving your error because you're trying to get the Request stream (what you send the server), and what you really want is the Response stream (what the server sends back to you). I've updated my code to get the WebResponse from your WebRequest and then retrieve the stream from that.
Public Shared Function GetWebPageString(ByVal address As Uri) As String
Using client As New Net.WebClient()
Return client.DownloadString(address)
End Using
End Function
There is also DownloadStringAsync if you don't want to block
request.GetRequestStreamAsync() is probably not a method. I think you're cribbing code from a site where someone wrote their own add-on methods to HttpWebRequest. Try request.GetResponse() to return a response object, then in the response object you can inspect the stream and convert it to text if you need to.
This worked for me in VB.Net 4.5
Public Async Sub GetHTML()
Dim PageHTML as string
Dim client As New HttpClient
Dim getStringTask As Task(Of String) = client.GetStringAsync(PageURL)
PageHTML = Await getStringTask
MsgBox(PageHTML)
End Sub
I am new to Web Crawling, and I am using HttpWebRequest to crawl data from sites.
As of now I was successfully able to crawl and get data from my wordpress site. This data was a simple user profile data. (like name, email, AIM id etc...)
Now as an exercise I want to crawl wikipedia, where I will search using the value entered into textbox at my end and then crawl wikipedia with the search value and get the appropriate title(s) from the search.
Now I have the following doubts/difficulties.
Firstly, is this even possible ? I have heard that wiki has robot.txt setup to block this. Though I have heard this only from a friend and hence not sure.
I am using the same procedure I used earlier, but I am not getting the required results.
Thanks !
Update :
After some explanation and help from #svick, I tried the below code, but still not able to get any value (see last line of code, there I am expecting an html markup of the search result page)
string searchUrl = "http://en.wikipedia.org/w/index.php?search=Wikipedia&title=Special%3ASearch";
var postData = new StringBuilder();
postData.Append("search=" + model.Query);
postData.Append("&");
postData.Append("title" + "Special:Search");
byte[] data2 = Crawler.GetEncodedData(postData.ToString());
var webRequest = (HttpWebRequest)WebRequest.Create(searchUrl);
webRequest.Method = "POST";
webRequest.UserAgent = "Crawling HW (http://yassershaikh.com/contact-me/)";
webRequest.AllowAutoRedirect = false;
ServicePointManager.Expect100Continue = false;
Stream requestStream = webRequest.GetRequestStream();
requestStream.Write(data2, 0, data2.Length);
requestStream.Close();
var responseCsv = (HttpWebResponse)webRequest.GetResponse();
Stream response = responseCsv.GetResponseStream();
// Todo Parsing
var streamReader = new StreamReader(response);
string val = streamReader.ReadToEnd();
// val is empty !! <-- this is my problem !
and here is my GetEncodedData method defination.
public static byte[] GetEncodedData(string postData)
{
var encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postData);
return data;
}
Pls help me on this.
You probably don't need to use HttpWebRequest. Using WebClient (or HttpClient if you're on .Net 4.5) will be much easier for you.
robots.txt doesn't actually block anything. If something doesn't support it (and .Net doesn't support it), it can access anything.
Wikipedia does block requests that don't have their User-Agent header set. And you should use an informative User-Agent string with your contact information.
A better way to access Wikipedia is to use its API, rather than scraping. This way, you will get an answer that's specifically meant to be read by a custom applications, formatted as XML or JSON. There are also dumps containing all information from Wikipedia available for download.
EDIT: The problem with your newly posted code is that your query returns a 302 Moved Temporarily response to the searched article, if it exists. Either remove the line that forbids AllowAutoRedirect, or add &fulltext=Search to your query, which will mean you won't get redirected.
I am having a RESTful service with the following method:
[WebInvoke]
string GetDataFromStringAsString(string xmlString);
My client call to the method is as below:
var client = new RestClient();
client.BaseUrl = serviceBaseUrl;
var request = new RestRequest(method){RequestFormat = DataFormat.Xml};
request.Resource = resourceUrl;
request.AddParameter("text/xml", requestBody,
ParameterType.RequestBody);
var response = client.Execute(request);
Let us take a string to post as "Hello World".
Now the string that i post to the above method gives me a 400 Bad
request. In order to get it working i had to wrap the above string in
a element as shown below:
<string xmlns="http://schemas.microsoft.com/2003/10/
Serialization/">Hello World</string>
Now when i post the above string i get a success response back from
the server.
Why is that i have to manually wrap the string to make it work. Is
there a way that i can achieve to post a string without doing the
above manually.
The only other way that I am aware of is to use stream as your input parameter. e.g.
[WebInvoke]
string GetDataFromStringAsString(stream xmlString);
The problem with .Net 4 WCF REST is that fundamentally WCF only knows how to pass two types of info, either XML or a stream of bytes. Personally, I would use WCF Web API instead of the standard WCF REST library because you are going run into lots more of these kinds of issues.
To get around twitters streaming API not having a crossdomain file to access it from client side( in this case Silverlight) I have made a Generic Handler file in a web project which basically downloads the stream from twitter and as it reads it, writes it to the client.
Here is the handler code:
context.Response.Buffer = false;
context.Response.ContentType = "text/plain";
WebRequest request = WebRequest.Create("http://stream.twitter.com/1/statuses/filter.json?locations=-180,-90,180,90");
request.Credentials = new NetworkCredential("username", "password");
StreamReader responseStream = new StreamReader(request.GetResponse().GetResponseStream(), Encoding.GetEncoding("utf-8"));
while (!responseStream.EndOfStream)
{
string line = "(~!-/" + responseStream.ReadLine() + "~!-/)";
context.Response.BinaryWrite((Encoding.UTF8.GetBytes(line)));}
And this does work, but the problem is that once the client disconnects the handler just carry's on downloading. So how do I tell if the client is still busy receiving the request and if not, end the while loop?
Also, my second problem is that on the client side doing a "ReadLine()" does nothing, presumably because it is counting the entire stream as one line so never gets the full response. To work around that I read it byte by byte and when it sees "(~!-/" around something it know that is one line. VERY hacky, I know.
Thanks!
Found the answer!
while (context.Response.IsClientConnected)
:)