A single "|" character in the URL breaks loading contents of URL - objective-c

I'm trying to load an article in JSON format from the Wikipedia API and I get these errors:
nil host used in call to allowsSpecificHTTPSCertificateForHost
nil host used in call to allowsAnyHTTPSCertificateForHost:
NSURLConnection finished with error - code -1002
error when trying to fetch from URL (null) - The file couldn’t be opened.
I get these errors only when the URL string includes the character "|"
The URL for an article with id=1 (pageids=1) is:
https://en.wikipedia.org/w/api.php?action=query&format=json&pageids=1&prop=extracts&exintro&explaintext
The URL above doesn't contain the character "I" so it works just fine.
In the wikipedia API you can ask for multiple articles by seperating their ids with the "|" character
The URL for the articles with ids=1,2 and 3 (pageids=1|2|3) is:
https://en.wikipedia.org/w/api.php?action=query&format=json&pageids=1|2|3&prop=extracts&exintro&explaintext
The URL above contains the "|" character and everything fails.
I use this snippet I found in another post just to catch the errors:
NSError *error = NULL;
NSStringEncoding actualEncoding;
NSString *string = [[NSString alloc] initWithContentsOfURL:url usedEncoding:&actualEncoding error:&error];
if(string)
{
NSLog( #"hey, I actually got a result of %#", string);
if(actualEncoding != NSUTF8StringEncoding)
{
NSLog( #"and look at that, the actual encoding wasn't NSUTF8StringEncoding");
}
} else {
NSLog( #"error when trying to fetch from URL %# - %#", [url absoluteString], [error localizedDescription]);
}
If you go through the code, url.absoluteString returns null when there's a "|" character in it.

The pipe (|) is a special character. You have to encode the URL by adding appropriate percent encoding.
This has nothing to do with the text encoding of a string.
NSString * string = #"https://en.wikipedia.org/w/api.php?action=query&format=json&pageids=1|2|3&prop=extracts&exintro&explaintext";
NSURL *url = [NSURL URLWithString:[string stringByAddingPercentEncodingWithAllowedCharacters:[NSCharacterSet URLQueryAllowedCharacterSet]]];

Related

JSON data has "bad" characters that causes NSJSONSerialization to die

I am using the ATV version of TVH Client - if you haven't looked at this it's worth looking at TVH to glimpse madness in the face. It has a JSON API that sends back data, including the electronic program guide. Sometimes the channels put accented characters in their data. Here is an example, this is the result from Postman, note the ? char in the description:
{
"eventId": 14277,
"episodeId": 14278,
"channelName": "49.3 CometTV",
"channelUuid": "02fe96403d58d53d71fde60649bf2b9a",
"channelNumber": "49.3",
"start": 1480266000,
"stop": 1480273200,
"title": "The Brain That Wouldn't Die",
"description": "Dr. Bill Cortner and his fianc�e, Jan Compton , are driving to his lab when they get into a horrible car accident. Compton is decapitated. But Cortner is not fazed by this seemingly insurmountable hurdle. His expertise is in transplants, and he is excited to perform the first head transplant. Keeping Compton's head alive in his lab, Cortner plans the groundbreaking yet unorthodox surgery. First, however, he needs a body."
},
If this data is fed into NSJSONSerialization, it returns an error. So to avoid this, the data is first fed into this function:
+ (NSDictionary*)convertFromJsonToObjectFixUtf8:(NSData*)responseData error:(__autoreleasing NSError**)error {
NSMutableData *FileData = [NSMutableData dataWithLength:[responseData length]];
for (int i = 0; i < [responseData length]; ++i) {
char *a = &((char*)[responseData bytes])[i];
if ( (int)*a >0 && (int)*a < 0x20 ) {
((char*)[FileData mutableBytes])[i] = 0x20;
} else {
((char*)[FileData mutableBytes])[i] = ((char*)[responseData bytes])[i];
}
}
NSDictionary* json = [NSJSONSerialization JSONObjectWithData:FileData //1
options:kNilOptions
error:error];
if( *error ) {
NSLog(#"[JSON Error (2nd)] output - %#", [[NSString alloc] initWithData:responseData encoding:NSUTF8StringEncoding]);
NSDictionary *userInfo = #{ NSLocalizedDescriptionKey:[NSString stringWithFormat:NSLocalizedString(#"Tvheadend returned malformed JSON - check your Tvheadend's Character Set for each mux and choose the correct one!", nil)] };
*error = [[NSError alloc] initWithDomain:#"Not ready" code:NSURLErrorBadServerResponse userInfo:userInfo];
return nil;
}
return json;
}
This cleans up the case when there is a control character in the data, but not an accent like the case above. When I feed in that data I get the "Tvheadend returned malformed JSON" error.
One problem is that the user can change the character set among a limited number of selections, and the server does not tell the client what it is. So one channel might use UTF8 and another ISO-8891-1, and there is no way to know which to use on the client side.
So: can anyone offer a suggestion on how to process this data so we feed clean strings into NSJSONSerialization?
I still do not know the root cause of the problem I am seeing - the server is sending not only high-bit characters like the ones I noted above, but I also found that it contained control characters too! Looking over other threads it appears I am not the only one seeing this problem, so hopefully others will find this useful...
The basic trick is to convert the original data from the server to a string using UTF8. If there are any of these "bad" chars in it, the conversion will fail. So you check if the resulting string is empty, and try another charset. Eventually you'll get data back. Now you take that string and strip out any control chars. Now you take that result, which is now UTF8 "clean", and convert it back to UTF8 NSData. That will pass through the JSON conversion without error. Phew!
Here is the solution I finally used:
// ... the original data from the URL is in responseData
NSString *str = [[NSString alloc] initWithData:responseData encoding:NSUTF8StringEncoding];
if ( str == nil ) {
str = [[NSString alloc] initWithData:responseData encoding:NSISOLatin1StringEncoding];
}
if ( str == nil ) {
str = [[NSString alloc] initWithData:responseData encoding:NSASCIIStringEncoding];
}
NSCharacterSet *controls = [NSCharacterSet controlCharacterSet];
NSString *stripped = [[str componentsSeparatedByCharactersInSet:controls] componentsJoinedByString:#""];
NSData *data = [stripped dataUsingEncoding:NSUTF8StringEncoding];
NSDictionary* json = [NSJSONSerialization JSONObjectWithData:data options:kNilOptions error:&error];
I hope someone finds this useful!

Web Service JSON nsdictionary unicode string [duplicate]

NSData* jsonData is the http response contains JSON data.
NSString* jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
NSLog(#"jsonString: %#", jsonString);
I got the result:
{ "result": "\u8aaa" }
What is the proper way to encoding the data to the correct string, not unicode string like "\uxxxx"?
If you convert the JSON data
{ "result" : "\u8aaa" }
to a NSDictionary (e.g. using NSJSONSerialization) and print the dictionary
NSError *error;
NSDictionary *jsonDict = [NSJSONSerialization JSONObjectWithData:jsonData options:0 error:&error];
NSLog(#"%#", jsonDict);
then you will get the output
{
result = "\U8aaa";
}
The reason is that the description method of NSDictionary uses "\Unnnn" escape sequences
for all non-ASCII characters. But that is only for display in the console, the dictionary is correct!
If you print the value of the key
NSLog(#"%#", [jsonDict objectForKey:#"result"]);
then you will get the expected output
說
I don't quite understand what the problem is. AFNetworking has given you a valid JSON packet. If you want the above code to output the character instead of the \u… escape sequence, you should coax the server feeding you the result to change its output. But this shouldn't be necessary. What you most likely want to do next is run it through a JSON deserializer…
NSDictionary * data = [NSJSONSerialization JSONObjectWithData:jsonData …];
…and you should get the following dictionary back: #{#"result":#"說"}. Note that the result key holds a string with a single character, which I'm guessing is what you want.
BTW: In future, I suggest you copy-paste output into your question rather than transcribing it by hand. It'll avoid several needless rounds of corrections and confusion.

Try to create NSString from bytes. It always returns an empty string

I am trying to create a string from bytes a received via network. The NSString I het is always an empty one.
if (stringLength > 0) {
NSData *bytes = [[NSData alloc] initWithBytes:data+1 length:stringLength];
result = [[NSString alloc] initWithData:bytes encoding:NSUTF8StringEncoding];
//result = [[NSString alloc] initWithBytes:data+1 length:stringLength encoding:NSASCIIStringEncoding];
}
As I said I get an empty NSString. The string is a base64 encoded value so it should be valid utf-8 since it only contains ascii symbols.
Maybe your raw bytes aren't UTF-8 after all. Or maybe, because you're passing data+1 instead of data itself, you're causing the encoding attempt to fail because the process thinks there's an improper UTF-8 encoding sequence. (Forgive me if I'm being presumptuous, but you have to take everything into account.) In any case, you're relying heavily on your assumption, and that's a trap we all fall into now and then.
Here's a strategy for you. If your attempt at creating an NSString instance using NSUTF8StringEncoding returns nil, then try using NSISOLatin1StringEncoding. And if that returns nil, then try using NSMacOSRomanStringEncoding.
Even if, after all of that, you get a string that's not quite right, it's still better than a nil string in that it could help to show you if there's some other area in which you've made a mistake.
Good luck to you in your endeavors.
Check if it's really base64 encoded..
CFErrorRef error = NULL;
CFDataRef decodedData;
SecTransformRef decoder;
decoder = SecDecodeTransformCreate(kSecBase64Encoding, &error);
if (error) {
CFShow(error);
exit(-1);
}
SecTransformSetAttribute(decoder,
kSecTransformInputAttributeName,
(CFDataRef *)bytes,
&error);
if (error) {
CFShow(error);
exit(-1);
}
decodedData = SecTransformExecute(decoder, &error);
if (error) {
CFShow(error);
exit(-1);
}

How to parse a JSON having no quotes with its KEY string?

I want to parse the json output resulting from the following url in SBJSON framework for iOS
http://maps.google.com/maps?q=school&mrt=yp&sll=13.006389,80.2575&output=json
while(1);{title:"school - Google Maps",url:"/maps?q=school\x26mrt=yp\x26sll=13.006389,80.2575\x26ie=UTF8\x26hq=school\x26hnear=",urlViewport:false,ei:"RCu3T4eeMqSiiAe7k-yZDQ",form:{selected:"q",q:{q:"school",mrt:"yp",what:"school",near:""},d:{saddr:"",daddr:"",dfaddr:""},geocode:""},
I am using http://www.bodurov.com/JsonFormatter/ to read it online.
In ASIHttpRequest response method I removed while(1); from the response
NSString *responseString = [[request resonseString]substringFromIndex:9]; //to remove while(1)
SBJSONParser * parser = [[SBJSONParser alloc]init];
NSDictionary *jsonDict = (NSDictionary*)[parser objectFromString:responseString];
NSLog(#"%#",jsonDict) // prints null
// [responseString JSONValue] also reports error
I guess JSON key without double quotes is causing problem.
Instead of {
"title": "hospital - Google Maps",
"urlViewport": false,
}, we get {
title: "hospital - Google Maps",
"urlViewport": false
}
Please help me to parse this complex JSON structure returned from Google.
This worked better for my case because my values contained times which caused the regular expression in the above answer to match incorrectly.
json = [json stringByReplacingOccurrencesOfString: #"(\\w*[A-Za-z]\\w*)\\s*:"
withString: #"\"$1\":"
options: NSRegularExpressionSearch
range: NSMakeRange(0, json.length)];
You need to add the missing quotes to the keys, so try this:
responseString = [responseString stringByReplacingOccurrencesOfString:#"(\\w+)\\s*:"
withString:#"\"$1\":"
options:NSRegularExpressionSearch
range:NSMakeRange(0, [responseString length])];
This should work well with the given JSON string.

dataWithContentsOfURL - What is expected from the server?

I am trying to create NSData with the contents of an URL:
NSString *theUrl = [NSString stringWithString:#"http://127.0.0.1:8090"];
NSError *connectionError = nil;
NSData *inData = [NSData dataWithContentsOfURL:[NSURL URLWithString:theUrl] options:NSDataReadingUncached error:&connectionError];
NSInteger code = [connectionError code];
if (code != 0)
{
NSString *locDesc = [NSString stringWithString:[connectionError localizedDescription]];
NSString *locFail = [NSString stringWithString:[connectionError localizedFailureReason]];
NSLog(#"Error: %d %# %#", code, locDesc, locFail);
}
else if ([inData length] == 0)
{
NSLog(#"No data");
}
I have a super simple Java http server running on the local host that returns Hello World to a client:
DataOutputStream os = new DataOutputStream(s.getOutputStream()); // s is the socket
os.writeBytes(new String("Hello World\0"));
os.flush();
os.close();
s.close();
When pointing Google Chrome to http://127.0.0.1:8090 it displays Hello World as expected so data is sent back. When I run the objective-c code the inData is empty (0x0, data length is 0), and the error code is 0 so I don't have an error to inspect. If I change theUrl to "http://www.google.com" it seems works fine as the data length becomes > 0.
So my question is why inData is empty when I go the to local http-server. Does the stream have to be terminated with a specific data sequence?
Is the server outputting an HTTP status code like it's supposed to? If the response doesn't contain a 200 status indicating that the request was completed successfully, that might be causing dataWithContentsOfURL:options:error: to fail.
A little more context would be useful, but a wild guess is that your "super simple" HTTP server does not send any headers or not the ones expected by NSURL.
Have you tried curl -i http://127.0.0.1:8090 to see what the output actually looks like?