Basic Sample (Java preferred) for Desire2Learn API - authentication

I have visited and read all the Valence, and specifically the REST API, pages. I have one approved key already and a second key that has not yet been approved by D2L, and it's not clear how I request that approval.
The documentation contains a lot of information, but it is difficult to put all the pieces together. For example, in order to make any REST API call, I have to add several parameters to the end of the call. The parameters are documented in one place, but it isn't clear in some cases how to construct them (for example, one of the keys is to contain the url, timestamp, and the type of call being made, but how are they to be concatenated?). Then they have to be signed, and the documentation that tells how to sign the keys is in a completely different page that is not even referenced from the page that tells you that you have to sign the parameters. On top of that, the documentation is not extremely clear about how to do the signing, and offers no further explanation or examples. So to get anywhere, we have to jump around a lot through the documentation, and go through a whole lot of trial and error. It appears that the documentation assumes that the reader has expertise in several areas, which may or may not be true.
Code examples would make a huge difference.

There aren’t a lot of samples yet; we are working to add more, and to make the ones that are present more obvious. As one example, there is a Java Android app that has all the authentication stuff and some basic calls (including the call “whoami” which is a great test call).
The specific auth related files are available as well. From the D2LSigner class, you can see the signing algorithm we use:
Mac hmacSha256 = Mac.getInstance("hmacSHA256");
byte[] keyBytes = key.getBytes("UTF-8");
Key k = new SecretKeySpec(keyBytes, "hmacSHA256");
hmacSha256.init(k);
byte[] dataBytes = data.getBytes("UTF-8");
byte[] sig = hmacSha256.doFinal(dataBytes)
String sigString = base64Url( sig );
From D2LOperationSecurityImpl, you can see how the query string fits together:
//uppercase METHOD, lowercase PATH, timestamp as string
private static /*final*/ String BASE_STRING_TEMPLATE = "{0}&{1}&{2}";
private static /*final*/ String APP_ID_QUERY_NAME = "x_a";
private static /*final*/ String APP_SIG_QUERY_NAME = "x_c";
private static /*final*/ String USER_ID_QUERY_NAME = "x_b";
private static /*final*/ String USER_SIG_QUERY_NAME = "x_d";
private static /*final*/ String TIMESTAMP_QUERY_NAME = "x_t";
...
#Override
public Uri createAuthenticatedUri(String path, String httpMethod) {
long timestamp = System.currentTimeMillis() +
mServerSkewCorrectionMillis.longValue();
Long timestampObjectSeconds = new Long(timestamp/1000);
Object[]formatParms = {httpMethod.toUpperCase(),
path.toLowerCase(),
timestampObjectSeconds.toString()};
String signatureBaseString = MessageFormat.format(BASE_STRING_TEMPLATE,
formatParms);
String appSig = D2LSigner.base64URLSig(mAppKey, signatureBaseString);
String userSig = D2LSigner.base64URLSig(mUserKey, signatureBaseString);
if ((appSig == null) || (userSig == null)) {
return null;
}
String scheme = mEncryptOperations?ENCRYPED_SCHEME:PLAIN_SCHEME;
Uri.Builder b = new Uri.Builder();
b.scheme(scheme);
b.authority(mHostName);
b.path(path);
b.appendQueryParameter(APP_ID_QUERY_NAME, mAppID);
b.appendQueryParameter(APP_SIG_QUERY_NAME, appSig);
b.appendQueryParameter(USER_ID_QUERY_NAME, mUserID);
b.appendQueryParameter(USER_SIG_QUERY_NAME, userSig);
b.appendQueryParameter(TIMESTAMP_QUERY_NAME, timestampObjectSeconds.toString());
Uri securedURI = b.build();
return securedURI;
}
Also, you need to sign the first URL you use for logging in, but only with the application key (because you haven't yet established a user context). It uses a different base string (to protect the URL that is used during auth):
String signature = D2LSigner.base64URLSig(mAppKey, resultURLString);
BasicNameValuePair appID = new BasicNameValuePair(APP_ID_NAME, mAppID);
BasicNameValuePair appSig = new BasicNameValuePair(APP_SIG_NAME, signature);
BasicNameValuePair callbackURL = new BasicNameValuePair(CALLBACK_NAME, resultURLString);

Related

ANTLR4 store all tokens in specific channel

I have a lexer which puts every token the parser is interested in into the default channel and all comment-tokens in channel 1.
The default channel is used to create the actual tree while the comment channel is used to seperate the tokens and to store all comments.
Look at this scribble:
In chapter 12.1 p. 206-208 in The Definitive ANTLR4 Reference there is a comparable situation where comment tokens are shifted inside the token stream. The represented approach is to read out the comment-channel in an exit-method inside the parser.
In my opinion this is a very rough option for my problem, because i don't want to overwhelm my listener with that back-looking operations. Is there a possibility to override a method which puts tokens inside the comment-channel?
It looks like you misunderstand how channels work in ANTLR. What happens is that the lexer, as it comes along a token, assigns the default channel (just a number) during initialization of the token. That value is only changed when the lexer finds a -> channel() command or you change it explicitely in a lexer action. So there is nothing to do in a listener or whatever to filter out such tokens.
Later when you want to get all tokens "in" a given channel (i.e. all tokens that have a specific channel number assigned) you can just iterate over all tokens returned by your token stream and compare the channel value. Alternatively you can create a new CommonTokenStream instance and pass it the channel your are interested in. It will then only give you those tokens from that channel (it uses a token source, e.g. a lexer, to get the actual tokens and cache them).
I found out, that there is a easy way to override how tokens are created. To do this, one can override a method inside the CommonTokenFactory and give it to the Lexer. At this point i can check the channel and i am able to push the tokens in a separate set.
In my opinion this is a little bit hacky, but i do not need to iterate over the whole commonTokenStream later on.
This code is only to demonstrate the idea behind (in C#) .
internal class HeadAnalyzer
{
#region Methods
internal void AnalyzeHeader(Stream headerSourceStream)
{
var antlrFileStream =
new AntlrInputStream(headerSourceStream);
var mcrLexer = new MCRLexer(antlrFileStream);
var commentSaverTokenFactory = new MyTokenFactory();
mcrLexer.TokenFactory = commentSaverTokenFactory;
var commonTokenStream = new CommonTokenStream(mcrLexer);
var mcrParser = new MCRParser(commonTokenStream);
mcrParser.AddErrorListener(new DefaultErrorListener());
MCRParser.ProgramContext tree;
try
{
tree = mcrParser.program(); // create the tree
}
catch (SyntaxErrorException syntaxErrorException)
{
throw new NotImplementedException();
}
var headerContext = new HeaderContext();
var headListener = new HeadListener(headerContext);
ParseTreeWalker.Default.Walk(headListener, tree);
var comments = commentSaverTokenFactory.CommentTokens; // contains all comments :)
}
#endregion
}
internal class MyTokenFactory : CommonTokenFactory
{
internal readonly List<CommonToken> CommentTokens = new List<CommonToken>();
public override CommonToken Create(Tuple<ITokenSource, ICharStream> source, int type, string text, int channel, int start, int stop, int line, int charPositionInLine)
{
var token = base.Create(source, type, text, channel, start, stop, line, charPositionInLine);
if (token.Channel == 1)
{
CommentTokens.Add(token);
}
return token;
}
}
Maybe there are some better approaches. For my usecase it works as expected.

Walmart Marketplace API Integration and Authentication

I am working on integrating my application Walmart Marketplace API using Ruby on Rails.
  1. if i try to generate Auth signature for multiple parameters, it does not generate it and returns exceptions. I am using a Jar file to generate Auth signature
    For e.g. -: https://marketplace.walmartapis.com/v3/orders?createdStartDate=2016-09-13&createdEndDate=2016-09-23 
Does anyone generate Auth Signature & timestamp for multiple parameter for Walmart Marketplace API
  2. Does Auth Signature & timestamp need to be generated for each API call for e.g . Pagination call Also?
Does Authentication need to do for each call?
Additional Comments
I know it is a month later and you already have your program figured out but in case you need some help with these parts or anyone else does, I thought I would include the following information I have on the Walmart API.
1.You might want to consider building a method in ruby since it'll be more interactive with the rest of your ruby program, it was kind of difficult but when I was doing it the most difficult part was wrapping the string in the with the SHA256 digest of string to sign. So I threw together a few methods and it works:
pem = make_pem('PRIVATE KEY', encodedKeyBytes)
digest = OpenSSL::Digest::SHA256.new
pkey = OpenSSL::PKey::RSA.new(pem)
signature = pkey.sign(digest, stringToSign)
def make_pem(tag, der)
box tag, Base64.strict_encode64(der).scan(/.{1,64}/)
end
def box(tag, lines)
lines.unshift "-----BEGIN #{tag}-----"
lines.push "-----END #{tag}-----"
lines.join("\n")
end
It's not perfect but ruby doesn't really have the functionality built in so you have to change it around to get it to work. If this still doesn't work feel free to contact me, but I started out using the jar they provide and I promise it is necessary when you are making thousands of different calls a day with different parameters and urls to be able to find the point of failure and if it isn't in ruby its going to be a lot harder to work with and fix.
2/3. You already answered that these need to be included in every call to the API and I don't really have anything else to add here except to not try to find a way around this, like submitting the same time stamp for a batch of calls. Even though it might work if the calls are made within a certain time window, Walmart uses the time stamp to determine which call came in last which is especially important for things like their price API. Again feel free to email me with any questions, I'll try to respond here too but I don't this website that often.
The variable names I am using these variable names just to reference the code provided in the walmart developer guide. I am just going to translate the java code there to ruby to show how I got the values for stringToSign and encodedKeyBytes.
# This is provided to you by walmart
consumerId = "b68d2a72...."
# Also provided by walmart
privateEncodedStr = "MIICeAIBADANBgkqhkiG9w0BAQEFAA......"
# Full path
baseUrl = "https://marketplace.walmartapis.com/v2/feeds"
# HTTP Method Verb
httpMethod = "GET"
timestamp = (Time.now.to_f * 1000).to_i.to_s
stringToSign = consumerId + "\n" + baseUrl + "\n" + httpMethod + "\n" + timestamp + "\n"
encodedKeyBytes = Base64.decode64(privateEncodedStr)
From there you just run it through the original code and then base64 encode the signature and remove white spaces and then you're good to make a request.
In Order to generate multiple parameter pass string as by escaping sting.
Auth Signature & timestamp need to be generated for each API call for e.g . Pagination call Also
if i try to generate Auth signature for multiple parameters, it does not generate it and returns exceptions. I am using a Jar file to generate Auth signature.
USE SHA class instead of jar file =>
It will generate signature for multiple parameters also.
import org.apache.commons.codec.binary.Base64;
import java.security.KeyFactory;
import java.security.PrivateKey;
import java.security.Signature;
import java.security.spec.PKCS8EncodedKeySpec;
public class SHA256WithRSAAlgo {
private static String consumerId = "b68d2a72...."; // Trimmed for security reason
private static String baseUrl = "https://marketplace.walmartapis.com/v2/feeds";
private static String privateEncodedStr = "MIICeAIBADANBgkqhkiG9w0BAQEFAA......"; //Trimmed for security reasons
public static void main(String[] args) {
String httpMethod = "GET";
String timestamp = String.valueOf(System.currentTimeMillis());
String stringToSign = consumerId + "\n" +
baseUrl + "\n" +
httpMethod + "\n" +
timestamp + "\n";
String signedString = SHA256WithRSAAlgo.signData(stringToSign, privateEncodedStr);
System.out.println("Signed String: " + signedString);
}
public static String signData(String stringToBeSigned, String encodedPrivateKey) {
String signatureString = null;
try {
byte[] encodedKeyBytes = Base64.decodeBase64(encodedPrivateKey);
PKCS8EncodedKeySpec privSpec = new PKCS8EncodedKeySpec(encodedKeyBytes);
KeyFactory kf = KeyFactory.getInstance("RSA");
PrivateKey myPrivateKey = kf.generatePrivate(privSpec);
Signature signature = Signature.getInstance("SHA256withRSA");
signature.initSign(myPrivateKey);
byte[] data = stringToBeSigned.getBytes("UTF-8");
signature.update(data);
byte[] signedBytes = signature.sign();
signatureString = Base64.encodeBase64String(signedBytes);
} catch (Exception e) {
e.printStackTrace();
}
return signatureString;
}
}
Does Auth Signature & timestamps need to be generated for each API call for e.g . Pagination call Also?
YES, for each and every call including pagination , you need to generate new Signature and Timestamps.
Does Authentication need to do for each call?
YES, Authentication need to do for each call.

Google diff-match-patch : How to unpatch to get Original String?

I am using Google diff-match-patch JAVA plugin to create patch between two JSON strings and storing the patch to database.
diff_match_patch dmp = new diff_match_patch();
LinkedList<Patch> diffs = dmp.patch_make(latestString, originalString);
String patch = dmp.patch_toText(diffs); // Store patch to DB
Now is there any way to use this patch to re-create the originalString by passing the latestString?
I google about this and found this very old comment # Google diff-match-patch Wiki saying,
Unpatching can be done by just looping through the diff, swapping
DIFF_INSERT with DIFF_DELETE, then applying the patch.
But i did not find any useful code that demonstrates this. How could i achieve this with my existing code ? Any pointers or code reference would be appreciated.
Edit:
The problem i am facing is, in the front-end i am showing a revisions module that shows all the transactions of a particular fragment (take for example an employee details), like which user has updated what details etc. Now i am recreating the fragment JSON by reverse applying each patch to get the current transaction data and show it as a table (using http://marianoguerra.github.io/json.human.js/). But some JSON data are not valid JSON and I am getting JSON.parse error.
I was looking to do something similar (in C#) and what is working for me with a relatively simple object is the patch_apply method. This use case seems somewhat missing from the documentation, so I'm answering here. Code is C# but the API is cross language:
static void Main(string[] args)
{
var dmp = new diff_match_patch();
string v1 = "My Json Object;
string v2 = "My Mutated Json Object"
var v2ToV1Patch = dmp.patch_make(v2, v1);
var v2ToV1PatchText = dmp.patch_toText(v2ToV1Patch); // Persist text to db
string v3 = "Latest version of JSON object;
var v3ToV2Patch = dmp.patch_make(v3, v2);
var v3ToV2PatchTxt = dmp.patch_toText(v3ToV2Patch); // Persist text to db
// Time to re-hydrate the objects
var altV3ToV2Patch = dmp.patch_fromText(v3ToV2PatchTxt);
var altV2 = dmp.patch_apply(altV3ToV2Patch, v3)[0].ToString(); // .get(0) in Java I think
var altV2ToV1Patch = dmp.patch_fromText(v2ToV1PatchText);
var altV1 = dmp.patch_apply(altV2ToV1Patch, altV2)[0].ToString();
}
I am attempting to retrofit this as an audit log, where previously the entire JSON object was saved. As the audited objects have become more complex the storage requirements have increased dramatically. I haven't yet applied this to the complex large objects, but it is possible to check if the patch was successful by checking the second object in the array returned by the patch_apply method. This is an array of boolean values, all of which should be true if the patch worked correctly. You could write some code to check this, which would help check if the object can be successfully re-hydrated from the JSON rather than just getting a parsing error. My prototype C# method looks like this:
private static bool ValidatePatch(object[] patchResult, out string patchedString)
{
patchedString = patchResult[0] as string;
var successArray = patchResult[1] as bool[];
foreach (var b in successArray)
{
if (!b)
return false;
}
return true;
}

How can I do a search with Google Custom Search API for .NET?

I just discovered the Google APIs Client Library for .NET, but because of lack of documentation I have a hard time to figure it out.
I am trying to do a simple test, by doing a custom search, and I have looked among other, at the following namespace:
Google.Apis.Customsearch.v1.Data.Query
I have tried to create a query object and fill out SearchTerms, but how can I fetch results from that query?
My bad, my first answer was not using the Google APIs.
As a pre-requisite, you need to get the Google API client library
(In particular, you will need to reference Google.Apis.dll in your project). Now, assuming you've got your API key and the CX, here is the same code that gets the results, but now using the actual APIs:
string apiKey = "YOUR KEY HERE";
string cx = "YOUR CX HERE";
string query = "YOUR SEARCH HERE";
Google.Apis.Customsearch.v1.CustomsearchService svc = new Google.Apis.Customsearch.v1.CustomsearchService();
svc.Key = apiKey;
Google.Apis.Customsearch.v1.CseResource.ListRequest listRequest = svc.Cse.List(query);
listRequest.Cx = cx;
Google.Apis.Customsearch.v1.Data.Search search = listRequest.Fetch();
foreach (Google.Apis.Customsearch.v1.Data.Result result in search.Items)
{
Console.WriteLine("Title: {0}", result.Title);
Console.WriteLine("Link: {0}", result.Link);
}
First of all, you need to make sure you've generated your API Key and the CX. I am assuming you've done that already, otherwise you can do it at those locations:
API Key (you need to create a new browser key)
CX (you need to create a custom search engine)
Once you have those, here is a simple console app that performs the search and dumps all the titles/links:
static void Main(string[] args)
{
WebClient webClient = new WebClient();
string apiKey = "YOUR KEY HERE";
string cx = "YOUR CX HERE";
string query = "YOUR SEARCH HERE";
string result = webClient.DownloadString(String.Format("https://www.googleapis.com/customsearch/v1?key={0}&cx={1}&q={2}&alt=json", apiKey, cx, query));
JavaScriptSerializer serializer = new JavaScriptSerializer();
Dictionary<string, object> collection = serializer.Deserialize<Dictionary<string, object>>(result);
foreach (Dictionary<string, object> item in (IEnumerable)collection["items"])
{
Console.WriteLine("Title: {0}", item["title"]);
Console.WriteLine("Link: {0}", item["link"]);
Console.WriteLine();
}
}
As you can see, I'm using a generic JSON deserialization into a dictionary instead of being strongly-typed. This is for convenience purposes, since I don't want to create a class that implements the search results schema. With this approach, the payload is the nested set of key-value pairs. What interests you most is the items collection, which is the search result (first page, I presume). I am only accessing the "title" and "link" properties, but there are many more than you can either see from the documentation or inspect in the debugger.
look at API Reference
using code from google-api-dotnet-client
CustomsearchService svc = new CustomsearchService();
string json = File.ReadAllText("jsonfile",Encoding.UTF8);
Search googleRes = null;
ISerializer des = new NewtonsoftJsonSerializer();
googleRes = des.Deserialize<Search>(json);
or
CustomsearchService svc = new CustomsearchService();
Search googleRes = null;
ISerializer des = new NewtonsoftJsonSerializer();
using (var fileStream = new FileStream(filename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
googleRes = des.Deserialize<Search>(fileStream);
}
with the stream you can also read off of webClient or HttpRequest, as you wish
Google.Apis.Customsearch.v1 Client Library
http://www.nuget.org/packages/Google.Apis.Customsearch.v1/
you may start from Getting Started with the API.

Authenticating with ClientLogin and using the Google Reader API in Android

I have to post a message from my android application to Google Reader. I'm Authenticating with ClientLogin and using the Google Reader API.When I'm trying to send the authentication request it's giving an error.
HTTP/1.1 404 Not Found
I think the client was able to communicate with the server but the server could not find what was requested.I'm giving the following url
postURL = "http://www.google.com/reader/api/0/edit" as
HttpPost post = new HttpPost(postURL);.
Please tell me the solution to resolve this issue.
If you're using ClientLogin you will probably need only 2 kinds of tokens to do what you need with the unnoficial Google Reader API:
Authentication Token - This is the main authentication token, and you'll need to send this in the header of every GET/POST request to the API, otherwise you'll get an error (not the one you're having, though). You can do a POST to this url for this (I have a full code example ahead): https://www.google.com/accounts/ClientLogin
Edit Token - This is the edit token, and you'll need to send this as a form parameter of POST request to any edit APIs (like the http://www.google.com/reader/api/0/edit you've mentioned). You can do a GET to this url for this (I have a code example ahead): http://www.google.com/reader/api/0/token
Getting to the 404 error you're reporting, it must be that you have a typo in your url. Copy my url and try it out. Double check it, side by side. Here is the Java code I've used for this - you can try it out also:
private static final String _AUTHPARAMS = "GoogleLogin auth=";
private static final String _GOOGLE_LOGIN_URL = "https://www.google.com/accounts/ClientLogin";
private static final String _READER_BASE_URL = "http://www.google.com/reader/";
private static final String _API_URL = _READER_BASE_URL + "api/0/";
private static final String _TOKEN_URL = _API_URL + "token";
private static final String _USER_INFO_URL = _API_URL + "user-info";
private static final String _USER_LABEL = "user/-/label/";
private static final String _TAG_LIST_URL = _API_URL + "tag/list";
private static final String _EDIT_TAG_URL = _API_URL + "tag/edit";
private static final String _RENAME_TAG_URL = _API_URL + "rename-tag";
private static final String _DISABLE_TAG_URL = _API_URL + "disable-tag";
private static final String _SUBSCRIPTION_URL = _API_URL
+ "subscription/edit";
private static final String _SUBSCRIPTION_LIST_URL = _API_URL
+ "subscription/list";
public static String getGoogleAuthKey() throws IOException {
String _USERNAME = "USER_EMAIL#gmail.com";
String _PASSWORD = "USER_PASSWORD";
Document doc = Jsoup
.connect(_GOOGLE_LOGIN_URL)
.data("accountType", "GOOGLE", "Email", _USERNAME, "Passwd",
_PASSWORD, "service", "reader", "source",
"[YOUR_APP_ID_GOES_HERE].apps.googleusercontent.com")
.userAgent("[YOUR_APP_ID_GOES_HERE].apps.googleusercontent.com")
.timeout(4000).post();
// RETRIEVES THE RESPONSE TEXT inc SID and AUTH. We only want the AUTH
// key.
String _AUTHKEY = doc
.body()
.text()
.substring(doc.body().text().indexOf("Auth="),
doc.body().text().length());
_AUTHKEY = _AUTHKEY.replace("Auth=", "");
return _AUTHKEY;
}
You can see a code example for getting the edit token and doing an edit in my answer to this other question.
See my answer to this one if you want documentation (unofficial but well structured) - there is no official doc...
The code is based on http://www.chrisdadswell.co.uk/android-coding-example-authenticating-clientlogin-google-reader-api/