I need a help from somebody experienced. I've built 2 microservices recently (let's call them Amber and Boris) which are communicating between each other using ClientProxy and REDIS. From time to time, when Amber is asking for data from Boris, it gets timeout Error.
This is Amber config:
constructor(companyName: string, userId: number) {
this.companyName = companyName;
this.userId = userId;
this.client = ClientProxyFactory.create({
transport: Transport.REDIS,
options: {
retryAttempts: 0,
retryDelay: 0,
url: 'redis://<some_url>:<some_port>,
},
});
}
Then request-response:
private async sendRequest(pattern: string, payload?: object): Promise<any[]> {
payload = payload || {};
try {
const result = await this.client.send(
{ type: pattern },
{ userId: this.userId, companyName: this.companyName, ...payload}
)
.pipe(
timeout(30000),
map((response: any) => { // Success...
return response;
}),
catchError((error) => { // Error...
return throwError(error);
}),
)
.toPromise();
return result;
} catch (err) {
Logger.error('Couldn\'t get data from Boris service: ' + err.message)
}
}
Then on Boris service, I have basically just Controller set with #MessagePattern and I'm just returning data:
#MessagePattern({type: 'getAvailableCases'})
findAll(#Payload() data: object): Promise<object> {
this.assignPayload(data);
return this.getData();
}
Important to say, Boris service is doing queries to database in order to return data. But on db side seems there is no problem.
What I'm interested in the most is:
whether I have ClientProxy set up properly
whether I have answer processing set up properly with pipe() and toPromise(), as I'm not well familiarized with ClientProxy and RxJs.
Thank you a hundred times for any response!
Turned out, the ClientProxy wasn't releasing connections to Redis after the communication is done. This way, the number of connections was increasing until there was no connection left.
The solution is to close connection after the data are returned:
this.client.close();
Related
In a Vue component controlling users subsciption to newsletters, I have the fellowing code:
async newSubscriber(event) {
// Validate email
//---------------
if (!this.isEmailValid(this.subscriber_email))
this.subscribeResult = "Email not valid";
else {
// If valid, check if email is not already recorded
//-------------------------------------------------
let alreadyRecorded = false;
let recordedEmails = await this.$apollo.query({ query: gql`query { newslettersEmails { email } }` });
console.log('length ' + recordedEmails.data.newslettersEmails.length);
console.log(recordedEmails.data.newslettersEmails);
for (let i = 0; !alreadyRecorded && i < recordedEmails.data.newslettersEmails.length; i++)
alreadyRecorded = this.subscriber_email === recordedEmails.data.newslettersEmails[i].email;
if (alreadyRecorded)
this.subscribeResult = "Email already recorded";
else {
// If not, record it and warn the user
//------------------------------------
this.$apollo.mutate({
mutation: gql`mutation ($subscriber_email: String!){
createNewslettersEmail(input: { data: { email: $subscriber_email } }) {
newslettersEmail {
email
}
}
}`,
variables: {
subscriber_email: this.subscriber_email,
}
})
.then((data) => { this.subscribeResult = "Email recorded"; })
.catch((error) => { this.subscribeResult = "Error recording the email: " + error.graphQLErrors[0].message; });
}
}
}
At the very first email subscription test, $apollo.query returns me the correct number of emails already recorded (let's say, 10) and record the new subscriber email. But if I try to record a second email without hard refreshing (F5) the browser, $apollo.query returns me the exact same result than the first time (10), EVEN IF the first test email has been correctly recorded by strapi (graphql palyground showns me the added email with the very same query!). Even if I add ten emails, apollo will always return me what it got during its first call (10 recorded emails), as if it uses a buffered result. Of course, that allows Vue to record several times the same email, which I obviously want to avoid!
Does it speaks to anyone ?
After a lot of Google digging (giving the desired results by simply changing in my requests, at the end, "buffering" by "caching" !), I understood that Apollo cache its queries by default (at least, in the configuration of the Vue project I received). To solve the problem I just added "fetchPolicy: 'network-only'" to the query I make:
let recordedEmails = await this.$apollo.query({
query: gql`query { newslettersEmails { email } }`,
});
became
let recordedEmails = await this.$apollo.query({
query: gql`query { newslettersEmails { email } }`,
fetchPolicy: 'network-only'
});
And problem solved ^^
I'm using google cloud big query service with nodejs client version 1.0x . I created a job successfully by function createQueryJob(). After that, I used an event listen when a callback create job response with getQueryResults() such as:
const options = {
query: sqlQuery,
useLegacySql: true,
dryRun: true
};
// this.bigquery is an constructor.
// this.bigquery = new BigQuery({
// projectId: this.projectId,
// keyFilename: this.keyFile,
// });
this.bigquery.createQueryJob(options, function (err, job) {
if (!err) {
// job id such as 731bf23b-5044-4842-894b-4d9f77485d9b
function manualPaginationCallback(err, rows, nextQuery, apiResponse) {
if (nextQuery) {
job.getQueryResults(nextQuery, manualPaginationCallback);
} else {
return Promise.resolve(rows);
}
}
return job.getQueryResults({
maxResults: 100000,
autoPaginate: false,
// timeoutMs : 60000
}, manualPaginationCallback);
}
});
But It throw an error
{"error":{"code":404,"message":"Not found: Job
[myProjectId]:731bf23b-5044-4842-894b-4d9f77485d9b","errors":[{"message":"Not
found: Job
[myProjectId]:731bf23b-5044-4842-894b-4d9f77485d9b","domain":"global","reason":"notFound"}],"status":"NOT_FOUND"}}
refrence
https://cloud.google.com/nodejs/docs/reference/bigquery/1.0.x/BigQuery#createQueryJob
https://cloud.google.com/nodejs/docs/reference/bigquery/1.0.x/Job#getQueryResults
What wrong's with me? Any help. Thank you!
You're setting the dryrun option in your request, which only validates the job but doesn't actually run the query. Dryrun jobs don't persist, which is why you get not found on the subsequent request.
I need advise for handling errors in front-end of web application.
When I call a service to get the community according to community in web app, I want it to catch an error. For example for catching errors like 404.
There is a service for getting community according to id provided.
getCommunity(id: number) {
return this.http.get(`${this.api}/communities/` + id + ``);
}
that is called in events.ts file
setCommunityBaseUrl() {
this.listingService.getCommunity(environment.communityId).subscribe((data: any) => {
this.communityUrl = data.url + `/` + data.domain;
});
}
The id is provided in environment. Let's say there are 20 communities in total. When I provide id = 1 the events according to community = 1 appears.
export const environment = {
production: ..,
version: 'v2',
apiUrl: '...',
organization: '...',
websiteTitle: '...',
communityId: 1,
googleMapsApiKey: '...'
};
The problem is that when I provide id = null all community events are occurring | all events list in the backend is occurring.
Please, help ^^
When you subscribe you subscribe with an Observer pattern. So the first function you pass in
.subscribe(() => {} );
fires when the Observable calls .next(...)
and after that you can provide another function which will fire whenever the Observable calls .error(...)
so
.subscribe(() => {}, (error) => { handleTheError(error); } );
The this.http.get(...); returns an Observable which will fire the .error(...) on http error
We also know that this.http.get(...) completes or "errors" and it's not an endless one (a one that never completes). So you can make it a promise and manipulate on it promise like.
async getMeSomething(...) {
try {
this.mydata = await this.http.get(...).toPromise();
}
catch(error) {
handleTheError(error)
}
}
But what I really recommend is to use Swagger for your backend and then generate the API Client class with NSwagStudio so you don't have to write the client manually or adjust it or deal with error catching. I use it all the time and it saves us an enormous amount of time
Because you are using ".subscribe" you can create your own error handler and catch the errors like this, directly on the method.
This is an example on how you can use this:
constructor(
private _suiteAPIService: SuitesAPIService,
private _testcaseService: TestcaseService,
public _tfsApiService: TfsApiService,
private _notificationService: NotificationService) { }
errorHandler(error: HttpErrorResponse) {
return observableThrowError(error.message || "Server Error")
}
public something = "something";
GetTestcasesFromSuiteSubscriber(Project, BuildNumber, SuiteId) {
this._suiteAPIService.GetTestResults(Project, BuildNumber, SuiteId).subscribe(
data => {
console.log(data);
this._testcaseService.ListOfTestcases = data;
//Notofication service to get data.
this._notificationService.TestcasesLoaded();
},
error => {
//Here we write som error
return this.something;
}
);
}
I'm following the example here: https://www.w3.org/TR/webrtc/#simple-peer-to-peer-example
I've modified the code because I only need one-way streaming:
var configuration = null; //{ "iceServers": [{ "urls": "stuns:stun.example.org" }] };
var peerConnection;
var outboundPeerStream = null;
var outboundPeerStreamSessionId = null;
var createPeerConnection = function () {
if (peerConnection)
return;
peerConnection = new RTCPeerConnection(configuration);
// send any ice candidates to the other peer
peerConnection.onicecandidate = function (event) {
signalrModule.sendClientNotification(JSON.stringify({ "candidate": event.candidate }));
};
// let the "negotiationneeded" event trigger offer generation
peerConnection.onnegotiationneeded = peerStreamingModule.sendOfferToPeers;
// once remote track arrives, show it in the remote video element
peerConnection.ontrack = function (event) {
var inboundPeerStream = event.streams[0];
remoteStreamHelper.pushStreamToDom(inboundPeerStream, foo);
}
}
// this gets called either on negotiationNeeded and every 30s to ensure all peers have the offer from the stream originator
peerStreamingModule.sendOfferToPeers = function () {
peerConnection.createOffer().then(function (offer) {
return peerConnection.setLocalDescription(offer);
}).then(function () {
// send the offer to the other peer
signalrModule.sendClientNotification(JSON.stringify({ "desc": peerConnection.localDescription}));
}).catch(logger.internalLog);
};
// this gets called by the stream originator when the stream is available to initiate streaming to peers
peerStreamingModule.initializeWithStream = function (outboundStream, sessionId) {
outboundPeerStream = outboundStream;
outboundPeerStreamSessionId = sessionId;
createPeerConnection();
peerConnection.addStream(outboundStream);
//peerStreamingModule.sendOfferToPeers(); I don't think I need this...
}
peerStreamingModule.handleP2PEvent = function (notification) {
if (!peerConnection)
createPeerConnection();
if (notification.desc) {
var desc = notification.desc;
// if we get an offer, we need to reply with an answer
if (desc.type == "offer") {
peerConnection.setRemoteDescription(desc).then(function () {
return peerConnection.createAnswer();
}).then(function (answer) {
return peerConnection.setLocalDescription(answer);
}).then(function () {
signalrModule.sendClientNotification(JSON.stringify({ "desc": peerConnection.localDescription, "sessionId": sessionManager.thisSession().deviceSessionId() }), app.username());
}).catch(logger.internalLog);
} else if (desc.type == "answer") {
peerConnection.setRemoteDescription(desc).catch(logger.internalLog);
} else {
logger.internalLog("Unsupported SDP type. Your code may differ here.");
}
} else
pc.addIceCandidate(notification.candidate).catch(logger.internalLog);
}
This seems to be working, but I'm stumped with two parts:
1) WebRTC - Failed to set remote answer sdp: Called in wrong state: STATE_INPROGRESS - this is appearing in my logs from time to time - am I doing something wrong in the above that is causing this?
2) Am I correctly implementing sendOfferToPeers and initializeWithStream? I'm afraid that the sendOfferToPeers getting triggered on interval from the originator isn't how the spec is intended to be used; my goal is to ensure that all peers eventually receive an offer no matter when they join or whether or not they're facing connectivity issues that drop the original offer / negotiation.
// this gets called either on negotiationNeeded and every 30s to ensure all peers have the offer
You can't send the same offer to multiple peers. It's peer-to-peer, not peer-to-peers. One-to-many requires at minimum a connection per participant, and probably a media server to scale.
Also, SDP is not for discovery. The offer/answer exchange is a fragile state-machine negotiation between two end-points only, to set up a single connection.
You should solve who's connecting with whom ahead of establishing the WebRTC connection.
I am building an Angular2 app and one of the components needs to make multiple API calls which are dependent on the previous ones.
I currently have a service which makes an API call to get a list of TV shows. For each show, I then need to call a different API multiple times to step through the structure to determine if the show exists on a Plex server.
The API documentation is here
For each show, I need to make the following calls and get the correct data to determine if it exists: (Assume we have variables <TVShow>, <Season>, <Episode>)
http://baseURL/library/sections/?X-Plex-Token=xyz will tell me:
title="TV Shows" key="2"
http://baseURL/library/sections/2/all?X-Plex-Token=xyz&title=<TVShow> will tell me: key="/library/metadata/2622/children"
http://baseURL/library/metadata/2622/children?X-Plex-Token=xyz will tell me: index="<Season>" key="/library/metadata/14365/children"
http://baseURL/library/metadata/14365/children?X-Plex-Token=xyz will tell me: index="<Episode>" which implies that the episode I have exists.
The responses are in json, I have removed a lot of the excess text. At each stage I need to check that the right fields exist (<TVShow>, <Season>, <Episode>) so that they can be used for the next call. If not, I need to return that the show does not exist. If it does, I will probably want to return an id for the show.
I have looked at lots of examples including promise, async & flatmap, but am not sure how to solve this based on the other examples I have seen.
How to chain Http calls in Angular2
Angular 2.0 And Http
Angular 2 - What to do when an Http request depends on result of another Http request
Angular 2 chained Http Get Requests with Iterable Array
nodejs async: multiple dependant HTTP API calls
How to gather the result of Web APIs on nodeJS with 'request' and 'async'
Here is what I have for getting the list of shows. (shows.service.ts)
export class ShowsHttpService {
getShows(): Observable<Show[]> {
let shows$ = this._http
.get(this._showHistoryUrl)
.map(mapShows)
.catch(this.handleError);
return shows$;
}
}
function mapShows(response:Response): Show[] {
return response.json().data.map(toShow);
}
function toShow(r:any): Show {
let show = <Show>({
episode: r.episode,
show_name: r.show_name,
season: r.season,
available : false, // I need to fill in this variable if the show is available when querying the Plex API mentioned above.
});
// My best guess is here would be the right spot to call the Plex API as we are dealing with a single show at a time at this point, but I cannot see how.
return show;
}
Here is the relevant code from the component (shows.component.ts)
public getShows():any {
this._ShowsHttpService
.getShows()
.subscribe(w => this.shows = w);
console.log(this.shows);
}
Bonus points
Here are the obvious next questions that are interesting, but not necessary:
The first API query will be much faster than waiting for all of the other queries to take place (4 queries * ~10 shows). Can the initial list be returned and then updated with the available status when it is ready.
The first Plex call to get the key="2" only needs to be performed once. It could be hard coded, but instead, can it be performmed once and remembered?
Is there a way to reduce the number of API calls? I can see that I could remove the show filter, and search through the results on the client, but this doesn't seam ideal either.
The 4 calls for each show must be done sequentially, but each show can be queried in parallel for speed. Is this achievable?
Any thoughts would be much appreciated!
Not sure if I totally understand your question, but here is what I do:
I make the first http call, then when the subscribe fires, it calls completeLogin. I could then fire another http call with its own complete function and repeat the chain.
Here is the component code. The user has filled in the login information and pressed login:
onSubmit() {
console.log(' in on submit');
this.localUser.email = this.loginForm.controls["email"].value;
this.localUser.password = this.loginForm.controls["password"].value;
this.loginMessage = "";
this.checkUserValidation();
}
checkUserValidation() {
this.loginService.getLoggedIn()
.subscribe(loggedIn => {
console.log("in logged in user validation")
if(loggedIn.error != null || loggedIn.error != undefined || loggedIn.error != "") {
this.loginMessage = loggedIn.error;
}
});
this.loginService.validateUser(this.localUser);
}
This calls the loginservice ValidateUser method
validateUser(localUser: LocalUser) {
this.errorMessage = "";
this.email.email = localUser.email;
var parm = "validate~~~" + localUser.email + "/"
var creds = JSON.stringify(this.email);
var headers = new Headers();
headers.append("content-type", this.constants.jsonContentType);
console.log("making call to validate");
this.http.post(this.constants.taskLocalUrl + parm, { headers: headers })
.map((response: Response) => {
console.log("json = " + response.json());
var res = response.json();
var result = <AdminResponseObject>response.json();
console.log(" result: " + result);
return result;
})
.subscribe(
aro => {
this.aro = aro
},
error => {
console.log("in error");
var errorObject = JSON.parse(error._body);
this.errorMessage = errorObject.error_description;
console.log(this.errorMessage);
},
() => this.completeValidateUser(localUser));
console.log("done with post");
}
completeValidateUser(localUser: LocalUser) {
if (this.aro != undefined) {
if (this.aro.errorMessage != null && this.aro.errorMessage != "") {
console.log("aro err " + this.aro.errorMessage);
this.setLoggedIn({ email: localUser.email, password: localUser.password, error: this.aro.errorMessage });
} else {
console.log("log in user");
this.loginUser(localUser);
}
} else {
this.router.navigate(['/verify']);
}
}
In my login service I make a call to the authorization service which returns an observable of token.
loginUser(localUser: LocalUser) {
this.auth.loginUser(localUser)
.subscribe(
token => {
console.log('token = ' + token)
this.token = token
},
error => {
var errorObject = JSON.parse(error._body);
this.errorMessage = errorObject.error_description;
console.log(this.errorMessage);
this.setLoggedIn({ email: "", password: "", error: this.errorMessage });
},
() => this.completeLogin(localUser));
}
In the authorization service:
loginUser(localUser: LocalUser): Observable<Token> {
var email = localUser.email;
var password = localUser.password;
var headers = new Headers();
headers.append("content-type", this.constants.formEncodedContentType);
var creds:string = this.constants.grantString + email + this.constants.passwordString + password;
return this.http.post(this.constants.tokenLocalUrl, creds, { headers: headers })
.map(res => res.json())
}
The point here in this code, is to first call the validateUser method of the login service, upon response, based on the return information, if its valid, I call the loginUser method on the login service. This chain could continue as long as you need it to. You can set class level variables to hold the information that you need in each method of the chain to make decisions on what to do next.
Notice also that you can subscribe to the return in the service and process it there, it doesn't have to return to the component.
Okay, Here goes:
public getShows():any {
this._ShowsHttpService
.getShows()
.subscribe(
w => this.shows = w,
error => this.errorMessage = error,
() => this.completeGetShows());
}
completeGetShow() {
//any logic here to deal with previous get;
this.http.get#2()
.subscribe(
w => this.??? = w),
error => this.error = error,
() => this.completeGet#2);
}
completeGet#2() {
//any logic here to deal with previous get;
this.http.get#3()
.subscribe(
w => this.??? = w),
error => this.error = error,
() => this.completeGet#3);
}
completeGet#3() {
//any logic here to deal with previous get;
//another http: call like above to infinity....
}