Sails JS with Redis for caching - api

As I said in my previous questions, I am trying to learn how to use sails.js, what I'm trying to do now is to cache the response of an api to redis. I have searched on how to do this, but I can't make it to work. Without caching, I call the api through ajax.
Any thoughts on how I will be able to do it using my controller? How can I call the api using the controller in sails.js and cache the response using redis?

You can use https://github.com/mranney/node_redis
Steps:
Add to package.json
"redis": "^0.12.1"
Run
npm install
Create a service module /api/services/CachedLookup.js
var redis = require("redis"),
client = redis.createClient();
module.exports = {
rcGet: function (key, cb) {
client.get(key, function (err, value) {
return cb(value);
});
},
fetchApi1: function (cb) {
var key = 'KEY'
CachedLookup.rcGet(key, function (cachedValue) {
if (cachedValue)
return cb(cachedValue)
else {//fetch the api and cache the result
var request = require('request');
request.post({
url: URL,
form: {}
}, function (error, response, body) {
if(error) {
//handle error
}
else {
client.set(key, response);
return cb(response)
}
});
}
});
}
}
Inside the controller
CachedLookup.fetchApi1(function (apiResponse) {
res.view({
apiResponse: apiResponse
});
});

Related

Nuxt handle fetch errors from Prismic API

I'm building a blog with Nuxt to and Prismic as CMS.
my nuxt.config.js looks like this:
mode: 'universal',
modules: ['#nuxtjs/prismic'],
target: 'static',
generate: {
fallback: '404.html',
},
Project is deployed on Netlify with build command "npm run generate"
In pages directory I have dynamic links ( _uid.vue ) where I use the new fetch to get the post according to route.
async fetch() {
const post = await this.$prismic.api.getByUID('blog-post', this.$route.params.uid)
this.post = post
},
This all works! However I want to handle fetch errors and display correspond error page. For example when the post we try to fetch does not exist or now is deleted. I tried as they show from the link I provide above about fetch, but I get error that post is undefined.
async fetch() {
const post = await await this.$prismic.api.getByUID('blog-post', this.$route.params.uid)
if (post.id === this.$route.params.id) {
this.post = post
} else {
// set status code on server and
if (process.server) {
this.$nuxt.context.res.statusCode = 404
}
// use throw new Error()
throw new Error('Post not found')
}
}
My project on GitHub
Also I'm not sure using the fetch hook inside a page is considered a best practice, I think you should prefer asyncData with the following pattern (or async/await one):
export default {
asyncData({ params, error }) {
return axios
.get(`https://my-api/posts/${params.id}`)
.then(res => {
return { title: res.data.title }
})
.catch(e => {
error({ statusCode: 404, message: 'Post not found' })
})
}
}
From Nuxt documentation~
Could you not just catch any exceptions like this:
try {
const post = await this.$prismic.api.getByUID('blog-post', this.$route.params.uid);
if (post.id === this.$route.params.id) {
this.post = post;
}
} catch ((error) => {
// set status code on server and
if (process.server) {
this.$nuxt.context.res.statusCode = 404;
}
// use throw new Error()
throw new Error('Post not found');
});
Of course you would have to actually check the kind of exception occurred.

Axios interceptors don't send data to API in production Heroku app

This is part 2 of me debugging my application in production
In part 1, I managed to at least see what was causing my problem and managed to solve that.
When I send a request to my API which is hosted on Heroku using axios interceptor, every single request object looks like this in the API
{ 'object Object': '' }
Before sending out data to the API, I console.log() the transformRequest in axios and I can see that the data I am sending is actually there.
Note: I have tested this process simply using
axios.<HTTP_METHOD>('my/path', myData)
// ACTUAL EXAMPLE
await axios.post(
`${process.env.VUE_APP_BASE_URL}/auth/login`,
userToLogin
);
and everything works and I get data back from the server.
While that is great and all, I would like to abstract my request implementation into a separate class like I did below.
Does anyone know why the interceptor is causing this issue? Am I misusing it?
request.ts
import axios from "axios";
import { Message } from "element-ui";
import logger from "#/plugins/logger";
import { UsersModule } from "#/store/modules/users";
const DEBUG = process.env.NODE_ENV === "development";
const service = axios.create({
baseURL: process.env.VUE_APP_BASE_URL,
timeout: 5000,
transformRequest: [function (data) {
console.log('data', data)
return data;
}],
});
service.interceptors.request.use(
config => {
if (DEBUG) {
logger.request({
method: config.method,
url: config.url
});
}
return config;
},
error => {
return Promise.reject(error);
}
);
service.interceptors.response.use(
response => {
console.log('axios interception response', response)
return response.data;
},
error => {
const { response } = error;
console.error('axios interception error', error)
if (DEBUG) {
logger.error(response.data.message, response);
}
Message({
message: `Error: ${response.data.message}`,
type: "error",
duration: 5 * 1000
});
return Promise.reject({ ...error });
}
);
export default service;
Login.vue
/**
* Sign user in
*/
async onClickLogin() {
const userToLogin = {
username: this.loginForm.username,
password: this.loginForm.password
};
try {
const res = await UsersModule.LOGIN_USER(userToLogin);
console.log("res", res);
this.onClickLoginSuccess();
} catch (error) {
throw new Error(error);
}
}
UsersModule (VUEX Store)
#Action({ rawError: true })
async [LOGIN_USER](params: UserSubmitLogin) {
const response: any = await login(params);
console.log('response in VUEX', response)
if (typeof response !== "undefined") {
const { accessToken, username, name, uid } = response;
setToken(accessToken);
this.SET_UID(uid);
this.SET_TOKEN(accessToken);
this.SET_USERNAME(username);
this.SET_NAME(name);
}
}
users api class
export const login = async (data: UserSubmitLogin) => {
return await request({
url: "/auth/login",
method: "post",
data
});
};
I'm not sure what you're trying to do with transformRequest but that probably isn't what you want.
A quote from the documentation, https://github.com/axios/axios#request-config:
The last function in the array must return a string or an instance of Buffer, ArrayBuffer, FormData or Stream
If you just return a normal JavaScript object instead it will be mangled in the way you've observed.
transformRequest is responsible for taking the data value and converting it into something that can actually be sent over the wire. The default implementation does quite a lot of work manipulating the data and setting relevant headers, in particular Content-Type. See:
https://github.com/axios/axios/blob/885ada6d9b87801a57fe1d19f57304c315703079/lib/defaults.js#L31
If you specify your own transformRequest then you are replacing that default, so none of that stuff will happen automatically.
Without knowing what you're trying to do it's difficult to advise further but you should probably use a request interceptor rather than transformRequest for whatever it is you're trying to do.

API Request in Dialogflow Fulfillment (Javascript)

So I'm trying to make a google action using Dialogflow that requires an external API. I've always used jQuery .getJSON() to make API calls, so I had no idea how to do this. After searching this up online, I found a way to do this using vanilla javascript (I also tested the way on my website and it worked fine). The code for that is below:
function loadXMLDoc() {
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == XMLHttpRequest.DONE) {
console.log(xmlhttp.responseText);
}
};
xmlhttp.open("GET", "https://translate.yandex.net/api/v1.5/tr.json/translate?lang=en-es&key=trnsl.1.1.20190105T052356Z.7f8f950adbfaa46e.9bb53211cb35a84da9ce6ef4b30649c6119514a4&text=eat", true);
xmlhttp.send();
}
The code worked fine on my website, but as soon as I added it to the Dialogflow, it would give me the error
XMLHttpRequest is not defined
Obviously that happened because I never defined it (using var), except it worked without me doing anything. So then, I tried adding this line
var XMLHttpRequest = require("xmlhttprequest").XMLHttpRequest;
to the code, and it stopped giving me the error (because I defined XMLHttpRequest). But then, my code wouldn't work.
TL;DR: How can I make an external API call using Dialogflow fulfillment?
You can use https. But make sure that you upgrade to Blaze Pay(or any other plans) to make external API calls, else you will receive an error such as
Error:
Billing account not configured. External network is not accessible and quotas are severely limited. Configure billing account to remove these restrictions.
Code to make external api call,
// See https://github.com/dialogflow/dialogflow-fulfillment-nodejs
// for Dialogflow fulfillment library docs, samples, and to report issues
"use strict";
const functions = require("firebase-functions");
const { WebhookClient } = require("dialogflow-fulfillment");
const { Card, Suggestion } = require("dialogflow-fulfillment");
const https = require("https");
process.env.DEBUG = "dialogflow:debug"; // enables lib debugging statements
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(
(request, response) => {
const agent = new WebhookClient({ request, response });
console.log(
"Dialogflow Request headers: " + JSON.stringify(request.headers)
);
console.log("Dialogflow Request body: " + JSON.stringify(request.body));
function getWeather() {
return weatherAPI()
.then(chat => {
agent.add(chat);
})
.catch(() => {
agent.add(`I'm sorry.`);
});
}
function weatherAPI() {
const url =
"https://samples.openweathermap.org/data/2.5/weather?q=London,uk&appid=b6907d289e10d714a6e88b30761fae22";
return new Promise((resolve, reject) => {
https.get(url, function(resp) {
var json = "";
resp.on("data", function(chunk) {
console.log("received JSON response: " + chunk);
json += chunk;
});
resp.on("end", function() {
let jsonData = JSON.parse(json);
let chat = "The weather is " + jsonData.weather[0].description;
resolve(chat);
});
});
});
}
function welcome(agent) {
agent.add(`Welcome to my agent!`);
}
function fallback(agent) {
agent.add(`I didn't understand`);
agent.add(`I'm sorry, can you try again?`);
}
let intentMap = new Map();
intentMap.set("Default Welcome Intent", welcome);
intentMap.set("Default Fallback Intent", fallback);
intentMap.set("Weather Intent", getWeather);
agent.handleRequest(intentMap);
}
);
This article is a diamond! It really helped to clarify what's going on and what's required in Dialogflow fullfilments.
A small suggestion is to gracefully catch the error in the connection to the webservice:
function weatherAPI() {
const url = "https://samples.openweathermap.org/data/2.5/weather?q=London,uk&appid=b6907d289e10d714a6e88b30761fae22";
return new Promise((resolve, reject) => {
https.get(url, function(resp) {
var json = "";
resp.on("data", function(chunk) {
console.log("received JSON response: " + chunk);
json += chunk;
});
resp.on("end", function() {
let jsonData = JSON.parse(json);
let chat = "The weather is " + jsonData.weather[0].description;
resolve(chat);
});
}).on("error", (err) => {
reject("Error: " + err.message);
});
});
}

Dynamic routes in express based on external api data

I need to automatically generate the routes on an expressjs app, based on the vimeo api
I thought that I need to loop through the api data, save the data in the db and then retrieve that data in a middleware. For example:
Api request:
const Vimeo = require("vimeo").Vimeo;
let client = new Vimeo("CLIENT_ID", "CLIENT_SECRET", "TOKEN");
client.request(
{
method: "GET",
path: "/my/path/videos"
},
function(error, body, status_code, headers) {
if (error) {
console.log(error);
}
let data = body.data;
for (var i = 0; i < data.length; i++) {
// save data in the db
}
});
Middleware:
app.use('/videos/:name', (req, res, next) {
if (req.params.name === myDBdata) {
console.log('It works!');
next();
} else {
// error code
}
});
Is this a good way to proceed? Thanks in advance
Make a function which takes two paremter like:
function makeRoute(path, handler) {
return app.use(path, handler)
}
And then call this for every data
makeRoute('test', (req, res) => { })

How do I serve a file from S3 through Meteor Iron Router

My question is very similar to this one which describes how to serve a local file using Iron Router. I need to do the same, but instead of reading the file synchronously from disk, I need to get the file from S3 which is an asynchronous call.
The problem appears to be the fact that the action method has returned before the asynchronous s3.getObject completes giving me the following error.
Error: Can't render headers after they are sent to the client.
I'm assuming that Iron Router is generating the response for me when it realizes that I haven't handled the response in my action method, but I'm stumped about how to tell it to wait for my asynchronous call to finish.
Here is my code.
Router.map(function () {
this.route('resumeDownload', {
where: 'server',
path: '/resume/:_id',
action: function () {
var response = this.response;
var candidate = Candidates.findOne(this.params._id);
if (!candidate || !candidate.resumeS3Key) {
// this works fine because the method hasn't returned yet.
response.writeHead(404);
return response.end();
}
var s3 = new AWS.S3();
s3.getObject({Bucket: 'myBucket', Key: candidate.resumeS3Key}, function (err, data) {
if (err) {
// this will cause the error to be displayed
response.writeHead(500);
return response.end();
}
// this will also cause the error to be displayed
response.writeHead(200, {'Content-Type': data.ContentType});
response.end(data.Body);
});
}
});
});
I was able to solve this one myself. I needed to use a future in my action method.
Here is the working code.
Router.map(function () {
this.route('resumeDownload', {
where: 'server',
path: '/resume/:_id',
action: function () {
var response = this.response,
candidate = Candidates.findOne(this.params._id);
if (!candidate || !candidate.resumeS3Key) {
response.writeHead(404);
return response.end();
}
var Future = Npm.require('fibers/future'),
s3 = new AWS.S3(),
futureGetObject = Future.wrap(s3.getObject.bind(s3)),
data = futureGetObject({Bucket: 'myBucket', Key: candidate.resumeS3Key}).wait();
response.writeHead(200, {'Content-Type': data.ContentType});
response.end(data.Body);
}
});
});