I've this code to get nearby places and nearby beaches from a point, with Google maps. This is called from a Next.js component, via the useSWR hook.
All the data is returned correctly, but before first Axios call (const fetchNearbyPlaces = async (urlWithToken = null) => {...), I'm receiving this error in the console:
API resolved without sending a response for /api/google/places/33.807501/-78.70039, this may result in stalled requests.
I can't figure out what the error is, although there may be several because I'm a novice. I appreciate any suggestion.
const axios = require("axios");
const GetNearbyPlaces = async (req, res) => {
const {
latitude,
longitude,
} = req.query;
const radius = 50000;
const types = [
"airport",
"tourist_attraction",
"amusement_park",
"aquarium",
"art_gallery",
"bar",
"museum",
"night_club",
"cafe",
"restaurant",
"shopping_mall",
"store",
"spa",
];
function checkFunc(arr, val) {
return arr.some(arrVal => val === arrVal);
}
const url = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=${latitude}%2C${longitude}&radius=${radius}&key=${process.env.CW_GOOGLE_MAPS_API_KEY}`;
const beachesUrl = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=${latitude}%2C${longitude}&radius=${radius}&type=natural_feature&key=${process.env.CW_GOOGLE_MAPS_API_KEY}`;
try {
let results = [];
let beaches = [];
const fetchNearbyBeaches = async (urlWithToken = null) => {
await axios.get(urlWithToken ? urlWithToken : beachesUrl).then(data => {
beaches = [...beaches, ...data.data.results];
if (data?.data?.next_page_token) {
const newUrl = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?key=${process.env.CW_GOOGLE_MAPS_API_KEY}&pagetoken=${data.data.next_page_token}`;
setTimeout(() => {
fetchNearbyBeaches(newUrl);
}, 2000);
} else {
beaches.length > 5 && beaches.splice(5);
results.length > 5 && results.splice(5);
const finalResults = [...beaches, ...results];
finalResults.length > 10 && finalResults.splice(10);
return res.status(200).json({
data: {
results: finalResults,
},
success: true,
});
}
});
};
const fetchNearbyPlaces = async (urlWithToken = null) => {
await axios.get(urlWithToken ? urlWithToken : url).then(data => {
results = [...results, ...data.data.results];
if (data?.data?.next_page_token) {
const newUrl = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?key=${process.env.CW_GOOGLE_MAPS_API_KEY}&pagetoken=${data.data.next_page_token}`;
setTimeout(() => {
fetchNearbyPlaces(newUrl);
}, 2000);
} else {
const dirtyResultsWithDuplicates = [];
results.map(result => {
return types.map(type => {
if (checkFunc(result.types, type) && !result.types.includes("lodging")) {
dirtyResultsWithDuplicates.push(result);
}
});
});
const set = new Set(dirtyResultsWithDuplicates);
const filtered = Array.from(set);
results = filtered.length > 10 ? filtered.splice(10) : filtered;
return fetchNearbyBeaches();
}
});
};
fetchNearbyPlaces();
} catch (err) {
res.status(500).json({
message: err.message,
statusCode: 500,
});
}
};
export default GetNearbyPlaces;
The problem is with the backend application not the frontend component.
Nextjs expects a response to have been sent when the api handler function exits. If for example you have a databaseCall.then(sendResponse) in your api handler function what happens is that the handler function exits before the database returns.
Now this is not a problem if the database does return after that and sends the response, but it is if for example the database has an error. Because the handler function exits without a response already being sent Nextjs can't be sure that at that point there isn't a stalled request.
One way to fix this is by await-ing the db call(or whatever other async function you call) thereby preventing the handler function from exiting before some kind of response has been send.
The solution was added this object to mi API code.
export const config = {
api: {
externalResolver: true,
},
};
Documentation: https://nextjs.org/docs/api-routes/request-helpers
Related
So I have a project using the latest Next js 13, React 18, Urql 3, and using typescript
Currently, I have issues when trying to query the urql from the getstaticprops function. My urql request needs a guest token, and I'm storing the token on session storage(other suggestions ?).
It has no issue when the query is running on the client, but I have it when querying inside the function.
My concern is related to the token reading, so the server cannot read the session storage value.
I'm asking what is the better and simplest way to make this work.
Does use cookies to store guest tokens will make this work?
Or the configuration that doesn't work?
This is my current config for urql.ts
import {
createClient,
ssrExchange,
dedupExchange,
cacheExchange,
fetchExchange,
} from "urql";
import { GRAPH_URL } from "#lib/constant/env";
import type { TypedDocumentNode } from "#urql/core";
const isServerSide = typeof window === "undefined";
const ssrCache = ssrExchange({
isClient: !isServerSide,
});
const client = createClient({
url: GRAPH_URL,
exchanges: [dedupExchange, cacheExchange, ssrCache, fetchExchange],
fetchOptions: () => {
const token = sessionStorage.getItem("accessToken");
return {
headers: {
authorization: token ? `Bearer ${token}` : "",
},
};
},
});
const query = async (
query: TypedDocumentNode<any, object>,
variables?: Record<string, string | string[] | unknown>
) => {
try {
const response = await client.query(query, variables as any).toPromise();
return response;
} catch (error) {
if (error instanceof Error) console.error(error.message);
}
};
const mutation = async (
mutation: TypedDocumentNode<any, object>,
variables?: Record<string, string | string[] | unknown>
) => {
try {
const response = await client
.mutation(mutation, variables as any)
.toPromise();
return response;
} catch (error) {
if (error instanceof Error) console.error(error.message);
}
};
export { client, query, mutation, ssrCache };
And this some of the code for the blog index page
export const getStaticProps = async () => {
await fetchArticlesSummary();
return {
props: {
urqlState: ssrCache.extractData(),
},
revalidate: 600,
};
};
export default withUrqlClient(() => ({
url: GRAPH_URL,
}))(BlogPage);
This is for the fetchArticlesSummary
export const fetchArticlesSummary = async () => {
try {
const {
data: { listArticles },
}: any = await query(getListArticle);
return listArticles.items;
} catch (error) {
return {
notFound: true,
};
}
};
I also doing a setup on _app.tsx
export default function App({ Component, pageProps }: AppProps) {
if (pageProps.urqlState) {
ssrCache.restoreData(pageProps.urqlState);
}
return (
<Provider value={client}>
<Component {...pageProps} />
</Provider>
);
}
Thank you
I have followed urql documentation about server-side configuration and many others but still don't have any solutions.
I have a react native application where i have two users using the app (customer and restaurant)
So on checkout I connect the customer to websocket on the express server and once the order is placed i send a message to the restaurant which is supposed to be connected to websocket all time.
However, sometimes the restaurant is disconnected somehow, so I am trying to keep the restaurant connected, and if disconnected then reconnect again automatically.
In react native restaurant side implementation i have the following code :
this is useWebSocketLite hook to handle connection, send, receive messages and retry connection to server when closed:
function useWebSocketLite({ socketUrl, retry: defaultRetry = 3, retryInterval = 1000 }) {
const [data, setData] = useState();
const [send, setSend] = useState(() => () => undefined);
const [retry, setRetry] = useState(defaultRetry);
const [readyState, setReadyState] = useState(false);
useEffect(() => {
const ws = new WebSocket(socketUrl);
ws.onopen = () => {
setReadyState(true);
setSend(() => {
return (data) => {
try {
const d = JSON.stringify(data);
ws.send(d);
return true;
} catch (err) {
return false;
}
};
});
ws.onmessage = (event) => {
const msg = formatMessage(event.data);
setData({ message: msg, timestamp: getTimestamp() });
};
};
ws.onclose = () => {
setReadyState(false);
if (retry > 0) {
setTimeout(() => {
setRetry((retry) => retry - 1);
}, retryInterval);
}
};
return () => {
ws.close();
};
}, [retry]);
return { send, data, readyState };
}
So based on this, every-time the connection is closed, the connection will retry again.
Besides, when a restaurant launches the app the following code will be implemented:
const ws = useWebSocketLite({
socketUrl: `wss://${url}/id=${user.user_id}&role=restaurant`
});
This useEffect to establish the connection:
useEffect(() => {
if (ws.readyState === true) {
setConnectionOpen(true);
}
}, [ws.readyState]);
and this useEffect to handle incoming messages
useEffect(() => {
if (ws.data) {
const message = ws.data;
//dispatch...
}
}, [ws.data]);
Express server implementation:
This is the code where i handle socket connections and messages in express server:
var webSockets = {}
function setupWebSocket(server) {
server.on('connection', (socket, req) => {
if (req) {
var clientId = req.url
let regexReplace = /[\[\]/]/g
let regex = /([^=#&]+)=([^?&#]*)/g,
params = {},
match;
while ((match = regex.exec(clientId))) {
params[decodeURIComponent(match[1]).replace(regexReplace, '')] = decodeURIComponent(match[2])
}
if (params.role === 'restaurant') {
webSockets[params.id] = socket
}
}
socket.on('message', data => {
let sData = JSON.parse(JSON.parse(data))
let {id, data} = sData.data
sendToClient(id, 'order', data)
})
socket.on('error', (err) => {
console.log(err)
})
socket.on('close', (code, req) => {
var clientId = req.url
let regexReplace = /[\[\]/]/g
let regex = /([^=#&]+)=([^?&#]*)/g,
params = {},
match;
while ((match = regex.exec(clientId))) {
params[decodeURIComponent(match[1]).replace(regexReplace, '')] = decodeURIComponent(match[2])
}
if (params.role === 'restaurant') {
delete webSockets[clientId]
console.log(`${webSockets[clientId]} disconnected with code ${code} !`);
}
});
});
// sends a message to a specific client
const sendToClient = (clientId, type, data = {}) => {
const payload = { type, data }
const messageToSend = JSON.stringify({ error: false, message: payload })
if (webSockets[clientId]) {
webSockets[clientId].send(messageToSend)
console.log(`${clientId} client notified with this order`)
} else {
console.log(`${clientId} websocket client is not connected.`)
}
}
}
So most of the time I get 13 websocket client is not connected. which means the restaurant has already been deleted from the webSockets object and its connection already closed.
Apologise for long question and hope someone can help me regarding this.
First of all, you should know that this is not a good practice of websockets, where you are forcing the client (the restaurant) to be connected.
Whatever, at the current state of your code, there is an illogical behavior: at the end of the useEffect of your “useWebSocketLite” function, you are closing the socket connection:
return () => {
ws.close();
};
Knowing that the useEffect hook is called twice: after the first render of the component, and then after every change of the dependencies (the “retry” state in your case); Your code can be ridden like so: everytime the “retry” state changes, we will close the socket! So for me that is why you got the client disconnected.
I know some questions about the subject has been opened here and there, but my issue is different :
all the other ones appear in dev mode, in my case it's in production,
a very big percentage of requests pass, a few of them is TypeError: Network request failed - but sometimes for critical requests
it's random, not always the same request. Sometimes it passes, sometimes not.
it appears to three on my projects, one is on AWS the other one on Clever-Cloud, both are projects between 1000 and 5000 users, servers are quite too big for what they do - I think I removed the risk of a server fault. Even if... I can reproduce locally when I don't start the api locally. So it's like the api is not responding, but as I said, I don't think so.
I have no clue where to dig anymore...
I can give you my API.js service file, maybe you'll find what's wrong ?
import URI from 'urijs';
import { Platform } from 'react-native';
import NetInfo from '#react-native-community/netinfo';
import { getUserToken, wipeData } from '../utils/data';
import { SCHEME, MW_API_HOST } from '../config';
import deviceInfoModule from 'react-native-device-info';
import { capture } from '../utils/sentry';
const unauthorisedHandler = (navigation) => {
wipeData();
navigation.reset({ index: 0, routes: [{ name: 'Auth' }] });
};
const checkNetwork = async (test = false) => {
const isConnected = await NetInfo.fetch().then((state) => state.isConnected);
if (!isConnected || test) {
await new Promise((res) => setTimeout(res, 1500));
return false;
}
return true;
};
class ApiService {
host = MW_API_HOST;
scheme = SCHEME;
getUrl = (path, query) => {
return new URI().host(this.host).scheme(this.scheme).path(path).setSearch(query).toString();
};
execute = async ({ method = 'GET', path = '', query = {}, headers = {}, body = null }) => {
try {
const config = {
method,
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
appversion: deviceInfoModule.getBuildNumber(),
appdevice: Platform.OS,
currentroute: this.navigation?.getCurrentRoute?.()?.name,
...headers,
},
body: body ? JSON.stringify(body) : null,
};
const url = this.getUrl(path, query);
console.log('url: ', url);
const canFetch = await checkNetwork();
if (!canFetch) return;
let response;
// To try to avoid mysterious `TypeError: Network request failed` error
// that throws an error directly
// we try catch and try one more time.
try {
response = await fetch(url, config);
} catch (e) {
if (e?.toString().includes('Network request failed')) {
// try again
await new Promise((res) => setTimeout(res, 250));
console.log('try again because Network request failed');
response = await fetch(url, config);
} else {
throw e;
}
}
if (!response.ok) {
if (response.status === 401) {
const token = await getUserToken();
if (token) unauthorisedHandler(API.navigation);
return response;
}
}
if (response.json) return await response.json();
return response;
} catch (e) {
capture(e, { extra: { method, path, query, headers, body } });
return { ok: false, error: "Sorry, an error occured, technical team has been warned." };
}
};
executeWithToken = async ({ method = 'GET', path = '', query = {}, headers = {}, body = null }) => {
const token = await getUserToken();
if (token) headers.Authorization = token;
return this.execute({ method, path, query, headers, body });
};
get = async (args) => this.executeWithToken({ method: 'GET', ...args });
post = async (args) => this.executeWithToken({ method: 'POST', ...args });
put = async (args) => this.executeWithToken({ method: 'PUT', ...args });
delete = async (args) => this.executeWithToken({ method: 'DELETE', ...args });
}
const API = new ApiService();
export default API;
Talking with experts here and there, it seems that it's normal : internet network is not 100% reliable, so sometimes, request fail, for a reason that we can't anticipate (tunnel, whatever).
I ended up using fetch-retry and I still have a few of those, but much less !
I have a test to test my cloudflare worker that looks like this:
const workerScript = fs.readFileSync(
path.resolve(__dirname, '../pkg-prd/worker.js'),
'utf8'
);
describe('worker unit test', function () {
// this.timeout(60000);
let worker;
beforeEach(() => {
worker = new Cloudworker(workerScript, {
bindings: {
HTMLRewriter
},
});
});
it('tests requests and responses', async () => {
const request = new Cloudworker.Request('https://www.example.com/pathname')
const response = await worker.dispatch(request);
console.log(response);
// const body = await response.json();
expect(response.status).to.eql(200);
// expect(body).to.eql({message: 'Hello mocha!'});
});
});
In my worker I do something like this:
const response = await fetch(BASE_URL, request);
const modifiedResponse = new Response(response.body, response);
// Remove the webflow badge
class ElementHandler {
element(element) {
element.append('<style type="text/css">body .w-webflow-badge {display: none!important}</style>', {html: true})
}
}
console.log(3);
return new HTMLRewriter()
.on('head', new ElementHandler()).transform(modifiedResponse);
Now when i run my test I get this error message:
● worker unit test › tests requests and responses
TypeError: Cannot read property 'transform' of undefined
at evalmachine.<anonymous>:1:1364
at FetchEvent.respondWith (node_modules/#dollarshaveclub/cloudworker/lib/cloudworker.js:39:17)
What seems to be wrong?
HTMLRewriter i created looks like this:
function HTMLRewriter() {
const elementHandler = {};
const on = (selector, handler) => {
if (handler && handler.element) {
if (!elementHandler[selector]) {
elementHandler[selector] = [];
}
elementHandler[selector].push(handler.element.bind(handler));
}
};
const transform = async response => {
const tempResponse = response.clone();
const doc = HTMLParser.parse(await tempResponse.text());
Object.keys(elementHandler).forEach(selector => {
const el = doc.querySelector(selector);
if (el) {
elementHandler[selector].map(callback => {
callback(new _Element(el));
});
}
});
return new Response(doc.toString(), response);
};
return {
on,
transform
};
}
Since HTMLRewriter() is called with new, the function needs to be a constructor. In JavaScript, a constructor function should set properties on this and should not return a value. But, your function is written to return a value.
So, try changing this:
return {
on,
transform
};
To this:
this.on = on;
this.transform = transform;
I'm using a static array to scaffold a user table, prior to refactoring with actual postgres db and some fetch()-ing code. At present, the tests work, but obviously they are working synchronously. Here's the placeholder API code:
// UserAPI.js
let findUserById = (credentials = {}) => {
const { userId } = credentials
if (userId) {
const foundUser = users.find(user => user.id === userId)
if (foundUser !== undefined) {
const { password: storedpassword, ...user } = foundUser
return user
}
}
return null
}
exports.byId = findUserById
And an example test as follows:
// excerpt from TokenAuth.test.js
const UserAPI = require('../lib/UserAPI')
describe('With TokenAuth middleware', () => {
beforeEach(() => {
setStatus(0)
})
it('should add user to req on authorised requests', () => {
const token = createToken(fakeUser)
const authReq = { headers: { authorization: 'Bearer ' + token } }
const myMiddleware = TokenAuth(UserAPI.byId)
myMiddleware(authReq, fakeRes, fakeNext)
// expect(authReq.user).toStrictEqual({ id: 1, username: 'smith#example.com' });
expect(authReq.user.username).toStrictEqual('smith#example.com')
expect(authReq.user.id).toStrictEqual(1)
})
})
This runs fine, and along with other tests gives me the coverage I want. However, I now want to check that the tests will deal with the async/await nature of the fetch() code I'm going to use for the proper UserAPI.js file. So I re-write the placeholder code as:
// UserAPI.js with added async/await pauses ;-)
let findUserById = async (credentials = {}) => {
const { userId } = credentials
// simulate url resolution
await new Promise(resolve => setTimeout(() => resolve(), 100)) // avoid jest open handle error
if (userId) {
const foundUser = users.find(user => user.id === userId)
if (foundUser !== undefined) {
const { password: storedpassword, ...user } = foundUser
return user
}
}
return null
}
exports.byId = findUserById
... at which point I start getting some lovely failures, due I think it's returning unresolved promises.
My problem is two-fold:
How should I alter the UserAPI.test.js tests to deal with the new async nature of findUserByCredentials() ?
Am I ok in my assumption that ExpressJS is happy with async functions as request handlers? Specifically, due to the async nature ofUserAPI.findUserByCredentials is this ok?
Main App.js uses curried UserAPI.byId() for the findUserById.
// App.js (massively simplified)
const express = require('express')
const TokenAuth = require('./middleware/TokenAuth')
const RequireAuth = require('./middleware/RequireAuth')
const UserAPI = require('./lib/UserAPI')
let router = express.Router()
const app = express()
app.use(TokenAuth(UserAPI.byId))
app.use(RequireAuth)
app.use('/users', UserRouter)
module.exports = app
My TokenAuth middleware would now run along these lines:
// TokenAuth.js (simplified)
const jwt = require('jsonwebtoken')
require('dotenv').config()
const signature = process.env.SIGNATURE
let TokenAuth = findUserById => async (req, res, next) => {
let header = req.headers.authorization || ''
let [type, token] = header.split(' ')
if (type === 'Bearer') {
let payload
try {
payload = jwt.verify(token, signature)
} catch (err) {
res.sendStatus(401)
return
}
let user = await findUserById(payload)
if (user) {
req.user = user
} else {
res.sendStatus(401)
return
}
}
next()
}
module.exports = TokenAuth
A partial answer us simply to add an async/await on the middleware call:
it('should add user to req on authorised requests', async () => {
const token = createToken(fakeUser)
const authReq = { headers: { authorization: 'Bearer ' + token } }
const myMiddleware = TokenAuth(UserAPI.byId)
await myMiddleware(authReq, fakeRes, fakeNext)
// expect(authReq.user).toStrictEqual({ id: 1, username: 'smith#example.com' });
expect(authReq.user.username).toStrictEqual('smith#example.com')
expect(authReq.user.id).toStrictEqual(1)
})