Testing a cloudflare worker with HTMLRewriter fails as its undefined - cloudflare

I have a test to test my cloudflare worker that looks like this:
const workerScript = fs.readFileSync(
path.resolve(__dirname, '../pkg-prd/worker.js'),
'utf8'
);
describe('worker unit test', function () {
// this.timeout(60000);
let worker;
beforeEach(() => {
worker = new Cloudworker(workerScript, {
bindings: {
HTMLRewriter
},
});
});
it('tests requests and responses', async () => {
const request = new Cloudworker.Request('https://www.example.com/pathname')
const response = await worker.dispatch(request);
console.log(response);
// const body = await response.json();
expect(response.status).to.eql(200);
// expect(body).to.eql({message: 'Hello mocha!'});
});
});
In my worker I do something like this:
const response = await fetch(BASE_URL, request);
const modifiedResponse = new Response(response.body, response);
// Remove the webflow badge
class ElementHandler {
element(element) {
element.append('<style type="text/css">body .w-webflow-badge {display: none!important}</style>', {html: true})
}
}
console.log(3);
return new HTMLRewriter()
.on('head', new ElementHandler()).transform(modifiedResponse);
Now when i run my test I get this error message:
● worker unit test › tests requests and responses
TypeError: Cannot read property 'transform' of undefined
at evalmachine.<anonymous>:1:1364
at FetchEvent.respondWith (node_modules/#dollarshaveclub/cloudworker/lib/cloudworker.js:39:17)
What seems to be wrong?
HTMLRewriter i created looks like this:
function HTMLRewriter() {
const elementHandler = {};
const on = (selector, handler) => {
if (handler && handler.element) {
if (!elementHandler[selector]) {
elementHandler[selector] = [];
}
elementHandler[selector].push(handler.element.bind(handler));
}
};
const transform = async response => {
const tempResponse = response.clone();
const doc = HTMLParser.parse(await tempResponse.text());
Object.keys(elementHandler).forEach(selector => {
const el = doc.querySelector(selector);
if (el) {
elementHandler[selector].map(callback => {
callback(new _Element(el));
});
}
});
return new Response(doc.toString(), response);
};
return {
on,
transform
};
}

Since HTMLRewriter() is called with new, the function needs to be a constructor. In JavaScript, a constructor function should set properties on this and should not return a value. But, your function is written to return a value.
So, try changing this:
return {
on,
transform
};
To this:
this.on = on;
this.transform = transform;

Related

Mock, jest and time

I read some tips on how to mock your request/response in Express framework in the blog:
https://codewithhugo.com/express-request-response-mocking/. However, I have no clue how to mock the controller below.
export const healthCheck = async (req, res, next) => {
log("debug", "healthCheck controller called");
const healthcheck = {
uptime: process.uptime(),
message: "Server is running!",
now_timestamp: Date.now()
};
try {
res.send(healthcheck);
} catch (error) {
healthcheck.message = error;
res.status(503).send();
}
};
I am glad to share my efforts below. My suspicion is that I must mock class Date as well.
import {
healthCheck
} from "../healthcheck.js";
const mockRequest = () => {
const req = {}
req.body = jest.fn().mockReturnValue(req)
req.params = jest.fn().mockReturnValue(req)
return req
};
const mockResponse = () => {
const res = {}
res.get = jest.fn().mockReturnValue(res)
res.send = jest.fn().mockReturnValue(res)
res.status = jest.fn().mockReturnValue(res)
res.json = jest.fn().mockReturnValue(res)
return res
};
const mockNext = () => {
return jest.fn()
};
describe("healthcheck", () => {
afterEach(() => {
// restore the spy created with spyOn
jest.restoreAllMocks();
});
it("should call mocked log for invalid from scaler", async () => {
let req = mockRequest();
let res = mockResponse();
let next = mockNext();
await healthCheck(req, res, next);
expect(res.send).toHaveBeenCalledTimes(1)
expect(res.send.mock.calls.length).toBe(1);
});
});

How do i modify onRequest body payload before send it using TestCafe

I try to modify the API body request with onRequest method but it do not work correctly.
I'm try to use example from documentation:
this is my Example:
export class MyRequestHook extends RequestHook {
onRequest(event) {
try {
if (event.requestOptions?.body) {
event.requestOptions.body = Buffer.from(
{
...JSON.parse(event?.requestOptions?.body.toString()),
newOption: true,
},
'utf8',
);
console.log('after buffer');
}
} catch (error) {
console.log(error);
}
}
onResponse(e) {}
}
// ....
const customHook = new MyRequestHook(/api\/get-some-staff/);
// ....
fixture.only`TEST FIXTURE`
.page(`url-url`)
.requestHooks(customHook).disablePageCaching;
And it seems it just ignore it at all.
When you change body, you also need to change the header content-length. For example:
//server.js
const http = require('http');
const hostname = 'localhost';
const port = '3001';
const server = http.createServer((req, res) => {
const chunks = [];
req.on('data', (chunk) => {
chunks.push(chunk);
});
req.on('end', () => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end(`Response: ${Buffer.concat(chunks).toString()}`);
})
});
server.listen(port, hostname, () => {
console.log(`http://${hostname}:${port}`);
})
//test.js
import { RequestHook } from "testcafe";
export class MyRequestHook extends RequestHook {
onRequest(event) {
const newBody = Buffer.from('Not test');
event.requestOptions.body = newBody;
event.requestOptions.headers['content-length'] = newBody.length;
}
onResponse(e) { }
}
const customHook = new MyRequestHook(/.*localhost.*/);
fixture('My fixture').requestHooks(customHook)
.page('https://devexpress.github.io/testcafe/example/');
test('First test', async t => {
const resp = await t.request.post('http://localhost:3001/', {
body: 'Test',
})
console.log(`Response body: ${resp.body}`);
});

API resolved without sending a response - Next.js

I've this code to get nearby places and nearby beaches from a point, with Google maps. This is called from a Next.js component, via the useSWR hook.
All the data is returned correctly, but before first Axios call (const fetchNearbyPlaces = async (urlWithToken = null) => {...), I'm receiving this error in the console:
API resolved without sending a response for /api/google/places/33.807501/-78.70039, this may result in stalled requests.
I can't figure out what the error is, although there may be several because I'm a novice. I appreciate any suggestion.
const axios = require("axios");
const GetNearbyPlaces = async (req, res) => {
const {
latitude,
longitude,
} = req.query;
const radius = 50000;
const types = [
"airport",
"tourist_attraction",
"amusement_park",
"aquarium",
"art_gallery",
"bar",
"museum",
"night_club",
"cafe",
"restaurant",
"shopping_mall",
"store",
"spa",
];
function checkFunc(arr, val) {
return arr.some(arrVal => val === arrVal);
}
const url = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=${latitude}%2C${longitude}&radius=${radius}&key=${process.env.CW_GOOGLE_MAPS_API_KEY}`;
const beachesUrl = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=${latitude}%2C${longitude}&radius=${radius}&type=natural_feature&key=${process.env.CW_GOOGLE_MAPS_API_KEY}`;
try {
let results = [];
let beaches = [];
const fetchNearbyBeaches = async (urlWithToken = null) => {
await axios.get(urlWithToken ? urlWithToken : beachesUrl).then(data => {
beaches = [...beaches, ...data.data.results];
if (data?.data?.next_page_token) {
const newUrl = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?key=${process.env.CW_GOOGLE_MAPS_API_KEY}&pagetoken=${data.data.next_page_token}`;
setTimeout(() => {
fetchNearbyBeaches(newUrl);
}, 2000);
} else {
beaches.length > 5 && beaches.splice(5);
results.length > 5 && results.splice(5);
const finalResults = [...beaches, ...results];
finalResults.length > 10 && finalResults.splice(10);
return res.status(200).json({
data: {
results: finalResults,
},
success: true,
});
}
});
};
const fetchNearbyPlaces = async (urlWithToken = null) => {
await axios.get(urlWithToken ? urlWithToken : url).then(data => {
results = [...results, ...data.data.results];
if (data?.data?.next_page_token) {
const newUrl = `https://maps.googleapis.com/maps/api/place/nearbysearch/json?key=${process.env.CW_GOOGLE_MAPS_API_KEY}&pagetoken=${data.data.next_page_token}`;
setTimeout(() => {
fetchNearbyPlaces(newUrl);
}, 2000);
} else {
const dirtyResultsWithDuplicates = [];
results.map(result => {
return types.map(type => {
if (checkFunc(result.types, type) && !result.types.includes("lodging")) {
dirtyResultsWithDuplicates.push(result);
}
});
});
const set = new Set(dirtyResultsWithDuplicates);
const filtered = Array.from(set);
results = filtered.length > 10 ? filtered.splice(10) : filtered;
return fetchNearbyBeaches();
}
});
};
fetchNearbyPlaces();
} catch (err) {
res.status(500).json({
message: err.message,
statusCode: 500,
});
}
};
export default GetNearbyPlaces;
The problem is with the backend application not the frontend component.
Nextjs expects a response to have been sent when the api handler function exits. If for example you have a databaseCall.then(sendResponse) in your api handler function what happens is that the handler function exits before the database returns.
Now this is not a problem if the database does return after that and sends the response, but it is if for example the database has an error. Because the handler function exits without a response already being sent Nextjs can't be sure that at that point there isn't a stalled request.
One way to fix this is by await-ing the db call(or whatever other async function you call) thereby preventing the handler function from exiting before some kind of response has been send.
The solution was added this object to mi API code.
export const config = {
api: {
externalResolver: true,
},
};
Documentation: https://nextjs.org/docs/api-routes/request-helpers

How to test Axios reject condition using Jest

I wrote a unit test for some Axios calls in my component. I verified the success path, where the call resolves successfully, but I am not able to verify the failure path, where the call rejects. How do I use mocks to verify this?
Here's a snippet of my FetchImage.vue component:
methods: {
preparedFetch() {
axios.get(this.imageurl).then(result => {
this.imageInformation.title = result.data.title;
this.imageInformation.copyright = result.data.copyright;
this.imageInformation.detailExplanation = result.data.explanation;
this.imageInformation.date = result.data.date;
this.imageInformation.urlinfo = result.data.url;
this.resultArrived = true;
this.$emit('imagefetched',this.imageInformation);
})
.catch( error => {
this.errorMessage = "Information not found";
this.resultArrived = true;
});
}
}
And my test for when the call rejects (for an invalid URL):
describe('Invalid response',async () => {
beforeEach(() => {
axios.get.mockClear();
axios.get.mockReturnValue(Promise.reject({}));
});
it('Invalid URL verfication', async () => {
// Given
const result = {
errorMessage : "Information not found",
resultArrived : true,
fetchStatus : true
};
// Fetch the error result
axios.get.mockReturnValue(Promise.resolve(result));
const fetchwrapper = mount(FetchImage);
fetchwrapper.vm.imageurl = "https://invalid.request.gov";
fetchwrapper.vm.preparedFetch();
await fetchwrapper.vm.$nextTick();
// Validate the result
expect(axios.get).not.toHaveBeenCalledWith('https://api.nasa.gov/planetary/apod?api_key=vME6LAMD7IhEiy7rDmjfIaG6MhiKbu1MNIqxtqd1');
expect(axios.get).toHaveBeenCalledWith("https://invalid.request.gov");
expect(axios.get).toHaveBeenCalledTimes(1);
expect(fetchwrapper.vm.errorMessage.length).not.toEqual(0);
expect(fetchwrapper.vm.errorMessage).toBe("Information not found");
});
});
Your catch block isn't running because the mock return value is using Promise.resolve() when it actually should be Promise.reject():
describe('Invalid response',async () => {
it('Invalid URL verfication', async () => {
// axios.get.mockReturnValue(Promise.resolve(result)); // DON'T DO THIS
axios.get.mockReturnValue(Promise.reject(result));
});
});
You have to reject the value by using the built-in jest method.
describe('Invalid response', async () => {
it('Invalid URL verfication', async () => {
axios.get.mockRejectedValue(result);
});
});

redux-observable you provided 'undefined' where a stream was expected

I'm using the fbsdk to get user details in an ajax request. So it makes sense to do this in a redux-observable epic. The way the fbsdk request goes, it doesn't have a .map() and .catch() it takes the success and failure callbacks:
code:
export const fetchUserDetailsEpic: Epic<*, *, *> = (
action$: ActionsObservable<*>,
store
): Observable<CategoryAction> =>
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => {
getDetails(store)
})
const getDetails = store => {
console.log(store)
let req = new GraphRequest(
'/me',
{
httpMethod: 'GET',
version: 'v2.5',
parameters: {
fields: {
string: 'email,first_name,last_name'
}
}
},
(err, res) => {
if (err) {
store.dispatch(fetchUserDetailsRejected(err))
} else {
store.dispatch(fetchUserDetailsFulfilled(res))
}
}
)
return new GraphRequestManager().addRequest(req).start()
}
It gives the error:
TypeError: You provided 'undefined' where a stream was expected. You
can provide an Observable, Promise, Array, or Iterable.
How do I return an observable from the epic so this error goes away?
Attempt at bindCallback from this SO answer:
const getDetails = (callBack, details) => {
let req = new GraphRequest(
'/me',
{
httpMethod: 'GET',
version: 'v2.5',
parameters: {
fields: {
string: 'email,first_name,last_name'
}
}
},
callBack(details)
)
new GraphRequestManager().addRequest(req).start()
}
const someFunction = (options, cb) => {
if (typeof options === 'function') {
cb = options
options = null
}
getDetails(cb, null)
}
const getDetailsObservable = Observable.bindCallback(someFunction)
export const fetchUserDetailsEpic: Epic<*, *, *> = (
action$: ActionsObservable<*>
): Observable<CategoryAction> =>
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => {
getDetailsObservable()
.mergeMap(details => {
return Observable.of(fetchUserDetailsFulfilled(details))
})
.catch(error => Observable.of(fetchUserDetailsRejected(error)))
})
Getting the same error
Looking into source code of GraphRequestManager .start:
start(timeout: ?number) {
const that = this;
const callback = (error, result, response) => {
if (response) {
that.requestCallbacks.forEach((innerCallback, index, array) => {
if (innerCallback) {
innerCallback(response[index][0], response[index][1]);
}
});
}
if (that.batchCallback) {
that.batchCallback(error, result);
}
};
NativeGraphRequestManager.start(this.requestBatch, timeout || 0, callback);
}
As you can see it does return nothing, so effectively undefined. Rx mergeMap requires an instance of Observable or something compatible with it (more info).
Since you dispatch further actions, you can modify your original code like that:
export const fetchUserDetailsEpic: Epic<*, *, *> = (
action$: ActionsObservable<*>,
store
): Observable<CategoryAction> =>
action$.ofType(FETCH_USER_DETAILS).do(() => { // .mergeMap changed to .do
getDetails(store)
})
const getDetails = store => {
console.log(store)
let req = new GraphRequest(
'/me',
{
httpMethod: 'GET',
version: 'v2.5',
parameters: {
fields: {
string: 'email,first_name,last_name'
}
}
},
(err, res) => {
if (err) {
store.dispatch(fetchUserDetailsRejected(err))
} else {
store.dispatch(fetchUserDetailsFulfilled(res))
}
}
)
return new GraphRequestManager().addRequest(req).start()
}
To be honest I find your second attempt bit better / less coupled. To make it working you could do something like:
const getDetails = Observable.create((observer) => {
let req = new GraphRequest(
'/me',
{
httpMethod: 'GET',
version: 'v2.5',
parameters: {
fields: {
string: 'email,first_name,last_name'
}
}
},
(error, details) => {
if (error) {
observer.error(error)
} else {
observer.next(details)
observer.complete()
}
}
)
new GraphRequestManager().addRequest(req).start()
})
export const fetchUserDetailsEpic: Epic<*, *, *> = (
action$: ActionsObservable<*>
): Observable<CategoryAction> =>
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => {
getDetails()
.map(details => fetchUserDetailsFulfilled(details)) // regular .map should be enough here
.catch(error => Observable.of(fetchUserDetailsRejected(error)))
})
I don't remember well how was working redux-observable before using RxJS >= 6 but I'll try to help ;)
First, you don't need to dispatch yourself, redux-observable will do it for you. In this article, they show how it works under the hood, so they call dispatch, but you don't have to. In the new implementation, they removed store as a second argument in favor of a state stream:
const epic = (action$, store) => { ... //before
const epic = (action$, state$) => { ... //after
But most importantly, the problem you experience is that you don't return a stream of actions, but a single (dispatched) action.
From their website:
It is a function which takes a stream of actions and returns a stream of actions.
So I think a quick solution would be to return observables from your callback:
(err, res) => {
if (err) {
return Observable.of(fetchUserDetailsRejected(err))
}
return Observable.of(fetchUserDetailsFulfilled(res))
}
I will update the answer based on your comments. Good luck!
I beleive this seems the possible reason for undefined. You are returning undefined in mergeMap callback.
This
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => {
getDetails(store)
})
should be either
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => getDetails(store))
or
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => {
return getDetails(store)
})
It looks like #artur grzesiak has a correct answer, but for completeness this is how I think bindCallback can be used.
The only issue I have with Artur's answer is I don't think we need to catch the error in the epic, since fetchUserDetailsRejected is an error-handling action (presumably the reducer deals with it appropriately).
I used this reference RxJs Static Public Methods: bindCallback
Give it a function f of type f(x, callback) and it will return a function g that when called as g(x) will output an Observable.
// This callback receives the results and returns one or other action (non-observable)
const callback = (err, res) => {
return err
? fetchUserDetailsRejected(err)
: fetchUserDetailsFulfilled(res)
}
// Here is the graph request uncluttered by concerns about the epic
const getDetails = (store, callback) => {
console.log(store)
let req = new GraphRequest(
'/me',
{
httpMethod: 'GET',
version: 'v2.5',
parameters: {
fields: {
string: 'email,first_name,last_name'
}
}
},
callback
)
new GraphRequestManager().addRequest(req).start()
}
// This bound function wraps the action returned from callback in an Observable
const getDetails$ = Observable.bindCallback(getDetails).take(1)
// The epic definition using bound callback to return an Observable action
export const fetchUserDetailsEpic: Epic<*, *, *> =
(action$: ActionsObservable<*>, store): Observable<CategoryAction> =>
action$.ofType(FETCH_USER_DETAILS).mergeMap(() => getDetails$(store))