Cloudflare worker scheduled response API KV - cloudflare

So previously I tried to store KV when CRON was processed and it works.
Now I tried to store KV based on API responses, and tried the following: https://gist.github.com/viggy28/522c4ed05e2bec051d3838ebaff27258 and Forward body from request to another url for Scheduled (CRON workers) but doesn't seems to work.
What I attempt to do is save response in KV when CRON triggers for access_token and use it in front-end code later on.
Attached is my current attempt:
addEventListener("scheduled", (event) => {
console.log("cron trigger processed..", event);
event.waitUntil(handleRequest());
});
async function handleRequest() {
const url = "<request_url>";
const body = { <credential> };
const init = {
body: JSON.stringify(body),
method: "POST",
headers: {
"content-type": "application/json;charset=UTF-8"
}
};
const response = await fetch(url, init);
const results = await gatherResponse(response);
//#ts-ignore
AUTH.put("attempt1", results);
//#ts-ignore
AUTH.put("attempt2", new Response(results, init));
return new Response(results, init);
}
/**
* gatherResponse awaits and returns a response body as a string.
* Use await gatherResponse(..) in an async function to get the response body
* #param {Response} response
*/
async function gatherResponse(response) {
const { headers } = response
const contentType = headers.get("content-type") || ""
if (contentType.includes("application/json")) {
return JSON.stringify(await response.json())
}
else if (contentType.includes("application/text")) {
return await response.text()
}
else if (contentType.includes("text/html")) {
return await response.text()
}
else {
return await response.text()
}
}
Both attempt doesn't seems to work (KV not saved):
//#ts-ignore
AUTH.put("attempt1", results);
//#ts-ignore
AUTH.put("attempt2", new Response(results, init));
Is there anything I'm missing?

The calls to NAMESPACE.put() return a Promise that should be awaited - I'd assume that since handleRequest() can return before those are completed (since you're missing the await), they're just being cancelled.
https://developers.cloudflare.com/workers/runtime-apis/kv/#writing-key-value-pairs

Related

How to cancel axios request?

I have TextInput and I need to send request every time when the text is changing
I have this code:
// Main.js
import Api from 'network/ApiManager';
const api = new Api();
// TextInput onChangeText function
const getSearch = useCallback(
async (searchName, sectorTypeId, type, filterData) => {
const result = await api.controller.search(searchName, sectorTypeId, type, filterData);
console.log(result)
},
[],
);
And i have this network layer
// NetworkManager.js
async getData(url) {
try {
const {data: response} = await axios.get(url);
return response;
} catch (e) {
return response;
}
}
controller = {
profile: async (search, sector, f_type, filterData = {}) => {
const res = await this.getData('/url/path');
return this.transformToOptions(res);
},
};
When onChangeText is called, I send a lot of requests, but I want to cancel previous requests and get the latest only. I know that I need to use CancelToken but I don't know how to pass it on my network layer
Please help
You can create a cancelToken, whenever a request comes, you can save the cancel token, when a new request comes, cancelToken won't be undefined, thus you can call cancelToken.cancel(). Try something like this:
let cancelToken
if (typeof cancelToken != typeof undefined) {
cancelToken.cancel("Operation canceled due to new request.")
}
//Save the cancel token for the current request
cancelToken = axios.CancelToken.source()
try {
const results = await axios.get(
`Your URL here`,
{ cancelToken: cancelToken.token } //Pass the cancel token
)

RN "TypeError: Network request failed" - production - random

I know some questions about the subject has been opened here and there, but my issue is different :
all the other ones appear in dev mode, in my case it's in production,
a very big percentage of requests pass, a few of them is TypeError: Network request failed - but sometimes for critical requests
it's random, not always the same request. Sometimes it passes, sometimes not.
it appears to three on my projects, one is on AWS the other one on Clever-Cloud, both are projects between 1000 and 5000 users, servers are quite too big for what they do - I think I removed the risk of a server fault. Even if... I can reproduce locally when I don't start the api locally. So it's like the api is not responding, but as I said, I don't think so.
I have no clue where to dig anymore...
I can give you my API.js service file, maybe you'll find what's wrong ?
import URI from 'urijs';
import { Platform } from 'react-native';
import NetInfo from '#react-native-community/netinfo';
import { getUserToken, wipeData } from '../utils/data';
import { SCHEME, MW_API_HOST } from '../config';
import deviceInfoModule from 'react-native-device-info';
import { capture } from '../utils/sentry';
const unauthorisedHandler = (navigation) => {
wipeData();
navigation.reset({ index: 0, routes: [{ name: 'Auth' }] });
};
const checkNetwork = async (test = false) => {
const isConnected = await NetInfo.fetch().then((state) => state.isConnected);
if (!isConnected || test) {
await new Promise((res) => setTimeout(res, 1500));
return false;
}
return true;
};
class ApiService {
host = MW_API_HOST;
scheme = SCHEME;
getUrl = (path, query) => {
return new URI().host(this.host).scheme(this.scheme).path(path).setSearch(query).toString();
};
execute = async ({ method = 'GET', path = '', query = {}, headers = {}, body = null }) => {
try {
const config = {
method,
headers: {
'Content-Type': 'application/json',
Accept: 'application/json',
appversion: deviceInfoModule.getBuildNumber(),
appdevice: Platform.OS,
currentroute: this.navigation?.getCurrentRoute?.()?.name,
...headers,
},
body: body ? JSON.stringify(body) : null,
};
const url = this.getUrl(path, query);
console.log('url: ', url);
const canFetch = await checkNetwork();
if (!canFetch) return;
let response;
// To try to avoid mysterious `TypeError: Network request failed` error
// that throws an error directly
// we try catch and try one more time.
try {
response = await fetch(url, config);
} catch (e) {
if (e?.toString().includes('Network request failed')) {
// try again
await new Promise((res) => setTimeout(res, 250));
console.log('try again because Network request failed');
response = await fetch(url, config);
} else {
throw e;
}
}
if (!response.ok) {
if (response.status === 401) {
const token = await getUserToken();
if (token) unauthorisedHandler(API.navigation);
return response;
}
}
if (response.json) return await response.json();
return response;
} catch (e) {
capture(e, { extra: { method, path, query, headers, body } });
return { ok: false, error: "Sorry, an error occured, technical team has been warned." };
}
};
executeWithToken = async ({ method = 'GET', path = '', query = {}, headers = {}, body = null }) => {
const token = await getUserToken();
if (token) headers.Authorization = token;
return this.execute({ method, path, query, headers, body });
};
get = async (args) => this.executeWithToken({ method: 'GET', ...args });
post = async (args) => this.executeWithToken({ method: 'POST', ...args });
put = async (args) => this.executeWithToken({ method: 'PUT', ...args });
delete = async (args) => this.executeWithToken({ method: 'DELETE', ...args });
}
const API = new ApiService();
export default API;
Talking with experts here and there, it seems that it's normal : internet network is not 100% reliable, so sometimes, request fail, for a reason that we can't anticipate (tunnel, whatever).
I ended up using fetch-retry and I still have a few of those, but much less !

How to return blob item from localForage in service worker?

in my service worker i store mp4 video in indexedDB with localforage library in a blob data. It's work ! but i don't know how can i return this blob data.
This is my fetchHandler code :
const fetchHandler = async (event) => {
const getResponse = async () => {
const request = event.request;
if( request.destination === 'video' ){
// Get from indexedDB
const value = await localforage.getItem('video');
// How return the video from indexedDB cache ?
if( value ) return value; // not working
// Add in indexedDB
var networkResponse = await fetch(event.request);
localforage.setItem('video', networkResponse.blob() ).then(function (value) {
// Do other things once the value has been saved.
console.log(value);
}).catch(function(err) {
// This code runs if there were any errors
console.log(err);
});
}else{
const openedCache = await caches.open(SW_CACHE_NAME);
const cacheResponse = await openedCache.match(request);
if (cacheResponse) return cacheResponse;
var networkResponse = await fetch(event.request);
const cachePutResponse = await openedCache.put(request, networkResponse.clone());
if (cachePutResponse) return cachePutResponse;
}
return networkResponse;
};
event.respondWith(getResponse());
};
thanks for your help
You need to pass a valid Response object to event.respondWith(). That entails a response body (which is what you get back from localforeage.getItem()), but also some response headers.
You can use the Response constructor to create that, and return it from your getResponse() function.
The code could look something like:
const value = await localforage.getItem('video');
if (value) {
// See https://fetch.spec.whatwg.org/#bodyinit for what's accepted
// as a BodyInit.
return new Response(value, {
headers: {
// Replace this with the actual MIME type for the video.
'content-type': 'video/mp4',
// Include any other headers here if desired.
}
});
}

Verify HMAC Hash Using Cloudflare Workers

I'm trying to verify a HMAC signature received from a WebHook. The details of the WebHook are https://cloudconvert.com/api/v2/webhooks#webhooks-events
This says that the HMAC is generated using hash_hmac (PHP) and is a SHA256 hash of the body - which is JSON. An example received is:
c4faebbfb4e81db293801604d0565cf9701d9e896cae588d73ddfef3671e97d7
This looks like lowercase hexits.
I'm trying to use Cloudflare Workers to process the request, however I can't verify the hash. My code is below:
const encoder = new TextEncoder()
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
async function handleRequest(request) {
const contentType = request.headers.get('content-type') || ''
const signature = request.headers.get('CloudConvert-Signature')
let data
await S.put('HEADER', signature)
if (contentType.includes('application/json')) {
data = await request.json()
await S.put('EVENT', data.event)
await S.put('TAG', data.job.tag)
await S.put('JSON', JSON.stringify(data))
}
const key2 = await crypto.subtle.importKey(
'raw',
encoder.encode(CCSigningKey2),
{ name: 'HMAC', hash: 'SHA-256' },
false,
['sign']
)
const signed2 = await crypto.subtle.sign(
'HMAC',
key2,
encoder.encode(JSON.stringify(data))
)
await S.put('V22', btoa(String.fromCharCode(...new Uint8Array(signed2))))
return new Response(null, {
status: 204,
headers: {
'Cache-Control': 'no-cache'
}
})
}
This will generate a hash of:
e52613e6ecebdf98bb085f04ca1f91bf9a5cf1dc085f89dcaa3e5fbf5ebf1b06
I've tried use the crypto.subtle.verify method, but that didn't work.
Can anyone see any issues with the code? Or have done this successfully using Cloudflare Workers?
Mark
I finally got this working using the verify method (I had previously tried the verify method, but it didn't work). The main problem seems to the use of request.json() wrapped in JSON.stringify. Changing this to request.text() resolved the issue. I can then use JSON.parse to access the data after verifying the signature. The code is as follows:
const encoder = new TextEncoder()
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
async function handleRequest(request) {
const signature = request.headers.get('CloudConvert-Signature')
const key = await crypto.subtle.importKey(
'raw',
encoder.encode(CCSigningKey2),
{ name: 'HMAC', hash: 'SHA-256' },
false,
['verify']
)
const data = await request.text()
const verified = await crypto.subtle.verify(
'HMAC',
key,
hexStringToArrayBuffer(signature),
encoder.encode(data)
)
if (!verified) {
return new Response('Verification failed', {
status: 401,
headers: {
'Cache-Control': 'no-cache'
}
})
}
return new Response(null, {
status: 204,
headers: {
'Cache-Control': 'no-cache'
}
})
}
function hexStringToArrayBuffer(hexString) {
hexString = hexString.replace(/^0x/, '')
if (hexString.length % 2 != 0) {
return
}
if (hexString.match(/[G-Z\s]/i)) {
return
}
return new Uint8Array(
hexString.match(/[\dA-F]{2}/gi).map(function(s) {
return parseInt(s, 16)
})
).buffer
}

filter query param not working through axios get from my vue js application?

whenever a user types anything into the textfield, an axios get request to the url
http://sandbox4.wootz.io:8080/api/data/1/action/?filter={id}like'%TE%' is made and it is supposed to return back all the filtered results based on the search(what user has typed) as a response. But currently rather than returning the filtered results as a response it is giving out all results(non-filtered results).
NOTE: I have tested the above mentioned URL through postman by making a get request and it gave out the filtered results perfectly.Why is the same not happening through my application code?plz help
getAsyncDataAction: debounce(function(name) {
if (!name.length) {
this.dataAction = [];
return;
}
this.isFetching = true;
api
.getSearchData(this.sessionData.key,`/action/?filter={id}like'%${name}%'`)
.then(response => {
this.dataAction = [];
response.forEach(item => {
this.dataAction.push(item);
});
console.log('action results are'+JSON.stringify(this.dataAction)) //displays all the results(non-filtered)
})
.catch(error => {
this.dataAction = [];
throw error;
})
.finally(() => {
this.isFetching = false;
});
}, 500),
api.js
import axios from 'axios';
const props = {
base_url: '/api/store',
search_url: '/api/entity/search',
cors_url: 'http://localhost',
oper_url: '/api'
};
axios.defaults.headers.get['Access-Control-Allow-Origin'] = props.cors_url;
axios.defaults.headers.post['Access-Control-Allow-Origin'] = props.cors_url;
axios.defaults.headers.patch['Access-Control-Allow-Origin'] = props.cors_url;
async function getSearchData(key, path) {
try {
console.log('inside getSearchData path value is'+path)
console.log('inside getSearchData and url for axios get is '+props.base_url + '/data' + path)
let response = await axios({
method: 'get',
url: props.base_url + '/data' + path,
headers: {'session_id': key}
});
if (response.status == 200) {
console.log(response.status);
}
return response.data;
} catch (err) {
console.error(err);
}
}
The problem is that you're not encoding the query string correctly. In particular, your % signs need to become %25
To do this, I highly recommend using the params options in Axios.
For example
async function getSearchData(key, path, params) { // 👈 added "params"
// snip
let response = await axios({
method: 'get',
url: `${props.base_url}/data${path}`,
params, // 👈 use "params" here
headers: {'session_id': key}
});
and call your function with
const params = {}
// check for empty or blank "name"
if (name.trim().length > 0) {
params.filter = `{id}like'%${name}%'`
}
api
.getSearchData(this.sessionData.key, '/action/', params)
Alternatively, encode the query parameter manually
const filter = encodeURIComponent(`{id}like'%${name}%'`)
const path = `/action/?filter=${filter}`
Which should produce something like
/action/?filter=%7Bid%7Dlike'%25TE%25'