What are cloudflare KV preview_ids and how to get one? - cloudflare

I have a following wrangler.toml. When I would like to use dev or preview (e.g. npx wrangler dev or npx wrangler preview) wrangler asks to add a preview_id to the KV namespaces. Is this an identifier to an existing KV namespace?
I see there is a ticket in Cloudflare Workers GH at https://github.com/cloudflare/wrangler/issues/1458 that tells this ought to be clarified but the ticket is closed with adding an error message
In order to preview a worker with KV namespaces, you must designate a preview_id in your configuration file for each KV namespace you'd like to preview."
which is the reason I'm here. :)
As for larger context I would be really glad if someone could clarify: I see that if I give a value of an existing namespace, I can preview and I see a KV namespace of type __some-worker-dev-1234-workers_sites_assets_preview is generated in Cloudflare. This has a different identifier than the KV namespace pointed by the identifier used in the preview_id and the KV namespace pointed by the identifier I used in preview_id is empty. Why does giving an identifier of an existing KV namespace remove the error message, deploys the assets and allow for previwing but the actual KV namespace is empty and a new one is created?
How do does kv-asset-handler know to look into this generated namespace to retrieve the assets?
I'm currently testing with the default generated Cloudare Worker to my site and I wonder if I have misunderstood something or if there is some mechanics that bundles during preview/publish the site namespace to the scipt.
If there is some random mechanics with automatic mapping, can this be then so that every developer can have their own private preview KV namespace?
type = "javascript"
name = "some-worker-dev-1234"
account_id = "<id>"
workers_dev = true
kv_namespaces = [
{ binding = "site_data", id = "<test-site-id>" }
]
[site]
# The location for the site.
bucket = "./dist"
# The entry directory for the package.json that contains
# main field for the file name of the compiled worker file in "main" field.
entry-point = ""
[env.production]
name = "some-worker-1234"
zone_id = "<zone-id>"
routes = [
"http://<site>/*",
"https://www.<site>/*"
]
# kv_namespaces = [
# { binding = "site_data", id = "<production-site-id>" }
# ]
import { getAssetFromKV, mapRequestToAsset } from '#cloudflare/kv-asset-handler'
/**
* The DEBUG flag will do two things that help during development:
* 1. we will skip caching on the edge, which makes it easier to
* debug.
* 2. we will return an error message on exception in your Response rather
* than the default 404.html page.
*/
const DEBUG = false
addEventListener('fetch', event => {
try {
event.respondWith(handleEvent(event))
} catch (e) {
if (DEBUG) {
return event.respondWith(
new Response(e.message || e.toString(), {
status: 500,
}),
)
}
event.respondWith(new Response('Internal Error', { status: 500 }))
}
})
async function handleEvent(event) {
const url = new URL(event.request.url)
let options = {}
/**
* You can add custom logic to how we fetch your assets
* by configuring the function `mapRequestToAsset`
*/
// options.mapRequestToAsset = handlePrefix(/^\/docs/)
try {
if (DEBUG) {
// customize caching
options.cacheControl = {
bypassCache: true,
}
}
const page = await getAssetFromKV(event, options)
// allow headers to be altered
const response = new Response(page.body, page)
response.headers.set('X-XSS-Protection', '1; mode=block')
response.headers.set('X-Content-Type-Options', 'nosniff')
response.headers.set('X-Frame-Options', 'DENY')
response.headers.set('Referrer-Policy', 'unsafe-url')
response.headers.set('Feature-Policy', 'none')
return response
} catch (e) {
// if an error is thrown try to serve the asset at 404.html
if (!DEBUG) {
try {
let notFoundResponse = await getAssetFromKV(event, {
mapRequestToAsset: req => new Request(`${new URL(req.url).origin}/404.html`, req),
})
return new Response(notFoundResponse.body, { ...notFoundResponse, status: 404 })
} catch (e) {}
}
return new Response(e.message || e.toString(), { status: 500 })
}
}
/**
* Here's one example of how to modify a request to
* remove a specific prefix, in this case `/docs` from
* the url. This can be useful if you are deploying to a
* route on a zone, or if you only want your static content
* to exist at a specific path.
*/
function handlePrefix(prefix) {
return request => {
// compute the default (e.g. / -> index.html)
let defaultAssetKey = mapRequestToAsset(request)
let url = new URL(defaultAssetKey.url)
// strip the prefix from the path for lookup
url.pathname = url.pathname.replace(prefix, '/')
// inherit all other props from the default request
return new Request(url.toString(), defaultAssetKey)
}
}

In case the format is not obvious (it wasn't to me) here is a sample config block from the docs with the preview_id specified for a couple of KV Namespaces:
kv_namespaces = [
{ binding = "FOO", id = "0f2ac74b498b48028cb68387c421e279", preview_id = "6a1ddb03f3ec250963f0a1e46820076f" },
{ binding = "BAR", id = "068c101e168d03c65bddf4ba75150fb0", preview_id = "fb69528dbc7336525313f2e8c3b17db0" }
]
You can generate a new namespace ID in the Workers KV section of the dashboard or with the Wrangler CLI:
wrangler kv:namespace create "SOME_NAMESPACE" --preview

This answer applies to versions of Wrangler >= 1.10.0
wrangler asks to add a preview_id to the KV namespaces. Is this an identifier to an existing KV namespace?
Yes! The reason there is a different identifier for preview namespaces is so that when developing with wrangler dev or wrangler preview you don't accidentally write changes to your existing production data with possibly buggy or incompatible code. You can add a --preview flag to most wrangler kv commands to interact with your preview namespaces.
For your situation here there are actually a few things going on.
You are using Workers Sites
You have a KV namespace defined in wrangler.toml
Workers Sites will automatically configure a production namespace for each environment you run wrangler publish on, and a preview namespace for each environment you run wrangler dev or wrangler preview on. If all you need is Workers Sites, then there is no need at all to specify a kv-namepsaces table in your manifest. That table is for additional KV namespaces that you may want to read data from or write data to. If that is what you need, you'll need to configure your own namespace and add id to wrangler.toml if you want to use wrangler publish, and preview_id (which should be different) if you want to use wrangler dev or wrangler preview.

Related

How to get API call origin in NextJS API endpoint

I have an API set up that receives a token, and I want to store that token in a database. But I also want to store the origin URL.
Let's say my API endpoint is located at https://myapp.com/api/connect
Now, I want to send a token from my website https://mywebsite.net
After I send a token, I want to be able to store the token and the website URL to the database in NextJS code.
My endpoint would store this info to the database:
{
token: someRandomToken
origin: https://mywebsite.net
}
I tried logging the whole req object from the handler to see if that info exist but the console log fills my terminal fast.
Inside Next's Server-Side environment you have access to req.headers.host as well as other headers set by Vercel's or other platforms' Reverse Proxies to tell the actual origin of the request, like this:
/pages/api/some-api-route.ts:
import { NextApiRequest } from "next";
const LOCAL_HOST_ADDRESS = "localhost:3000";
export default async function handler(req: NextApiRequest) {
let host = req.headers?.host || LOCAL_HOST_ADDRESS;
let protocol = /^localhost(:\d+)?$/.test(host) ? "http:" : "https:";
// If server sits behind reverse proxy/load balancer, get the "actual" host ...
if (
req.headers["x-forwarded-host"] &&
typeof req.headers["x-forwarded-host"] === "string"
) {
host = req.headers["x-forwarded-host"];
}
// ... and protocol:
if (
req.headers["x-forwarded-proto"] &&
typeof req.headers["x-forwarded-proto"] === "string"
) {
protocol = `${req.headers["x-forwarded-proto"]}:`;
}
let someRandomToken;
const yourTokenPayload = {
token: someRandomToken,
origin: protocol + "//" + host, // e.g. http://localhost:3000 or https://mywebsite.net
};
// [...]
}
Using Typescript is really helpful when digging for properties as in this case. I couldn't tell if you are using Typescript, but in case you don't, you'll have to remove NextApiRequest.

Scheduling localhost serverless cron: The model could not be resolved in registry

I'm hosting an ExpressJS/NodeJS API on AWS Lambda with Serverless framework. The API uses Knex.js and Bookshelf.js ORM.
I want to test scheduling a cron job locally. I'm using serverless-offline-scheduler to do this.
Question: My API runs fine if I call it from my client, but if I call a function via serverless scheduler, it complains that no models are in the registry. Why is this? I've already definitely included all necessary Models at the top of the OrderService.js file.
{
"errorMessage": "The model User could not be resolved from the registry plugin.",
"errorType": "Error",
"stackTrace": [
"Error: The model User could not be resolved from the registry plugin.",
" at new ModelNotResolved (/Users/danielturcotte/Sites/d2c/api_v4/node_modules/bookshelf/lib/plugins/registry.js:70:133)",
Serverless.yml:
functions:
app:
handler: handler.handler
events: ...
dequeue:
handler: ./services/OrderService.dequeue // Call dequeue function
events:
- schedule: rate(1 minute)
The handler calls root/services/OrderService.dequeue function, which contains
...
const dequeue = async function() {
await checkQueuedOrders();
};
module.exports = {
dequeue,
};
In my knexService.js file, I register Bookshelf models to the registry to remove circular dependencies:
const knexfile = require('./knexfile');
const config = require('./environment');
const environment = config.env.NODE_ENV || 'development';
const knex = require('knex')(knexfile[environment]);
knex.client.pool.numPendingCreates();
const bookshelf = require('bookshelf')(knex);
bookshelf.plugin('registry'); // Resolve circular dependencies with relations
bookshelf.plugin('visibility');
bookshelf.plugin('pagination');
module.exports.knex = knex;
module.exports.bookshelf = bookshelf;

How to change multiple sites dns in CloudFlare account?

I have a lot of sites on an CloudFlare account, sometimes when servers are migrate, i need to change every domain DNS in CF manually. How can I use some tool or script, that helps me to download all domains info, and than easy change it?
Maybe some Terraform example? I didnt use Terraform yet, so just thinking about ways how to automate this proccess.
Tnx.
Yes, you can use Terraform for this. There are an official Cloudflare Provider, the documentation for which you can find here.
When using the provider "directly", your Terraform configuration will look like this:
terraform {
required_providers {
cloudflare = {
source = "cloudflare/cloudflare"
version = ">= 3.12.1"
}
}
}
variable "cloudflare_api_token" {
type = string
sensitive = true
}
provider "cloudflare" {
api_token = var.cloudflare_api_token
}
resource "cloudflare_zone" "acme_com" {
zone = "acme.com"
}
You may be interested in the following Cloudflare resources to use them in your configuration:
cloudflare_zone
cloudflare_zone_settings_override
cloudflare_record
Also, you can use this module. Then your configuration may look like this:
terraform {
required_providers {
cloudflare = {
source = "cloudflare/cloudflare"
version = ">= 3.12.1"
}
}
}
variable "cloudflare_api_token" {
type = string
sensitive = true
}
provider "cloudflare" {
api_token = var.cloudflare_api_token
}
module "acme_com" {
source = "registry.terraform.io/alex-feel/zone/cloudflare"
version = "1.7.0"
zone = "acme.com"
}
There are examples to help you get started with the module.
And here is a concrete, ready-to-use example that you can use in your specific case when using the module:
terraform {
required_providers {
cloudflare = {
source = "cloudflare/cloudflare"
version = ">= 3.12.1"
}
}
}
variable "cloudflare_api_token" {
type = string
sensitive = true
}
provider "cloudflare" {
api_token = var.cloudflare_api_token
}
locals {
# All your zones go here
zones = ["acme.com", "example.com"]
# Your IP for A records for all the zones goes here
ip = "192.0.2.1"
}
module "all_domains" {
source = "registry.terraform.io/alex-feel/zone/cloudflare"
version = "1.7.0"
for_each = toset(local.zones)
zone = each.value
records = [
{
record_name = "a_main"
type = "A"
value = local.ip
}
]
}
In this case, it will be enough for you to list all your domains in the zones variable and specify the desired IP in the ip variable. As a result, an A record with the specified IP will be created for each of your domains.
To get all your zones you can use Cloudflare API List Zones method. So your request will look like this:
curl --request GET \
--url https://api.cloudflare.com/client/v4/zones \
--header 'Authorization: Bearer YOUR_TOKEN'

Mixing Wildcards and Parameters in Express

I have web content generated by Minecraft Overviewer in:
/home/mc/backups/servername/latest/overviewer
I have a very simple server with express/nodejs. Here is my app.js:
var express = require('express');
var app = express();
//----------------------------------------------------------------------------+
// Each server's root points to the latest overviewer page |
//----------------------------------------------------------------------------+
app.get('/minecraft/:server/*', function(req, res) {
console.log('HELLO?');
res.send('Finally some luck!');
/*
var
server = req.params.server,
file = req.params[0] ? req.params[0] : 'index.html',
dir = '../backups/' + server + '/latest/overviewer';
res.sendFile(file, { root: dir });
*/
});
app.use(express.static('www'));
app.listen(80, function () {
console.log('Web server listening on port 80');
});
I included a little more code than what is running so you can see my intent in case this is an xy kind of problem. I'm wanting to route static files, but I don't think I can use express.static because I want the URL to be mapped a little differently than my file structure (and based on a server name).
So what's the problem with my simple server? When I try and navigate to mysite.com/minecraft/isopre I see a white page saying Cannot GET /minecraft/isopre. If I remove the * from the end of the string I'm routing on line 7 I see Finally some luck!. But I want the star there so I can map mysite.com/minecraft/isopre to index.html or mysite.com/minecraft/isopre/overviewer.js.
So what's the right way to go about doing this?
In order to perform desirable operation i suggest you to use ? symbol for regular expression:
app.get('/minecraft/:server/:file?', function(req, res, next) {
if('undefined' != typeof req.params.file && req.params.file) {
var file = req.params.file;
}
if('undefined' != typeof req.params.server && req.params.server) {
var server = req.params.server;
}
});
In this case :file become optional and node.js won't fail your entire application if wildcard is absent.
Hereby, if req.params.file variable is undefined you could serve index.html file.

How to consume WCF soap web service in node.js

I tried lot of examples available in the net using node module wcf.js. But could not get any appropriate result. I'm using the below url
https://webservice.kareo.com/services/soap/2.1/KareoServices.svc?wsdl
Any one who can explain me with the help of code will be really helpful. I want to know how to access the wsdl in node.js
Thanks.
Please have a look at wcf.js
In short you can follow these steps:
npm install wcf.js
Write your code like this:
code
var Proxy = require('wcf.js').Proxy;
var BasicHttpBinding = require('wcf.js').BasicHttpBinding;
var binding = new BasicHttpBinding();
//Ensure the proxy variable created below has a working wsdl link that actually loads wsdl
var proxy = new Proxy(binding, "http://YourHost/YourService.svc?wsdl");
/*Ensure your message below looks like a valid working SOAP UI request*/
var message = "<soapenv:Envelope xmlns:soapenv='http://schemas.xmlsoap.org/soap/envelope/' xmlns:sil='http://YourNamespace'>" +
"<soapenv:Header/>" +
"<soapenv:Body>" +
"<sil:YourMethod>" +
"<sil:YourParameter1>83015348-b9dc-41e5-afe2-85e19d3703f9</sil:YourParameter1>" +
"<sil:YourParameter2>IMUT</sil:YourParameter2>" +
"</sil:YourMethod>" +
"</soapenv:Body>" +
"</soapenv:Envelope>";
/*The message that you created above, ensure it works properly in SOAP UI rather copy a working request from SOAP UI*/
/*proxy.send's second argument is the soap action; you can find the soap action in your wsdl*/
proxy.send(message, "http://YourNamespace/IYourService/YourMethod", function (response, ctx) {
console.log(response);
/*Your response is in xml and which can either be used as it is of you can parse it to JSON etc.....*/
});
You don't have that many options.
You'll probably want to use one of:
node-soap
douche
soapjs
i tried node-soap to get INR USD rate with following code.
app.get('/getcurr', function(req, res) {
var soap = require('soap');
var args = {FromCurrency: 'USD', ToCurrency: 'INR'};
var url = "http://www.webservicex.net/CurrencyConvertor.asmx?WSDL";
soap.createClient(url, function(err, client) {
client.ConversionRate(args, function(err, result) {
console.log(result);
});
});
});
Code Project has got a neat sample which uses wcf.js for which api's are wcf like so no need to learn new paradigm.
I think that an alternative would be to:
use a tool such as SoapUI to record input and output xml messages
use node request to form input xml message to send (POST) the request to the web service (note that standard javascript templating mechanisms such as ejs or mustache could help you here) and finally
use an XML parser to deserialize response data to JavaScript objects
Yes, this is a rather dirty and low level approach but it should work without problems
You'll probably want to use one of:
node-soap
douche
soapjs
Aslo, there's an existing question.
In my case, I used https://www.npmjs.com/package/soap. By default forceSoap12Headers option was set to false which prevented node-soap to generate correct soap message according to SOAP 1.2. Check for more details: I am confused about SOAP namespaces. After I set it to true, I was able to make a call to .NET WCF service. Here is a TypeScript code snipper that worked for me.
import * as soap from 'soap';
import { IOptions } from 'soap';
// ...
const url = 'https://www.your-domain.com/stock.svc?wsdl';
const opt: IOptions = {
forceSoap12Headers: true,
};
soap.createClient(url, opt, (err, client: soap.Client) => {
if (err) {
throw err;
}
const wsSecurityOptions = {
hasTimeStamp: false,
};
const wsSecurity = new soap.WSSecurity('username', 'password', wsSecurityOptions);
client.setSecurity(wsSecurity);
client.addSoapHeader({ Action: 'http://tempuri.org/API/GetStockDetail' }, undefined, 'wsa', 'http://www.w3.org/2005/08/addressing');
client.addSoapHeader({ To: 'https://www.your-domain.com/stock.svc' }, undefined, 'wsa', 'http://www.w3.org/2005/08/addressing');
const args = {
symbol: 'GOOG',
};
client.GetStockDetail(
args,
(requestErr, result) => {
if (requestErr) {
throw requestErr;
}
console.log(result);
},
);
});
Here couple links to the documentation of node-soap usage:
https://github.com/vpulim/node-soap/tree/master/test
https://github.com/vpulim/node-soap