Cloudflare sessions - cloudflare

I wanted to add a feature to my site similar to this:
When a new session starts look at the utm_source/utm_medium querystring values and also the referrer. Based on that display a differnt phone number of the site, so for example google cpc, bing cpc, google organic, bing organic would have different numbers.
The number of calls to each number should then give an indication of which traffic source generated the calls.
The problem is, because we're using clouldflare, if a user is served a page from the cache then there is no session_start event on the origin server.
Is there a solution to get around this? Is there anyway to do this on cloudflare itself, perhaps using its "workers"?
Thanks

Cloudflare workers can be used to accomplish that. The worker script will first need to determine which phone number to show. This can be done by checking the query params or a cookie or any other aspect of the request. Then the worker script can take the original response body (from the cache or the origin server) and replace all occurences of the original phone number with the new phone number.
Here's an example worker script that does that. To determine which phone number to show, it will first check the query parameters, as you mentioned. When it sees the utm_source query param, it will also set a cookie that can then be checked in all subsequent requests to show the same phone number.
// The name of the cookie that will be used to determine which phone number to show
const cookieName = "phone_num_id";
// The list of all phone numbers to use
const phoneNumbers = [
{
id: "google-cpc",
utmSource: "google",
utmMedium: "cpc",
phoneNumber: "222-222-2222"
},
{
id: "bing-cpc",
utmSource: "bing",
utmMedium: "cpc",
phoneNumber: "333-333-3333"
}
];
// This adds a "fetch" event listener which will be called for all incoming requests
addEventListener("fetch", event => {
event.respondWith(handleRequest(event.request));
});
async function handleRequest(request) {
// Forward the incoming request and get the original response. If Cloudflare has already cached
// this request, this will return the cached response, otherwise it will make the request to
// the origin
let response = await fetch(request);
// Check the content type of the response and fallback to an empty string
// if there is no content-type header
let contentType = response.headers.get("content-type") || "";
// We're only interested in changing respones that have a content-type starting
// with "text/html". Anything else will be returned without any modifications
if (/^text\/html/.test(contentType)) {
// `newPhoneNumberData` will be the new phone number to show (if any)
let newPhoneNumberData;
// searchParams are the query parameters for this request
let searchParams = new URL(request.url).searchParams;
// If the request has a `utm_source` query param, use that to determine which phone number to show
if (searchParams.has("utm_source")) {
let utmSource = searchParams.get("utm_source") || "";
let utmMedium = searchParams.get("utm_medium") || "";
// Lookup the phone number based on the `utmSource` and `utmMedium`
newPhoneNumberData = phoneNumbers.find(
phoneNumber =>
phoneNumber.utmSource === utmSource &&
phoneNumber.utmMedium === utmMedium
);
// If we found a match, set a cookie so that subsequent requests get the same phone number
if (newPhoneNumberData) {
// In order to modify the response headers, we first have to duplicate the response
// so that it becomes mutable
response = new Response(response.body, response);
// Now set a cookie with the id of the new phone number to use. You should modify the properties
// of the cookie for your use case. See this page for more information:
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie
response.headers.append(
"Set-Cookie",
`${cookieName}=${newPhoneNumberData.id}; Max-Age=2147483647`
);
}
}
// If we weren't able to determine the new phone number based on the query params, try
// checking the cookies next
if (!newPhoneNumberData) {
let cookieHeader = request.headers.get("cookie") || "";
// split each of the cookies and remove leading/trailing whitespace
let cookies = cookieHeader.split(";").map(str => str.trim());
// Find the phone number cookie
let phoneNumberCookieString = cookies.find(cookieString =>
cookieString.startsWith(`${cookieName}=`)
);
// If the request has the phone number cookie, use that
if (phoneNumberCookieString) {
// Extract the phone number id from the cookie
const phoneNumberId = phoneNumberCookieString.split("=")[1];
// Lookup the phone number data based on the ID
newPhoneNumberData = phoneNumbers.find(
phoneNumber => phoneNumber.id === phoneNumberId
);
}
}
// If we found a matching phone number to use, now we'll need to replace all occurences
// of the original phone number with the new one before returning the response
if (newPhoneNumberData) {
// Get the original response body
let responseBody = await response.text();
// Use a regex with the `g` flag to find/replace all occurences of the original phone number.
// This would look for a phone number like "(111)-111-1111" but you can modify this to fit
// however your original phone number appears
responseBody = responseBody.replace(
/\(?111\)?[-\s]*111[-\s]*1111/g,
newPhoneNumberData.phoneNumber
);
// Create a new response with the updated responseBody. We also pass the original `response` as the
// second argument in order to copy all other properties from the original response (status, headers, etc)
response = new Response(responseBody, response);
}
}
return response;
}

Related

How to securely store phone numbers in Twilio Functions?

I am using Twilio Functions. I wonder if the phone numbers stored in the Function's code are secure?
I am using code similar to that found here: https://support.twilio.com/hc/en-us/articles/223180548-How-Can-I-Stop-Receiving-or-Block-Incoming-Phone-Calls-#blacklistNumbers
Basically, it rejects numbers that on a blacklist of your choosing. The blacklist is in the Function's code itself.
Perhaps this is already secure. Forgive my misunderstanding.
The aforementioned code:
exports.handler = function(context, event, callback) {
// Listing all the blocked phone numbers, at the moment "+1(212)555-1234" and "+1(702)555-6789"
let blacklist = event.blacklist || [ "+12125551234", "+17025556789" ];
let twiml = new Twilio.twiml.VoiceResponse();
let blocked = true;
if (blacklist.length > 0) {
if (blacklist.indexOf(event.From) === -1) {
blocked = false;
}
}
if (blocked) {
twiml.reject();
}
else {
// if the caller's number is not blocked, redirecting to another TwiML which includes instructions for what to do
twiml.redirect("https://demo.twilio.com/docs/voice.xml");
}
callback(null, twiml);
};
Heyooo. Twilio Developer Evangelist here. 👋
I don't think you have to worry about the security of these numbers. But what you could do it to add the blacklist to the function configuration instead. This way you wouldn't have to change the function code whenever you want to add a new number to the blacklist.
These values below will be available in the context object that is passed into your function.
exports.handler = async function(context, event, callback) {
console.log(context.BLACKLIST); // 1212...
}

Getting SessionID During Socket.io Authorization?

I'm trying to get the sessionID from express-session when a new WebSocket connection comes in from a user. I'm able to find the sessionID, I just have a question about its format.
When I make a HTTP request to my messenger page say I get 'X' as a sessionID, if I then made a WebSocket connection I can find the session ID 'AXB', the session ID X is in there, but also surrounded with other information.
var express = require('express');
var app = express();
var server = require('http').createServer(app);
var session = require('express-session');
var io = require('socket.io')(server);
var store = new MemoryStore({
checkPeriod: 86400000
});
app.use(session({
store: store,
secret: 'jpcs-0001080900DRXPXL',
saveUninitialized: false,
resave: true
}));
// ...
app.get('/messenger/:uid', authorizationRedirect, (req, res) => {
console.log(req.sessionID);
// prints "EIVUudPTckmojrkv6FN9Cdb5NAQq5oQU"
// ...
});
io.set('authorization', (data, accept) => {
if (data && data.headers && data.headers.cookie) {
console.log(data.headers.cookie);
cookies_str = data.headers.cookie;
cookies_arr = cookies_str.split(';');
cookies = {};
for (index in cookies_arr) {
cookie = cookies_arr[index].split('=');
key = cookie[0].replace(/ /g,'');
val = cookie[1];
cookies[key] = val;
}
sessionId = cookies['connect.sid'].split('.')[0];
console.log(sessionId);
// prints "s%3AEIVUudPTckmojrkv6FN9Cdb5NAQq5oQU.AQkvP..."
// ...
});
So basically, in io.set('authorization', ...) I get:
s%3AEIVUudPTckmojrkv6FN9Cdb5NAQq5oQU.AQkvPsfoxieH3EAs8laFWN28dr1C%2B9zIT%2BMXtKTRPBg
But in app.get('/...', ...) I get:
EIVUudPTckmojrkv6FN9Cdb5NAQq5oQU
You can notice that the string from socket.io does contain the session id in this format: "s%3A" + sessionID + ".xxxxxxxxxxx..."
So obviously I can get the sessionID from here, but I'm curious why the sessionID is shown like this when I get socket connections? Will it ALWAYS be shown like this regardless of browser, WebSocket implementations, etc? What does the other information contained mean? I mostly want to make sure that this is a reliable way to get the sessionID. Thanks!
I would first like to clarify that io.set('authorization',...) has been deprecated. Here's the updated version Documentation
So obviously I can get the sessionID from here, but I'm curious why the sessionID is shown like this when I get socket connections? Will it ALWAYS be shown like this regardless of browser, WebSocket implementations, etc?
It's not reserved for socket connections at all. That is simply how it is fixed on the browser. So yes, it will always be shown like that.
What does the other information contained mean? I mostly want to make sure that this is a reliable way to get the sessionID. (s%3AEIVUudPTckmojrkv6FN9Cdb5NAQq5oQU.AQkvPsfoxieH3EAs8laFWN28dr1C%2B9zIT%2BMXtKTRPBg)
The first three characters are just encoded, and I believe every sessionID containts that. DecodedURIComponent("s%3A") = "s:"
After that is the sessionID itself (EIVUudPTckmojrkv6FN9Cdb5NAQq5oQU)
Now, after the dot(AQkvPsfoxieH3EAs8laFWN28dr1C%2B9zIT%2BMXtKTRPBg) is the signature portion. That verifies the authenticity of the cookie and is actually given when you sign the cookie. AND yes, I would say it is a trusted and reliable way.

How to generate authentication credentials for Google Sheets API?

Step 1 (completed):
As instructed, I have followed every step in Google's Node.js Quickstart:
https://developers.google.com/sheets/api/quickstart/nodejs
and it ran perfectly without any errors. My understanding was that after I run index.js, a token.json is created. After which, I can somehow use the token.json for any future authentication purpose without needing credentials.json every time. Please, correct me if I am wrong in this assumption. I am only making this assumption based on what Google is saying in the previously given link - "Authorization information is stored on the file system, so subsequent executions will not prompt for authorization."
Step 2 (problem):
Below is another code snippet is given by a Google Sheet's API's documentation page for reading multiple range data:
https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets.values/batchGet
I, however, do not know how to generate authentication credentials for authClient. Am I suppose to do something with token.json? If yes, what is the protocol of using the token.json file?
Please feel free to ask any follow-up questions if needed.
enter code here// BEFORE RUNNING:
// ---------------
// 1. If not already done, enable the Google Sheets API
// and check the quota for your project at
// https://console.developers.google.com/apis/api/sheets
// 2. Install the Node.js client library by running
// `npm install googleapis --save`
const {google} = require('googleapis');
var sheets = google.sheets('v4');
authorize(function(authClient) {
var request = {
// The ID of the spreadsheet to retrieve data from.
spreadsheetId: 'my-spreadsheet-id', // TODO: Update placeholder value.
// The A1 notation of the values to retrieve.
ranges: [], // TODO: Update placeholder value.
// How values should be represented in the output.
// The default render option is ValueRenderOption.FORMATTED_VALUE.
valueRenderOption: '', // TODO: Update placeholder value.
// How dates, times, and durations should be represented in the output.
// This is ignored if value_render_option is
// FORMATTED_VALUE.
// The default dateTime render option is [DateTimeRenderOption.SERIAL_NUMBER].
dateTimeRenderOption: '', // TODO: Update placeholder value.
auth: authClient,
};
sheets.spreadsheets.values.batchGet(request, function(err, response) {
if (err) {
console.error(err);
return;
}
// TODO: Change code below to process the `response` object:
console.log(JSON.stringify(response, null, 2));
});
});
function authorize(callback) {
// TODO: Change placeholder below to generate authentication credentials. See
// https://developers.google.com/sheets/quickstart/nodejs#step_3_set_up_the_sample
//
// Authorize using one of the following scopes:
// 'https://www.googleapis.com/auth/drive'
// 'https://www.googleapis.com/auth/drive.file'
// 'https://www.googleapis.com/auth/drive.readonly'
// 'https://www.googleapis.com/auth/spreadsheets'
// 'https://www.googleapis.com/auth/spreadsheets.readonly'
var authClient = null;
if (authClient == null) {
console.log('authentication failed');
return;
}
callback(authClient);
}

API Request Pagination

I am making a simple API request to Github to get all the repositories. The problem is that Github has a limitation and the max that it can send is 100 per request. There are users that have more than 100 repositories and I don't know how to access it or how to make pagination.
I am making GET request with Axios like this:
https://api.github.com/users/<AccountName>/repos?per_page=100
I can also put page number like so
https://api.github.com/users/<AccountName>/repos?page=3&per_page=100
But how do I make this work in app without making 10 API requests? I wouldn't even know how many requests I should make because I don't know what is the number that gets returned, does somebody have 100 or 1000 repos? I would like for everything to be returned and saved in array, for example.
EDIT:
Example: I am passing in accountName
var config = {
headers: {'Authorization': `token ${ACCESS_TOKEN}`}
}
const REQUEST: string = 'https://api.github.com/users/'
const apiCall = {
getData: async function (accountName) {
const encodedAccountName = encodeURIComponent(accountName)
const requestUrl = `${REQUEST}${encodedAccountName}`
const user = await axios.get(requestUrl, config)
// This return user and inside of user there is a link for fetching repos
const repo = await axios.get(`${user.data.repos_url}?per_page=100`, config)
...
You can get the repo count by requesting from the user account URL first. For example here is mine:
https://api.github.com/users/erikh2000
The response there includes a "public_repos" value. Bam! That's the magic number you want.
You next need to make multiple fetches if the repo count is over 100. I know you didn't want to, but hey... can't blame web services for trying to conserve their bandwidth. The good news is you can probably put them in a Promise.all() block and have them all fetch together and return at once. So code like...
const fetchAllTheRepos = (userName, repoCount) => {
const MAX_PER_PAGE = 100;
const baseUrl = 'https://api.github.com/users/' + userName +
'/repos?per_page=' + MAX_PER_PAGE;
//Start fetching every page of repos.
const fetchPromises = [], pageCount = Math.ceil(repoCount /
MAX_PER_PAGE);
for (let pageI = 1; pageI <= pageCount; ++pageI) {
const fetchPagePromise = fetch(baseUrl + '&page=' + pageI);
fetchPromises.push(fetchPagePromise);
}
//This promise resolves after all the fetching is done.
return Promise.all(fetchPromises)
.then((responses) => {
//Parse all the responses to JSON.
return Promise.all( responses.map((response) => response.json()) );
}).then((results) => {
//Copy the results into one big array that has all the friggin repos.
let repos = [];
results.forEach((result) => {
repos = repos.concat(result);
});
return repos;
});
};
//I left out the code to get the repo count, but that's pretty easy.
fetchAllTheRepos('erikh2000', 7).then((repos) => {
console.log(repos.length);
});
Simultaneously fetching all the pages may end up being more than Github wants to let you do at once for those accounts with lots of repos. I would put some "good citizen" limit on the number of repos you'll try to get at once, e.g. 1000. And then see if api.github.com agrees with your definition of a good citizen by watching for HTTP error responses. You can get into throttling solutions if needed, but probably a "grab it all at once" approach like above works fine.
On the other hand, if you are spidering through multiple accounts in one session, then maybe design the throttling in from the beginning just to you know... be nice. For that, look at a queue/worker pattern.

Stripe, is it possible to search a customer by their email?

Update: Since around January 2018, it is now possible to search using the email parameter on Stripe. See the accepted answer.
I was wondering if it was possible to search a customer only by their email address when using the Stripe API.
The documentation only indicates searching by:
created,
ending_before,
limit,
starting_after
But not email.
I'd like to avoid having to list over all my customers to find which ones have the same email addresses.
Stripe now allows you to filter customers by email.
https://stripe.com/docs/api#list_customers
Map<String, Object> options = new HashMap<>();
options.put("email", email);
List<Customer> customers = Customer.list(options).getData();
if (customers.size() > 0) {
Customer customer = customers.get(0);
...
This is important to help ensure you don't create duplicate customers. Because you can't put creating a customer in Stripe and the storage of the Stripe customer ID in your system inside a single transaction you need to build in some fail safes that check to see if a particular customer exists before you create a new one. Searching customers by email is important in that regard.
I did this by using the following API request. This was not available in stripe docs.I got this by tracking down their search in the dashboard area using Browser Developer Tools.
url :https://api.stripe.com/v1/search?query="+email+"&prefix=false",
method: GET
headers: {
"authorization": "Bearer Your_seceret Key",
"content-type": "application/x-www-form-urlencoded",
}
Warning This uses an undocumented API that is specific to the dashboard. While it might work today, there is no guarantee it will continue to work in the future.
You need to retrieve and store the Stripe customer ID along with the other customer details in your database. You can then search for the email address in your database and retrieve the customer record from Stripe by using the Stripe customer ID.
UPDATE: Stripe now allows searching via email
https://stripe.com/docs/api/php#list_customers
/**
* Remember that Stripe unfortunately allows multiple customers to have the same email address.
* #see https://stackoverflow.com/a/38492724/470749
*
* #param string $emailAddress
* #return array
*/
public function getCustomersByEmailAddress($emailAddress) {
try {
$matchingCustomers = [];
$lastResult = null;
$hasMoreResults = true;
while ($hasMoreResults) {
$searchResults = \Stripe\Customer::all([
"email" => $emailAddress,
"limit" => 100,
"starting_after" => $lastResult
]);
$hasMoreResults = $searchResults->has_more;
foreach ($searchResults->autoPagingIterator() as $customer) {
$matchingCustomers[] = $customer;
}
$lastResult = end($searchResults->data);
}
return $matchingCustomers;
} catch (\Exception $e) {
Log::error($e);
return [];
}
}
You can't directly search by email.
However, you can hack a little bit to list all users, and look after your email.
Here's my code (PHP) :
$last_customer = NULL;
$email = "EmailYou#AreLooking.for";
while (true) {
$customers = \Stripe\Customer::all(array("limit" => 100, "starting_after" => $last_customer));
foreach ($customers->autoPagingIterator() as $customer) {
if ($customer->email == $email) {
$customerIamLookingFor = $customer;
break 2;
}
}
if (!$customers->has_more) {
break;
}
$last_customer = end($customers->data);
}
You only need to write this line
\Stripe\Customer::all(["email" => "YourDesiredEmail"]);
In NodeJs we can search for our desired customer with their email address like the following:
const stripeSecretKey = process.env.STRIPE_SECRET_KEY;
const stripe = require('stripe')(stripeSecretKey);
const findCustomerByEmail = async (email) => {
try {
const customer = await stripe.customers.list( {
email: email,
limit: 1
});
if(customer.data.length !== 0){
return customer.data[0].id;
}
} catch (e) {
return (e);
}
};
The actual call to stripe is using the stripe.customers.list. If the email exists in our stripe account then the returned object will contain an element called data.
Using "list()" and "search()", you can get customers by email with these Python code below:
customers = stripe.Customer.list(
email="example#gmail.com",
)
customer = stripe.Customer.search(
query="email:'example#gmail.com'"
)
You can also limit customers to get with "limit" parameter as shown below:
customers = stripe.Customer.list(
email="example#gmail.com",
limit=3
)
customer = stripe.Customer.search(
query="email:'example#gmail.com'",
limit=3
)
Since you specified that
The documentation only indicate to search by created, ending_before, limit and starting_after, but no "email".
You are right, you can't search using emails.
If you still wish to do that, What you can do instead is to get a list of all the customer and filter on the response you get using email.
For Example, in ruby you can do it as follows:
customers = Stripe::Customer.all
customer_i_need = customers.select do |c|
c.email == "foo#bar.com"
end
PS: Stripe can have multiple customers associated with one email address.
Please bear in mind when using Stripe API that it is case sensitive email (which is a bit stupid). Hopefully they change this.
Stripe API does not supports any search-by-email feature. They have this search in their dashboard but not released any to API; from the architectural concept it seems that there is no possibility or plan from stripe to include this in API; every object in their API is retrievable only by that specific objects id given by stripe while its created. May be, they have kept it as a scope for third party application developers involvement!!
So, the obvious solution is to store the customers in your own database that you want to be searchable in future - as Simeon Visser has said above
btw, for a workaround, if you already have used the stripe API a lot and there are many customer data which you now need to be searchable - the only way is to go thru the 'List all customers' functionality of API & build the database for your own purpose; ofcourse, you've to use pagination shortcut to iterate thru the whole list for doing so.
$customers = \Stripe\Customer::all(array("limit" => 3));
foreach ($customers->autoPagingIterator() as $customer) {
// Do something with $customer
}
You can try this. It worked for me. Below code will return empty list if not found data matching with email.
$stripe = new \Stripe\StripeClient("YOUR_STRIPE_SECRET");
$customers = $stripe->customers->all(['email'=>'jane.doe#example.com',
'limit' => 15,
]);
Stripe Search API Beta now is available
youtube link
Here is The Async- Await Way This Method can Be Used For All Third Party Hits with Nodejs Particularly
const configuration = {
headers: {
"authorization": `Bearer ${Your stripe test key}`,
"content-type": "application/x-www-form-urlencoded",
}
};
const requestUrl = `https://api.stripe.com/v1/search?
query=${'email you are to use'} &prefix=false`
const emailPayment = await axios.get(requestUrl,
configuration)
Axiom is Npm for creating http requests... very cool and dead simple
Stripe allows the ability to have more than once customer with the same email. That said, if you wanted to, you can pass a filters hash param to the list method to return a single customer or array of customers.
Stripe::Customer.list(email: user.email).data
The above will return an array of Stripe::Customer instances with the email