Preload / Pre-populate PouchDB in React Native Application - react-native

Is it possible to preload / pre-populate a database in my React Native application and then the first time it is run, simply do a sync? I already have most, if not all of the database information before the app is distributed, it would be awesome if it just had to do a quick sync when the app is run. Any ideas how I would go about doing that?
I found this - https://pouchdb.com/2016/04/28/prebuilt-databases-with-pouchdb.html but it doesn't mention React Native
Using:
pouchdb-find: ^7.0.0
pouchdb-react-native: ^6.4.1
react: 16.3.1
react-moment: ^0.7.9
react-native: ~0.55.2
Thanks for any pointers.
Update Here is the code I'm using to try the loading of a dump file. This code exists in /screens/Home.js
The dump file is located in /client/dbfile/database.txt
var db = new PouchDB("cs1");
db.get("_local/initial_load_complete")
.catch(function(err) {
console.log("loading dumpfile");
if (err.status !== 404) {
// 404 means not found
throw err;
}
db.load("/client/dbfile/database.txt").then(function() {
return db.put({ _id: "_local/initial_load_complete" });
});
})
.then(function() {
// at this point, we are sure that
// initial replication is complete
console.log("loading is complete!");
return db.allDocs({ include_docs: true });
})
.then(
function(res) {
// display all docs for debugging purposes (empty)
console.log(res);
});
this.localDB = db;
When this runs my console displays this - showing there have been 0 rows added.
Object {
"offset": 0,
"rows": Array [],
"total_rows": 0,
}
Possible Unhandled Promise Rejection (id: 0):
Object {
"message": undefined,
"name": "unknown",
"status": 0,
}

In my project I have couple of db docs I distribute with app (translations JSON is the one good example).
So at app init I just try to read translations doc from db, if there is none - I import content from js module and store in db.
Then translations changes just being replicated from server to local db.
//transmob.js
const transMobFile = {
//content goes here
);
module.exports = transMobFile
//dbInit.js
import transMobFile from 'data/transMob';
..
PDB.getDoc('TransMob')
.then((doc)=> {
if (!doc) {
global.locales = transMobFile.localesMob; // locales
global.translations = transMobFile.langMob; // translations
return PDB.saveDoc('TransMob', transMobFile)
}
})

You can use react-native-fs to load a file from /android/app/src/main/assets. Just put the file into the assets folder and read it with RNFS.readFileAssets.
import PouchDB from 'pouchdb-core';
PouchDB
.plugin(require('pouchdb-find'))
.plugin(require('pouchdb-load'));
import RNFS from 'react-native-fs';
const localDB = new PouchDB("cs1", {adapter: 'asyncstorage'});
localDB.get("_local/initial_load_complete")
.catch(function(err) {
console.log("loading dumpfile");
if (err.status !== 404) {
// 404 means not found
throw err;
}
RNFS.readFileAssets('yourdb.txt', 'utf8')
.then((contents) => {
localDB.load(contents).then(function() {
return localDB.put({ _id: "_local/initial_load_complete" });
}).then(function() {
// at this point, we are sure that
// initial replication is complete
console.log("loading is complete!");
return localDB.allDocs({ include_docs: true }).then(
function(res) {
// display all docs for debugging purposes (empty)
console.log(res);
});
}, function(err) {
console.log(err);
});
})
})
You'll need to rebuild your project, reloading is not sufficient.
My project crashes when I attempt to load a 30MB file, so I probably will split it into a few smaller files. Check out https://github.com/pouchdb-community/pouchdb-load to see how this works if needed.

I found that the db.load() function from the pouchdb-load module requires a URL. I was pointing it to a file path on the device's filesystem. I placed my database.txt file on my server, changed it to use the url and it worked.
In my mind this isn't ideal because if they install the app and have slow wireless, it still has to pull the file from the server. It is still much faster than performing a full-on replicate when the app opens for the first time however.

Related

Sveltekit: Can't call API from +layout.ts. Error: "Failed to parse URL from /api/..."

This is the code in my +layout.ts
import type { LayoutLoad } from './$types';
export const load: LayoutLoad = async () => {
const response = await fetch('/api/thumbnails', { method: 'GET' })
if (response.ok) {
return { json: await response.json() }
}
else {
console.log('There were no blog posts to get.')
// TODO: do something!
}
};
I had the exact same code in a +page.ts file (the only difference was LayoutLoad was changed to PageLoad), and it worked there. I was calling my thumbnails api and populating my page. I don't understand why it doesn't work at all in +layout. It just crashes my web app.
On top of this, LayoutLoad has an error in VsCode that reads Module '"./$types"' has no exported member 'LayoutLoad'. I don't understand why this is. Can somebody help me?
I figured it out, the answer was here: https://kit.svelte.dev/docs/load#making-fetch-requests
To get data from an external API or a +server.js handler, you can use the provided fetch function, which behaves identically to the native fetch web API with a few additional features:
Basically, in my code I had to change this line
export const load: LayoutLoad = async () => {
to this
export const load: LayoutLoad = async ({fetch}) => {
Because without that change I was using the native fetch and not sveltekit's provided fetch.
Also, I seemed to have fixed this error: Module '"./$types"' has no exported member 'LayoutLoad' by updating sveltkit to the latest version.

VUE2 + Electron + Flask -> Python not spawning on build

I've just finished my first vue+electron+flask project and I am having quite a hard time trying to package it. Everything is workig "perfectly" when using "npm run electron:serve" but when running "npm run electron:build" I do not get any error, but Flask is not launched at all. I do not really know how to fix the problem, my guess is that when building the dist folder the path to app.py is not correct, but I tried to fix it without luck.
Here is the background.js code:
'use strict'
import { app, protocol, BrowserWindow } from 'electron'
import { createProtocol } from 'vue-cli-plugin-electron-builder/lib'
import installExtension, { VUEJS_DEVTOOLS } from 'electron-devtools-installer'
const isDevelopment = process.env.NODE_ENV !== 'production'
// Scheme must be registered before the app is ready
protocol.registerSchemesAsPrivileged([
{ scheme: 'app', privileges: { secure: true, standard: true } }
])
async function createWindow() {
// spawn flask app (https://medium.com/red-buffer/integrating-python-flask-backend-with-electron-nodejs-frontend-8ac621d13f72)
var python = require('child_process').spawn('py', ['../server/app.py']);
python.stdout.on('data', function (data) {
console.log("data: ", data.toString('utf8'));
});
python.stderr.on('data', (data) => {
console.log(`stderr: ${data}`); // when error
});
// Create the browser window.
const win = new BrowserWindow({
width: 1500,
height: 1200,
webPreferences: {
// Use pluginOptions.nodeIntegration, leave this alone
// See nklayman.github.io/vue-cli-plugin-electron-builder/guide/security.html#node-integration for more info
nodeIntegration: process.env.ELECTRON_NODE_INTEGRATION,
contextIsolation: !process.env.ELECTRON_NODE_INTEGRATION
}
})
if (process.env.WEBPACK_DEV_SERVER_URL) {
// Load the url of the dev server if in development mode
await win.loadURL(process.env.WEBPACK_DEV_SERVER_URL)
if (!process.env.IS_TEST) win.webContents.openDevTools()
} else {
createProtocol('app')
// Load the index.html when not in development
win.loadURL('app://./index.html')
}
}
// Quit when all windows are closed.
app.on('window-all-closed', () => {
// On macOS it is common for applications and their menu bar
// to stay active until the user quits explicitly with Cmd + Q
if (process.platform !== 'darwin') {
app.quit()
}
})
app.on('activate', () => {
// On macOS it's common to re-create a window in the app when the
// dock icon is clicked and there are no other windows open.
if (BrowserWindow.getAllWindows().length === 0) createWindow()
})
// This method will be called when Electron has finished
// initialization and is ready to create browser windows.
// Some APIs can only be used after this event occurs.
app.on('ready', async () => {
if (isDevelopment && !process.env.IS_TEST) {
// Install Vue Devtools
try {
await installExtension(VUEJS_DEVTOOLS)
} catch (e) {
console.error('Vue Devtools failed to install:', e.toString())
}
}
createWindow()
})
// Exit cleanly on request from parent process in development mode.
if (isDevelopment) {
if (process.platform === 'win32') {
process.on('message', (data) => {
if (data === 'graceful-exit') {
app.quit()
}
})
} else {
process.on('SIGTERM', () => {
app.quit()
})
}
}
The relevant part of the code calling app.py is the following:
async function createWindow() {
// spawn flask app (https://medium.com/red-buffer/integrating-python-flask-backend-with-electron-nodejs-frontend-8ac621d13f72)
var python = require('child_process').spawn('py', ['../server/app.py']);
python.stdout.on('data', function (data) {
console.log("data: ", data.toString('utf8'));
});
python.stderr.on('data', (data) => {
console.log(`stderr: ${data}`); // when error
});
// Create the browser window.
const win = new BrowserWindow({
width: 1500,
height: 1200,
webPreferences: {
// Use pluginOptions.nodeIntegration, leave this alone
// See nklayman.github.io/vue-cli-plugin-electron-builder/guide/security.html#node-integration for more info
nodeIntegration: process.env.ELECTRON_NODE_INTEGRATION,
contextIsolation: !process.env.ELECTRON_NODE_INTEGRATION
}
})
I tried to put 3 dots insted of 2 in the app.py path ['.../server/app.py] just in case when creating the dist folder I need this extra dot to find the app.py file, but this is not working either.
My folder structure is the follwing:
Vue-Electron
client
dist_electron
node_modules
public
src
assets
components
router
views
App.vue
background.js
main.js
other config files
server
data
env
app.py
requirements.txt
other python scripts imported to app.py
sqlite_portofolio.db
As this program will only be used by me in my personal pc, I did not want to bother using pyInstaller (I thought it would be easier to not package the python side, but if I am wrong please let me know). I would like to have a electron .exe file that I can just doble click to open the electron build and then spawn the Flask server.
Also, my feeling is that I am not killing the Flask server correctly when closing the app. I think Flask is still running when closing electron. What should I do to ensure Flask server is properly closed.
There is not a lot of information of those topics that I can follow. Any help will be aprreaciated.
I´m having the same problem. I followed the link to this article (https://medium.com/red-buffer/integrating-python-flask-backend-with-electron-nodejs-frontend-8ac621d13f72), and it has the answer about killing the python flask server. And if you follow everything the article says, it's supposed to run the backend when opening the electron.exe, but this is not happening here on my end.
EDIT: I found the error, you need to change the path on your spawn. I sugest you to run the electron.exe on the cmd so you can see the error on it, so you will see the path that spawn is trying to run.
it´s probably:
var python = require('child_process').spawn('py', ['../resources/app/server/app.py']);
you will need to acess the app.py through [resources/app] as spawn start at the base dir of the electron build.
PS: I used electron-packeger that´s why mine need to add resources/app, and I used pyinstaller on my backend
Hope it will help you.

Is there any way in Nuxt3 to load a plugin once?

I'm trying to integrate Sequelize to my Nuxt 3 project. However, I couldn't figure out how to make it load only once instead of reloading it every time the page was refreshed / navigating to another routes.
I couldn't find any information on the docs. Is it even possible?
~/plugins/sequelize.server.ts
import { Sequelize } from "sequelize"
export default defineNuxtPlugin(async (nuxtApp) => {
const config = useRuntimeConfig()
const sequelize = new Sequelize(config.dbName, config.dbUser, config.dbPass,{
host: config.dbHost,
port: parseInt(config.dbPort),
dialect: 'mysql',
})
try {
await sequelize.authenticate()
// this log was executed every time I navigate to a new route
// or refreshing the browser.
console.log('Connection has been established successfully.');
} catch (error) {
console.error('Unable to connect to the database:', error);
}
return {
provide: {
db: sequelize
}
}
})
OP solved his issue by removing a composable that was initialized on a component's mounted lifecycle hook.
Just a remaining piece of code.

react-native (Expo) upload file on background

In my Expo (react-native) application, I want to do the upload task even if the application is in the background or killed.
the upload should be done to firebase storage, so we don't have a REST API.
checked out the Expo task manager library, but I could not figure out how it should be done. is it even possible to achieve this goal with Expo? is the TaskManager the correct package for this task?
there are only some Expo packages that could be registered as a task (e.g. backgroundFetch), and it is not possible to register a custom function (in this case uploadFile method).
I even got more confused as we should enable add UIBackgroundModes key for iOS but it only has audio,location,voip,external-accessory,bluetooth-central,bluetooth-peripheral,fetch,remote-notification,processing as possible values.
I would appreciate it if you can at least guide me on where to start or what to search for, to be able to upload the file even if the app is in the background is killed/terminated.
import { getStorage, ref, uploadBytes } from "firebase/storage";
const storage = getStorage();
const storageRef = ref(storage, 'videos');
const uploadFile = async (file)=>{
// the file is Blob object
await uploadBytes(storageRef, file);
}
I have already reviewed react-native-background-fetch, react-native-background-upload, react-native-background-job . upload should eject Expo, job does not support iOS, and fetch is a fetching task designed for doing task in intervals.
if there is a way to use mentioned libraries for my purpose, please guide me :)
to my understanding, the Firebase Cloud JSON API does not accept files, does it ? if so please give me an example. If I can make storage json API work with file upload, then I can use Expo asyncUpload probably without ejecting.
I have done something similar like you want, you can use expo-task-manager and expo-background-fetch. Here is the code as I used it. I Hope this would be useful for you.
import * as BackgroundFetch from 'expo-background-fetch';
import * as TaskManager from 'expo-task-manager';
const BACKGROUND_FETCH_TASK = 'background-fetch';
const [isRegistered, setIsRegistered] = useState(false);
const [status, setStatus] = useState(null);
//Valor para que se ejecute en IOS
BackgroundFetch.setMinimumIntervalAsync(60 * 15);
// Define the task to execute
TaskManager.defineTask(BACKGROUND_FETCH_TASK, async () => {
const now = Date.now();
console.log(`Got background fetch call at date: ${new Date(now).toISOString()}`);
// Your function or instructions you want
return BackgroundFetch.Result.NewData;
});
// Register the task in BACKGROUND_FETCH_TASK
async function registerBackgroundFetchAsync() {
return BackgroundFetch.registerTaskAsync(BACKGROUND_FETCH_TASK, {
minimumInterval: 60 * 15, // 1 minutes
stopOnTerminate: false, // android only,
startOnBoot: true, // android only
});
}
// Task Status
const checkStatusAsync = async () => {
const status = await BackgroundFetch.getStatusAsync();
const isRegistered = await TaskManager.isTaskRegisteredAsync(
BACKGROUND_FETCH_TASK
);
setStatus(status);
setIsRegistered(isRegistered);
};
// Check if the task is already register
const toggleFetchTask = async () => {
if (isRegistered) {
console.log('Task ready');
} else {
await registerBackgroundFetchAsync();
console.log('Task registered');
}
checkStatusAsync();
};
useEffect(() => {
toggleFetchTask();
}, []);
Hope this isn't too late to be helpful.
I've been dealing with a variety of expo <-> firebase storage integrations recently, and here's some info that might be helpful.
First, I'd recommend not using the uploadBytes / uploadBytesResumable methods from Firebase. This Thread has a long ongoing discussion about it, but basically it's broken in v9. Maybe in the future the Firebase team will solve the issues, but it's pretty broken with Expo right now.
Instead, I'd recommend either going down the route of writing a small Firebase function that either gives a signed-upload-url or handles the upload itself.
Basically, if you can get storage uploads to work via an http endpoint, you can get any kind of upload mechanism working. (e.g. the FileSystem.uploadAsync() method you're probably looking for here, like #brentvatne pointed out, or fetch, or axios. I'll show a basic wiring at the end).
Server Side
Option 1: Signed URL Upload.
Basically, have a small firebase function that returns a signed url. Your app calls a cloud function like /get-signed-upload-url , which returns the url, which you then use. Check out: https://cloud.google.com/storage/docs/access-control/signed-urls for how you'd go about this.
This might work well for your use case. It can be configured just like any httpsCallable function, so it's not much work to set up, compared to option 2.
However, this doesn't work for the firebase storage / functions emulator! For this reason, I don't use this method, because I like to intensively use the emulators, and they only offer a subset of all the functionalities.
Option 2: Upload the file entirely through a function
This is a little hairier, but gives you a lot more fidelity over your uploads, and will work on an emulator! I like this too because it allows doing upload process within the endpoint execution, instead of as a side effect.
For example, you can have a photo-upload endpoint generate thumbnails, and if the endpoint 201's, then you're good! Rather than the traditional Firebase approach of having a listener to cloud storage which would generate thumbnails as a side effect, which then has all kinds of bad race conditions (checking for processing completion via exponentiational backoff? Gross!)
Here are three resources I'd recommend to go about this approach:
https://cloud.google.com/functions/docs/writing/http#multipart_data
https://github.com/firebase/firebase-js-sdk/issues/5848
https://github.com/mscdex/busboy
Basically, if you can make a Firebase cloud endpoint that accepts a File within formdata, you can have busboy parse it, and then you can do anything you want with it... like upload it to Cloud Storage!
an outline of this:
import * as functions from "firebase-functions";
import * as busboy from "busboy";
import * as os from "os";
import * as path from "path";
import * as fs from "fs";
type FieldMap = {
[fieldKey: string]: string;
};
type Upload = {
filepath: string;
mimeType: string;
};
type UploadMap = {
[fileName: string]: Upload;
};
const MAX_FILE_SIZE = 2 * 1024 * 1024; // 2MB
export const uploadPhoto = functions.https.onRequest(async (req, res) => {
verifyRequest(req); // Verify parameters, auth, etc. Better yet, use a middleware system for this like express.
// This object will accumulate all the fields, keyed by their name
const fields: FieldMap = {};
// This object will accumulate all the uploaded files, keyed by their name.
const uploads: UploadMap = {};
// This will accumulator errors during the busboy process, allowing us to end early.
const errors: string[] = [];
const tmpdir = os.tmpdir();
const fileWrites: Promise<unknown>[] = [];
function cleanup() {
Object.entries(uploads).forEach(([filename, { filepath }]) => {
console.log(`unlinking: ${filename} from ${path}`);
fs.unlinkSync(filepath);
});
}
const bb = busboy({
headers: req.headers,
limits: {
files: 1,
fields: 1,
fileSize: MAX_FILE_SIZE,
},
});
bb.on("file", (name, file, info) => {
verifyFile(name, file, info); // Verify your mimeType / filename, etc.
file.on("limit", () => {
console.log("too big of file!");
});
const { filename, mimeType } = info;
// Note: os.tmpdir() points to an in-memory file system on GCF
// Thus, any files in it must fit in the instance's memory.
console.log(`Processed file ${filename}`);
const filepath = path.join(tmpdir, filename);
uploads[filename] = {
filepath,
mimeType,
};
const writeStream = fs.createWriteStream(filepath);
file.pipe(writeStream);
// File was processed by Busboy; wait for it to be written.
// Note: GCF may not persist saved files across invocations.
// Persistent files must be kept in other locations
// (such as Cloud Storage buckets).
const promise = new Promise((resolve, reject) => {
file.on("end", () => {
writeStream.end();
});
writeStream.on("finish", resolve);
writeStream.on("error", reject);
});
fileWrites.push(promise);
});
bb.on("close", async () => {
await Promise.all(fileWrites);
// Fail if errors:
if (errors.length > 0) {
functions.logger.error("Upload failed", errors);
res.status(400).send(errors.join());
} else {
try {
const upload = Object.values(uploads)[0];
if (!upload) {
functions.logger.debug("No upload found");
res.status(400).send("No file uploaded");
return;
}
const { uploadId } = await processUpload(upload, userId);
cleanup();
res.status(201).send({
uploadId,
});
} catch (error) {
cleanup();
functions.logger.error("Error processing file", error);
res.status(500).send("Error processing file");
}
}
});
bb.end(req.rawBody);
});
Then, that processUpload function can do anything you want with the file, like upload it to cloud storage:
async function processUpload({ filepath, mimeType }: Upload, userId: string) {
const fileId = uuidv4();
const bucket = admin.storage().bucket();
await bucket.upload(filepath, {
destination: `users/${userId}/${fileId}`,
{
contentType: mimeType,
},
});
return { fileId };
}
Mobile Side
Then, on the mobile side, you can interact with it like this:
async function uploadFile(uri: string) {
function getFunctionsUrl(): string {
if (USE_EMULATOR) {
const origin =
Constants?.manifest?.debuggerHost?.split(":").shift() || "localhost";
const functionsPort = 5001;
const functionsHost = `http://${origin}:${functionsPort}/{PROJECT_NAME}/${PROJECT_LOCATION}`;
return functionsHost;
} else {
return `https://{PROJECT_LOCATION}-{PROJECT_NAME}.cloudfunctions.net`;
}
}
// The url of your endpoint. Make this as smart as you want.
const url = `${getFunctionsUrl()}/uploadPhoto`;
await FileSystem.uploadAsync(uploadUrl, uri, {
httpMethod: "POST",
uploadType: FileSystem.FileSystemUploadType.MULTIPART,
fieldName: "file", // Important! make sure this matches however you want bussboy to validate the "name" field on file.
mimeType,
headers: {
"content-type": "multipart/form-data",
Authorization: `${idToken}`,
},
});
});
TLDR
Wrap Cloud Storage in your own endpoint, treat it like a normal http upload, everything plays nice.

Nuxt: Inside a plugin, how to add dynamic script tag to head?

I'm trying to build a Google Analytics plugin to Nuxt that will fetch tracking IDs from the CMS. I am really close I think.
I have a plugin file loading on client side only. The plugin is loaded from nuxt.config.js via the plugins:[{ src: '~/plugins/google-gtag.js', mode: 'client' }] array.
From there the main problem is that the gtag script needs the UA code in it's URL, so I can't just add that into the regular script object in nuxt.config.js. I need to get those UA codes from the store (which is hydrated form nuxtServerInit.
So I'm using head.script.push in the plugin to add the gtag script with the UA code in the URL. But that doesn't result in the script being added on first page load, but it does for all subsequent page transitions. So clearly I'm running head.script.push too late in the render of the page.
But I don't know how else to fetch tracking IDs, then add script's to the head.
// plugins/google.gtag.client.js with "mode": "client
export default ({ store, app: { head, router, context } }, inject) => {
// Remove any empty tracking codes
const codes = store.state.siteMeta.gaTrackingCodes.filter(Boolean)
// Add script tag to head
head.script.push({
src: `https://www.googletagmanager.com/gtag/js?id=${codes[0]}`,
async: true
})
console.log('added script')
// Include Google gtag code and inject it (so this.$gtag works in pages/components)
window.dataLayer = window.dataLayer || []
function gtag() {
dataLayer.push(arguments)
}
inject('gtag', gtag)
gtag('js', new Date())
// Add tracking codes from Vuex store
codes.forEach(code => {
gtag('config', code, {
send_page_view: false // necessary to avoid duplicated page track on first page load
})
console.log('installed code', code)
// After each router transition, log page event to Google for each code
router.afterEach(to => {
gtag('event', 'page_view', { page_path: to.fullPath })
console.log('afterEach', code)
})
})
}
I ended up getting this to work and we use it in production here.
Code as of this writing looks like this:
export default ({ store, app: { router, context } }, inject) => {
// Remove any empty tracking codes
let codes = _get(store, "state.siteMeta.gaTrackingCodes", [])
codes = codes.filter(Boolean)
// Abort if no codes
if (!codes.length) {
if (context.isDev) console.log("No Google Anlaytics tracking codes set")
inject("gtag", () => {})
return
}
// Abort if in Dev mode, but inject dummy functions so $gtag events don't throw errors
if (context.isDev) {
console.log("No Google Anlaytics tracking becuase your are in Dev mode")
inject("gtag", () => {})
return
}
// Abort if we already added script to head
let gtagScript = document.getElementById("gtag")
if (gtagScript) {
return
}
// Add script tag to head
let script = document.createElement("script")
script.async = true
script.id = "gtag"
script.src = "//www.googletagmanager.com/gtag/js"
document.head.appendChild(script)
// Include Google gtag code and inject it (so this.$gtag works in pages/components)
window.dataLayer = window.dataLayer || []
function gtag() {
dataLayer.push(arguments)
}
inject("gtag", gtag)
gtag("js", new Date())
// Add tracking codes from Vuex store
codes.forEach(code => {
gtag("config", code, {
send_page_view: false // Necessary to avoid duplicated page track on first page load
})
// After each router transition, log page event to Google for each code
router.afterEach(to => {
gtag("event", code, { page_path: to.fullPath })
})
})
}
If not in a plug-in, this was a good read on how to load 3rd party scripts: How to Load Third-Party Scripts in Nuxt.js