In my Expo (react-native) application, I want to do the upload task even if the application is in the background or killed.
the upload should be done to firebase storage, so we don't have a REST API.
checked out the Expo task manager library, but I could not figure out how it should be done. is it even possible to achieve this goal with Expo? is the TaskManager the correct package for this task?
there are only some Expo packages that could be registered as a task (e.g. backgroundFetch), and it is not possible to register a custom function (in this case uploadFile method).
I even got more confused as we should enable add UIBackgroundModes key for iOS but it only has audio,location,voip,external-accessory,bluetooth-central,bluetooth-peripheral,fetch,remote-notification,processing as possible values.
I would appreciate it if you can at least guide me on where to start or what to search for, to be able to upload the file even if the app is in the background is killed/terminated.
import { getStorage, ref, uploadBytes } from "firebase/storage";
const storage = getStorage();
const storageRef = ref(storage, 'videos');
const uploadFile = async (file)=>{
// the file is Blob object
await uploadBytes(storageRef, file);
}
I have already reviewed react-native-background-fetch, react-native-background-upload, react-native-background-job . upload should eject Expo, job does not support iOS, and fetch is a fetching task designed for doing task in intervals.
if there is a way to use mentioned libraries for my purpose, please guide me :)
to my understanding, the Firebase Cloud JSON API does not accept files, does it ? if so please give me an example. If I can make storage json API work with file upload, then I can use Expo asyncUpload probably without ejecting.
I have done something similar like you want, you can use expo-task-manager and expo-background-fetch. Here is the code as I used it. I Hope this would be useful for you.
import * as BackgroundFetch from 'expo-background-fetch';
import * as TaskManager from 'expo-task-manager';
const BACKGROUND_FETCH_TASK = 'background-fetch';
const [isRegistered, setIsRegistered] = useState(false);
const [status, setStatus] = useState(null);
//Valor para que se ejecute en IOS
BackgroundFetch.setMinimumIntervalAsync(60 * 15);
// Define the task to execute
TaskManager.defineTask(BACKGROUND_FETCH_TASK, async () => {
const now = Date.now();
console.log(`Got background fetch call at date: ${new Date(now).toISOString()}`);
// Your function or instructions you want
return BackgroundFetch.Result.NewData;
});
// Register the task in BACKGROUND_FETCH_TASK
async function registerBackgroundFetchAsync() {
return BackgroundFetch.registerTaskAsync(BACKGROUND_FETCH_TASK, {
minimumInterval: 60 * 15, // 1 minutes
stopOnTerminate: false, // android only,
startOnBoot: true, // android only
});
}
// Task Status
const checkStatusAsync = async () => {
const status = await BackgroundFetch.getStatusAsync();
const isRegistered = await TaskManager.isTaskRegisteredAsync(
BACKGROUND_FETCH_TASK
);
setStatus(status);
setIsRegistered(isRegistered);
};
// Check if the task is already register
const toggleFetchTask = async () => {
if (isRegistered) {
console.log('Task ready');
} else {
await registerBackgroundFetchAsync();
console.log('Task registered');
}
checkStatusAsync();
};
useEffect(() => {
toggleFetchTask();
}, []);
Hope this isn't too late to be helpful.
I've been dealing with a variety of expo <-> firebase storage integrations recently, and here's some info that might be helpful.
First, I'd recommend not using the uploadBytes / uploadBytesResumable methods from Firebase. This Thread has a long ongoing discussion about it, but basically it's broken in v9. Maybe in the future the Firebase team will solve the issues, but it's pretty broken with Expo right now.
Instead, I'd recommend either going down the route of writing a small Firebase function that either gives a signed-upload-url or handles the upload itself.
Basically, if you can get storage uploads to work via an http endpoint, you can get any kind of upload mechanism working. (e.g. the FileSystem.uploadAsync() method you're probably looking for here, like #brentvatne pointed out, or fetch, or axios. I'll show a basic wiring at the end).
Server Side
Option 1: Signed URL Upload.
Basically, have a small firebase function that returns a signed url. Your app calls a cloud function like /get-signed-upload-url , which returns the url, which you then use. Check out: https://cloud.google.com/storage/docs/access-control/signed-urls for how you'd go about this.
This might work well for your use case. It can be configured just like any httpsCallable function, so it's not much work to set up, compared to option 2.
However, this doesn't work for the firebase storage / functions emulator! For this reason, I don't use this method, because I like to intensively use the emulators, and they only offer a subset of all the functionalities.
Option 2: Upload the file entirely through a function
This is a little hairier, but gives you a lot more fidelity over your uploads, and will work on an emulator! I like this too because it allows doing upload process within the endpoint execution, instead of as a side effect.
For example, you can have a photo-upload endpoint generate thumbnails, and if the endpoint 201's, then you're good! Rather than the traditional Firebase approach of having a listener to cloud storage which would generate thumbnails as a side effect, which then has all kinds of bad race conditions (checking for processing completion via exponentiational backoff? Gross!)
Here are three resources I'd recommend to go about this approach:
https://cloud.google.com/functions/docs/writing/http#multipart_data
https://github.com/firebase/firebase-js-sdk/issues/5848
https://github.com/mscdex/busboy
Basically, if you can make a Firebase cloud endpoint that accepts a File within formdata, you can have busboy parse it, and then you can do anything you want with it... like upload it to Cloud Storage!
an outline of this:
import * as functions from "firebase-functions";
import * as busboy from "busboy";
import * as os from "os";
import * as path from "path";
import * as fs from "fs";
type FieldMap = {
[fieldKey: string]: string;
};
type Upload = {
filepath: string;
mimeType: string;
};
type UploadMap = {
[fileName: string]: Upload;
};
const MAX_FILE_SIZE = 2 * 1024 * 1024; // 2MB
export const uploadPhoto = functions.https.onRequest(async (req, res) => {
verifyRequest(req); // Verify parameters, auth, etc. Better yet, use a middleware system for this like express.
// This object will accumulate all the fields, keyed by their name
const fields: FieldMap = {};
// This object will accumulate all the uploaded files, keyed by their name.
const uploads: UploadMap = {};
// This will accumulator errors during the busboy process, allowing us to end early.
const errors: string[] = [];
const tmpdir = os.tmpdir();
const fileWrites: Promise<unknown>[] = [];
function cleanup() {
Object.entries(uploads).forEach(([filename, { filepath }]) => {
console.log(`unlinking: ${filename} from ${path}`);
fs.unlinkSync(filepath);
});
}
const bb = busboy({
headers: req.headers,
limits: {
files: 1,
fields: 1,
fileSize: MAX_FILE_SIZE,
},
});
bb.on("file", (name, file, info) => {
verifyFile(name, file, info); // Verify your mimeType / filename, etc.
file.on("limit", () => {
console.log("too big of file!");
});
const { filename, mimeType } = info;
// Note: os.tmpdir() points to an in-memory file system on GCF
// Thus, any files in it must fit in the instance's memory.
console.log(`Processed file ${filename}`);
const filepath = path.join(tmpdir, filename);
uploads[filename] = {
filepath,
mimeType,
};
const writeStream = fs.createWriteStream(filepath);
file.pipe(writeStream);
// File was processed by Busboy; wait for it to be written.
// Note: GCF may not persist saved files across invocations.
// Persistent files must be kept in other locations
// (such as Cloud Storage buckets).
const promise = new Promise((resolve, reject) => {
file.on("end", () => {
writeStream.end();
});
writeStream.on("finish", resolve);
writeStream.on("error", reject);
});
fileWrites.push(promise);
});
bb.on("close", async () => {
await Promise.all(fileWrites);
// Fail if errors:
if (errors.length > 0) {
functions.logger.error("Upload failed", errors);
res.status(400).send(errors.join());
} else {
try {
const upload = Object.values(uploads)[0];
if (!upload) {
functions.logger.debug("No upload found");
res.status(400).send("No file uploaded");
return;
}
const { uploadId } = await processUpload(upload, userId);
cleanup();
res.status(201).send({
uploadId,
});
} catch (error) {
cleanup();
functions.logger.error("Error processing file", error);
res.status(500).send("Error processing file");
}
}
});
bb.end(req.rawBody);
});
Then, that processUpload function can do anything you want with the file, like upload it to cloud storage:
async function processUpload({ filepath, mimeType }: Upload, userId: string) {
const fileId = uuidv4();
const bucket = admin.storage().bucket();
await bucket.upload(filepath, {
destination: `users/${userId}/${fileId}`,
{
contentType: mimeType,
},
});
return { fileId };
}
Mobile Side
Then, on the mobile side, you can interact with it like this:
async function uploadFile(uri: string) {
function getFunctionsUrl(): string {
if (USE_EMULATOR) {
const origin =
Constants?.manifest?.debuggerHost?.split(":").shift() || "localhost";
const functionsPort = 5001;
const functionsHost = `http://${origin}:${functionsPort}/{PROJECT_NAME}/${PROJECT_LOCATION}`;
return functionsHost;
} else {
return `https://{PROJECT_LOCATION}-{PROJECT_NAME}.cloudfunctions.net`;
}
}
// The url of your endpoint. Make this as smart as you want.
const url = `${getFunctionsUrl()}/uploadPhoto`;
await FileSystem.uploadAsync(uploadUrl, uri, {
httpMethod: "POST",
uploadType: FileSystem.FileSystemUploadType.MULTIPART,
fieldName: "file", // Important! make sure this matches however you want bussboy to validate the "name" field on file.
mimeType,
headers: {
"content-type": "multipart/form-data",
Authorization: `${idToken}`,
},
});
});
TLDR
Wrap Cloud Storage in your own endpoint, treat it like a normal http upload, everything plays nice.
Question
How do I play a local audio file inside Tone.Offline so that when it finishes and return it's promised buffer it will contain the played audio?
Code
Tone.Offline(({ transport }) => {
const p = new Tone.Player(src, () => {
transport.start();
}).toDestination();
transport.schedule((time) => {
p.start(time + 5).stop(time + 9); //this never happen!
});
}, 10).then((buffer) => { // how do I push parts of my audio file onto the buffer?
const wav = toWav(buffer);
saveAs(new Blob([wav], { type: "audio/wav" }), "./blibli.wav");
});
Issue
I've noticed the callback part inside transport.schedule never happen,
and so I never get to build the final buffer.
Here's the documentation for Pizzip for those unfamiliar: https://github.com/open-xml-templating/pizzip/tree/master/documentation
Also, I'm using VueJS (2), ES6, TS
I am trying to download individual files with Axios and then put them into a zip file using Pizzip.
async getFile(mat: Material): Promise<any> {
try {
const file = await axios.get(mat.url, { responseType: 'arraybuffer' })
return file.data
} catch (error) {
console.log(error)
return {}
}
}
This is giving me the ArrayBuffer correctly. Then I have the following:
async getZip() {
let zip = new pizzip()
this.materials.forEach(async mat => {
const record = await PortalService.getFile(mat)
console.log(record) // logs the ArrayBuffer
zip.file(mat.name, record)
})
let files = zip.generate({ type: 'blob' })
return saveAs(files, `${this.request.id}-files.zip`)
}
Pizzip docs said that the data argument of zip.file() can be of type ArrayBuffer, but when I download the zip it's just an empty archive. I've tried using the optional options parameter for zip.file(), but nothing has worked. Thank you in advance.
The issue is probably to do with the async for-loop. The forEach method does not support async loop functions; it will just execute each iteration without waiting for the previous to finish. So basically you're generating the zip file before any files had been added to it.
There are two ways you can handle this:
Fire all requests concurrently and wait for them all to finish. This will be faster since it is sending the requests concurrently, but be careful not to spam hundreds of requests at once!
await Promise.all(this.materials.map(async mat => {
const record = await PortalService.getFile(mat)
zip.file(mat.name, record)
}))
Fire each request sequentially, one at a time. Slower, but doesn't spam requests.
for (const mat of this.materials) {
const record = await PortalService.getFile(mat)
zip.file(mat.name, record)
}
i hope y'all will be fine,
I am a beginner in webRTC sorry if my question feels like a noob one but i was wondering that is there any proper way to close the connection among peers especially using simple-peer.js, looking forward for your awesome replies
here is my sample react code,
if (navigator.getUserMedia) {
navigator.getUserMedia({ audio: { echoCancellation: true }, video: true }, stream => {
let peer = new Peer({
initiator: this.props.isInitiator,
stream
});
this.localStream.current.srcObject = stream;
this.localStream.current.play();
peer.on('signal', (data) => {
socket.emit('offer', { data: JSON.stringify(data), conversation_id: this.props.conversation_id })
});
socket.on('offer', (data) => {
peer.signal(JSON.parse(data))
})
socket.on('DESTROY-VIDEO-CALL-SESSION', () => {
stream.getTracks().forEach(track => track.stop());
peer.removeAllListeners();
peer.destroy();
});
peer.on('stream', (streamData) => {
this.remoteStream.current.srcObject = streamData;
this.remoteStream.current.play();
});
this.setState({
endCall: () => {
socket.emit('VIDEO-CALL-ENDED', this.props.conversation_id);
}
})
}, error => {
alert('Please allow video and audio permission to make this call')
});
}
The function is peer.destroy().
Fyi, if you need to find undocumented API functions in Javascript libraries, you can do this by running your JS file in the browser and pressing F12 to bring up the debugging console. Set a breakpoint after the object that you want to investigate is instantiated:
I set a breakpoint AFTER I instantiated peer1 (line 51). Then I hovered over peer1 (line 50) to pop up a menu of all of the properties (methods and fields) belonging to this object. If you're looking for a particular function as in this case, sometimes you find it immediately in this menu, but often you have to expand the property called __proto__ :
You can see the destroy function located here. It's just a guess whether or not a function found in this manner actually does what you want it to do, but luckily in this case it does what we want.
Is it possible to preload / pre-populate a database in my React Native application and then the first time it is run, simply do a sync? I already have most, if not all of the database information before the app is distributed, it would be awesome if it just had to do a quick sync when the app is run. Any ideas how I would go about doing that?
I found this - https://pouchdb.com/2016/04/28/prebuilt-databases-with-pouchdb.html but it doesn't mention React Native
Using:
pouchdb-find: ^7.0.0
pouchdb-react-native: ^6.4.1
react: 16.3.1
react-moment: ^0.7.9
react-native: ~0.55.2
Thanks for any pointers.
Update Here is the code I'm using to try the loading of a dump file. This code exists in /screens/Home.js
The dump file is located in /client/dbfile/database.txt
var db = new PouchDB("cs1");
db.get("_local/initial_load_complete")
.catch(function(err) {
console.log("loading dumpfile");
if (err.status !== 404) {
// 404 means not found
throw err;
}
db.load("/client/dbfile/database.txt").then(function() {
return db.put({ _id: "_local/initial_load_complete" });
});
})
.then(function() {
// at this point, we are sure that
// initial replication is complete
console.log("loading is complete!");
return db.allDocs({ include_docs: true });
})
.then(
function(res) {
// display all docs for debugging purposes (empty)
console.log(res);
});
this.localDB = db;
When this runs my console displays this - showing there have been 0 rows added.
Object {
"offset": 0,
"rows": Array [],
"total_rows": 0,
}
Possible Unhandled Promise Rejection (id: 0):
Object {
"message": undefined,
"name": "unknown",
"status": 0,
}
In my project I have couple of db docs I distribute with app (translations JSON is the one good example).
So at app init I just try to read translations doc from db, if there is none - I import content from js module and store in db.
Then translations changes just being replicated from server to local db.
//transmob.js
const transMobFile = {
//content goes here
);
module.exports = transMobFile
//dbInit.js
import transMobFile from 'data/transMob';
..
PDB.getDoc('TransMob')
.then((doc)=> {
if (!doc) {
global.locales = transMobFile.localesMob; // locales
global.translations = transMobFile.langMob; // translations
return PDB.saveDoc('TransMob', transMobFile)
}
})
You can use react-native-fs to load a file from /android/app/src/main/assets. Just put the file into the assets folder and read it with RNFS.readFileAssets.
import PouchDB from 'pouchdb-core';
PouchDB
.plugin(require('pouchdb-find'))
.plugin(require('pouchdb-load'));
import RNFS from 'react-native-fs';
const localDB = new PouchDB("cs1", {adapter: 'asyncstorage'});
localDB.get("_local/initial_load_complete")
.catch(function(err) {
console.log("loading dumpfile");
if (err.status !== 404) {
// 404 means not found
throw err;
}
RNFS.readFileAssets('yourdb.txt', 'utf8')
.then((contents) => {
localDB.load(contents).then(function() {
return localDB.put({ _id: "_local/initial_load_complete" });
}).then(function() {
// at this point, we are sure that
// initial replication is complete
console.log("loading is complete!");
return localDB.allDocs({ include_docs: true }).then(
function(res) {
// display all docs for debugging purposes (empty)
console.log(res);
});
}, function(err) {
console.log(err);
});
})
})
You'll need to rebuild your project, reloading is not sufficient.
My project crashes when I attempt to load a 30MB file, so I probably will split it into a few smaller files. Check out https://github.com/pouchdb-community/pouchdb-load to see how this works if needed.
I found that the db.load() function from the pouchdb-load module requires a URL. I was pointing it to a file path on the device's filesystem. I placed my database.txt file on my server, changed it to use the url and it worked.
In my mind this isn't ideal because if they install the app and have slow wireless, it still has to pull the file from the server. It is still much faster than performing a full-on replicate when the app opens for the first time however.