React-Native-Image-Picker Auto video recording possible? - react-native

I'm a beginner at React Native.
I am trying to access a native(built-in) camera app on Android device.
I used React-Native-Image-Picker to open the camera app but I would like to record a video somehow automatically(?) I mean not using my finger.
I need codes that make it to record and stop the video.
(I don't mean to give me a code rather, please advise if it is even possible?)
Any help would be very appreciated.
Thank you!

It is possible.
Package: https://github.com/mrousavy/react-native-vision-camera
Review the API and Guide section to see how to start and stop recording programmatically.
They also show an example app that demonstrates different types of capture including video recording, ref: https://github.com/mrousavy/react-native-vision-camera/blob/28fc6a68a5744efc85b532a338e2ab1bc8fa45fe/example/src/views/CaptureButton.tsx
...
const onStoppedRecording = useCallback(() => {
isRecording.current = false;
cancelAnimation(recordingProgress);
console.log('stopped recording video!');
}, [recordingProgress]);
const stopRecording = useCallback(async () => {
try {
if (camera.current == null) throw new Error('Camera ref is null!');
console.log('calling stopRecording()...');
await camera.current.stopRecording();
console.log('called stopRecording()!');
} catch (e) {
console.error('failed to stop recording!', e);
}
}, [camera]);
const startRecording = useCallback(() => {
try {
if (camera.current == null) throw new Error('Camera ref is null!');
console.log('calling startRecording()...');
camera.current.startRecording({
flash: flash,
onRecordingError: (error) => {
console.error('Recording failed!', error);
onStoppedRecording();
},
onRecordingFinished: (video) => {
console.log(`Recording successfully finished! ${video.path}`);
onMediaCaptured(video, 'video');
onStoppedRecording();
},
});
// TODO: wait until startRecording returns to actually find out if the recording has successfully started
console.log('called startRecording()!');
isRecording.current = true;
} catch (e) {
console.error('failed to start recording!', e, 'camera');
}
}, [camera, flash, onMediaCaptured, onStoppedRecording]);
//#endregion
...

Related

React Native - Cannot use Speech To Text and Text To Speech together

I am using both react-native-voice and expo-speech libraries to transcript my voice and to convert a text to a speech. The problem is, when i end registering my voice and start a speech with expo-voice, there is no sound. It seems like react-native-voice completly mutes the audio when the voice recording is ended. The speech starts, but i must press on the mic button (activating the voice recognition) to hear it.
The only way i found to make everything work together is by stopping the voice recording after the text to speech has ended. Here is a part of the code :
const startRecognizing = async () => {
setButtonColor('#38b000');
// Starts listening for speech for a specific locale
try {
await Voice.start('en-EN');
} catch (e) {
console.error(e);
}
};
const destroyRecognizer = async () => {
//Destroys the current SpeechRecognizer instance
try {
await Voice.destroy();
} catch (e) {
console.error(e);
}
};
const stopRecognizing = async () => {
setButtonColor('#344E41');
setTimeout(() => {
Speech.speak("I did not understand, can you repeat please ?", {
language: 'en-EN',
onDone: () => {
setTimeout(
async() => {
await destroyRecognizer();
}, 1000);
},
});
}, 1000);
};
return (
<View style={styles.microphoneButtonContainer}>
<TouchableOpacity
onPressIn={startRecognizing}
onPressOut={stopRecognizing}>
<Image
style={styles.microphoneButton}
source={require('../img/microphone-icon.png')}
/>
</TouchableOpacity>
</View>
);
This solution brings a lot of edgecases so i can't work with it. None of the methods given to stop the recording solve the issue. And i did not found any help in the libraries docs. Is there a solution to this ? Thank you for your help !
Because there was no way to solve the issue with react-native-voice methods, i had the idea to go search directly in the library if i can modify the native code. For ios the code is in ios/voice/Voice.m. I found this :
- (void) teardown {
self.isTearingDown = YES;
[self.recognitionTask cancel];
self.recognitionTask = nil;
// Set back audio session category
[self resetAudioSession];
So i tried to comment out [self resetAudioSession];, i then rebuilt the packages with npx-pod-install (i use cocoapod), and it worked!
Doing this may cause edgecases, i did not fully test the methods yet, and i did not try for android.

Closing WebRTC track will not close camera device or tab camera indicator

Banging my head to the wall with this one, I can't seem to understand what is holding on the camera video stream and not closing when MediaStreamTrack.stop() called.
I have a typescript class where I handle getting the WebRTC stream track and passing it using an observable event to a functional reactjs component, the below code is the component registering to the event and using state for the stream track.
const [videoStreamTrack, setVideoStreamTrack] = useState < MediaStreamTrack > (
null
)
useEffect(() => {
return () => {
videoStreamTrack?.stop()
videoElement.current.srcObject.getVideoTracks().forEach((track) => {
track.stop()
videoElement.current.srcObject.removeTrack(track)
})
videoElement.current.srcObject = null
}
}, [])
case RoomEvents.WebcamProducerAdded:
case RoomEvents.VideoStreamReplaced: {
if (result.data?.track) {
if (result.data.track.kind === 'video') {
previewVideoStreamTrack?.stop()
setPreviewVideoStreamTrack(null)
setVideoStreamTrack(result.data.track)
}
}
break
}
In the "Room" class I use the below code to grab the stream.
const videoDevice = this.webcam.device
if (!videoDevice) {
throw new Error('no webcam devices')
}
const userMedia = await navigator.mediaDevices.getUserMedia({
video: this.environmentPlatformService.isMobile ?
true : {
deviceId: {
exact: this.webcam.device.deviceId
},
...VIDEO_CONSTRAINS[this.webcam.resolution],
},
})
const videoTrack = userMedia.getVideoTracks()[0]
this.eventSubject.next({
eventName: RoomEvents.WebcamProducerAdded,
data: {
track: videoTrack,
},
})
I am holding to this.webcam.device details using the code below.
async updateInputOutputMediaDevices(): Promise < MediaDeviceInfo[] > {
await navigator.mediaDevices.getUserMedia({
audio: true,
video: true
})
const devices = await navigator.mediaDevices.enumerateDevices()
await this.updateWebcams(devices)
await this.updateAudioInputs(devices)
await this.updateAudioOutputs(devices)
return devices
}
private async updateWebcams(devices: MediaDeviceInfo[]) {
this.webcams = new Map < string, MediaDeviceInfo > ()
for (const device of devices.filter((d) => d.kind === 'videoinput')) {
this.webcams.set(device.deviceId, device)
}
const array = Array.from(this.webcams.values())
this.eventSubject.next({
eventName: RoomEvents.CanChangeWebcam,
data: {
canChangeWebcam: array.length > 1,
mediaDevices: array,
},
})
}
Refreshing the page will close the camera and tab indicator.
useEffect(() => {
return () => {
videoStreamTrack?.stop()
videoElement.current.srcObject.getVideoTracks().forEach((track) => {
track.stop()
videoElement.current.srcObject.removeTrack(track)
})
videoElement.current.srcObject = null
}
}, [])
So here you are search and destroying video tracks. Seems right-ish; we'll see
async updateInputOutputMediaDevices(): Promise < MediaDeviceInfo[] > {
await navigator.mediaDevices.getUserMedia({
audio: true,
video: true
})
const devices = await navigator.mediaDevices.enumerateDevices()
await this.updateWebcams(devices)
await this.updateAudioInputs(devices)
await this.updateAudioOutputs(devices)
return devices
}
Above I see there's a call for audio might be where the hiccups are? Can't overly examine but maybe you're opening both and closing just video? Try doing a loop through all tracks not just video and see what's there?
#blanknamefornow answer helped me nail the issue.
We are calling getUserMedia in multiple places not only in the
“room” class handling mediasoup actions but also fore
preview/device-selection/etc and didn’t really ever closed the
tracks retrieved.
Sometimes, those tracks are held into useState
variables and when component unmounted if you try to access the
variables they are already nulled by reactjs. The workaround is
since the HTML elements are still referenced stop the track when
needed. I believe this was the missing ingredient when trying to
figure it out.

React Native Expo Audio | Play live stream from latest position

I'm writing an audio player, using Expo Audio, for an app I'm making for an online radio.
The audio comes from an online live stream and, I've successfully added the player and all the things related to it; however, the one issue I'm having is that if I pause the audio when I resume playing it the audio continues from when I paused it rather than from the current position and I need to pause it and play it again to get it to update to what's currently being played.
I play it with playAsync() and I've tried pausing with pauseAsync(), stopAsync(), setStatusAsync({ shouldPlay: false, positionMillis: 0 });
Any tips on how I can get it to work the way it should?
Here's the code I have for the audio player, it's a class from which then I create an instance of to be able to manage it from different places in the app:
class audioPlayer {
static instance = null;
static createInstance() {
var object = new audioPlayer();
return object;
}
_radioStream;
/**
* #returns {audioPlayer}
*/
static getInstance() {
if (audioPlayer.instance == null) {
audioPlayer.instance = audioPlayer.createInstance();
}
return audioPlayer.instance;
}
// Call this first to create a new audio element
createAudio() {
this._radioStream = new Audio.Sound();
};
async loadAudioAsync() {
try {
await this._radioStream.loadAsync(
{ uri: "radio straem"},
);
store.dispatch(setLiveState(true));
this.toggleAudio(); // Autoplay at start
return true;
} catch (error) {
if (error.code === "E_LOAD_ERROR") {
// In the case of an error we try to load again
setTimeout(this.loadAudioAsync, 10000);
throw new Error(error.code);
} else {
throw new Error(error);
};
};
};
async unloadAudioAsync() {
await this._radioStream.unloadAsync();
};
async getStatusAsync() {
return await this._radioStream.getStatusAsync();
};
async toggleAudio() {
// We're gonna play or pause depending on the status
let { isLoaded, isPlaying } = await this._radioStream.getStatusAsync();
// If the user presses the audio and the stream connection has been lost or something
// we try to load it again
if (!isLoaded) {
let res = await this.loadAudioAsync(); // Try to loadAudio again
if (res) this.toggleAudio(); // Retrigger the toggle to start playing
}
if (isLoaded && !isPlaying) {
store.dispatch(setPlayingStatus(true));
await this._radioStream.playAsync();
} else if (isLoaded && isPlaying) {
store.dispatch(setPlayingStatus(false));
await this._radioStream.setStatusAsync({ shouldPlay: false, positionMillis: 0 });
};
};
};
I just had the same exact problem (for my internet radio https://notylus.fr).
It's seems that I found a solution : instead of using
playbackInstance.pauseAsync()
I now use
playbackInstance.stopAsync()
AND for the play part, I add
await playbackInstance.playAsync() //play stream
playbackInstance.setPositionAsync(0) //ensure that you're at position 0
Last
Regards,

Deep links in react-native-firebase notifications

I am using react-native-firebase with messaging to deliver notifications to my app with cloud functions, with admin.messaging().send(message), very similar to here: https://medium.com/the-modern-development-stack/react-native-push-notifications-with-firebase-cloud-functions-74b832d45386 .
I receive notifications when the app is in the background. Right now I am sending a text in the body of the notification, like 'a new location has been added to the map'. I want to be able to add some sort of deep link, so that when I swipe View on the notification (on iOS for example), it will take me to a specific screen inside the app. How do I pass data from the notification to the app?
I am using react-native-navigation in the app. I can only find code about deep links from inside the app (https://wix.github.io/react-native-navigation/#/deep-links?id=deep-links).
My solution was to use add what information I need in the data object of the notification message object:
in functions/index.js:
let message = {
notification: {
body: `new notification `
},
token: pushToken,
data: {
type: 'NEW_TRAINING',
title: locationTitle
}
};
and process as follows in the app for navigation:
this.notificationOpenedListener =
firebase.notifications().onNotificationOpened((notificationOpen: NotificationOpen) => {
if (notification.data.type === 'NEW_TRAINING') {
this.props.navigator.push({
screen: 'newtrainingscreen',
title: notification.data.title,
animated: true
});
}
I think you are fine with the "how firebase notification work"... cause of this, here is only an description of the Logic how you can Deeplinking into your App.
If you send a notification, add a data-field. Let's say your app has a Tab-Navigator and the sections "News","Service" and "Review".
In your Push-Notification - Datafield (let's name it "jumpToScreen" you define your value:
jumpToScreen = Service
I assume you still have the Handling to recieve Notifications from Firebase implemented.
So create an /lib/MessageHandler.js Class and put your business-logic inside.
import firebase from 'react-native-firebase';
/*
* Get a string from Firebase-Messages and return the Screen to jump to
*/
const getJumpPoint = (pointer) => {
switch (pointer) {
case 'News':
return 'NAV_NewsList'; // <= this are the names of your Screens
case 'Service':
return 'NAV_ServiceList';
case 'Review':
return 'NAV_ReviewDetail';
default: return false;
}
};
const MessageHandler = {
/**
* initPushNotification initialize Firebase Messaging
* #return fcmToken String
*/
initPushNotification: async () => {
try {
const notificationPermission = await firebase.messaging().hasPermission();
MessageHandler.setNotificationChannels();
if (notificationPermission) {
try {
return await MessageHandler.getNotificationToken();
} catch (error) {
console.log(`Error: failed to get Notification-Token \n ${error}`);
}
}
} catch (error) {
console.log(`Error while checking Notification-Permission\n ${error}`);
}
return false;
},
clearBadges: () => {
firebase.notifications().setBadge(0);
},
getNotificationToken: () => firebase.messaging().getToken(),
setNotificationChannels() {
try {
/* Notification-Channels is a must-have for Android >= 8 */
const channel = new firebase.notifications.Android.Channel(
'app-infos',
'App Infos',
firebase.notifications.Android.Importance.Max,
).setDescription('General Information');
firebase.notifications().android.createChannel(channel);
} catch (error) {
console.log('Error while creating Push_Notification-Channel');
}
},
requestPermission: () => {
try {
firebase.messaging().requestPermission();
firebase.analytics().logEvent('pushNotification_permission', { decision: 'denied' });
} catch (error) {
// User has rejected permissions
firebase.analytics().logEvent('pushNotification_permission', { decision: 'allowed' });
}
},
foregroundNotificationListener: (navigation) => {
// In-App Messages if App in Foreground
firebase.notifications().onNotification((notification) => {
MessageHandler.setNotificationChannels();
navigation.navigate(getJumpPoint(notification.data.screen));
});
},
backgroundNotificationListener: (navigation) => {
// In-App Messages if App in Background
firebase.notifications().onNotificationOpened((notificationOpen) => {
const { notification } = notificationOpen;
notification.android.setChannelId('app-infos');
if (notification.data.screen !== undefined) {
navigation.navigate(getJumpPoint(notification.data.screen));
}
});
},
appInitNotificationListener: () => {
// In-App Messages if App in Background
firebase.notifications().onNotificationOpend((notification) => {
notification.android.setChannelId('app-infos');
console.log('App-Init: Da kommt ne Message rein', notification);
firebase.notifications().displayNotification(notification);
});
},
};
export default MessageHandler;
In your index.js you can connect it like this:
import MessageHandler from './lib/MessageHandler';
export default class App extends Component {
state = {
loading: null,
connection: null,
settings: null,
};
async componentDidMount() {
const { navigation } = this.props;
await MessageHandler.initPushNotification();
this.notificationForegroundListener = MessageHandler.foregroundNotificationListener(navigation);
this.notificationBackgroundListener = MessageHandler.backgroundNotificationListener(navigation);
this.setState({ loading: false, data });
}
componentWillUnmount() {
this.notificationForegroundListener();
this.notificationBackgroundListener();
}
async componentDidMount() {
MessageHandler.requestPermission();
AppState.addEventListener('change', this.handleAppStateChange);
MessageHandler.clearBadges();
}
componentWillUnmount() {
AppState.removeEventListener('change', this.handleAppStateChange);
}
handleAppStateChange = (nextAppState) => {
if (nextAppState.match(/inactive|background/)) {
MessageHandler.clearBadges();
}
....
I hope this give you an Idea how to implement it for your needs.
I think you don't need to use deep links nor dynamic links but just use Firebase/Notifications properly. If I were you I would add the following logic in the componentDidMount method of your parent container:
async componentDidMount() {
// 1. Check notification permission
const notificationsEnabled = await firebase.messaging().hasPermission();
if (!notificationsEnabled) {
try {
await firebase.messaging().requestPermission(); // Request notification permission
// At this point the user has authorized the notifications
} catch (error) {
// The user has NOT authorized the notifications
}
}
// 2. Get the registration token for firebase notifications
const fcmToken = await firebase.messaging().getToken();
// Save the token
// 3. Listen for notifications. To do that, react-native-firebase offer you some methods:
firebase.messaging().onMessage(message => { /* */ })
firebase.notifications().onNotificationDisplayed(notification => { /* */ })
firebase.messaging().onNotification(notification => { /* */ })
firebase.messaging().onNotificationOpened(notification => {
/* For instance, you could use it and do the NAVIGATION at this point
this.props.navigation.navigate('SomeScreen');
// Note that you can send whatever you want in the *notification* object, so you can add to the notification the route name of the screen you want to navigate to.
*/
})
}
You can find the documentation here: https://rnfirebase.io/docs/v4.3.x/notifications/receiving-notifications

React Native, store information before app exits

Is this at all possible? I'm currently using react-native-track-player to stream audio files and I would love to be able to store the last position when my users exit the app and resume when they re-open (e.g. similar to how Spotify works)
Right now I'm tracking this info via a simple interval:
this.keepTime = setInterval(async () => {
const state = await TrackPlayer.getState()
if (state == TrackPlayer.STATE_PLAYING) {
const ID = await TrackPlayer.getCurrentTrack()
const position = await TrackPlayer.getPosition()
await AsyncStorage.setItem(ID, String(position))
}
}, 10000)
Problem is, I need to clear the interval when my app moves to the background or else it will crash. I would also much rather only need to call this code once as opposed to periodically if that is possible.
I know I could use headless JS on android but the app is cross platform, so my iOS user experience would be lesser.
Any suggestions?
I think you can use componentWillUnmount() function for this.
You could add a listener to get the App State and then log the position when it goes to background.
class App extends React.Component {
state = {
appState: AppState.currentState
}
componentDidMount() {
AppState.addEventListener('change', this.handleAppStateChange);
}
componentWillUnmount() {
AppState.removeEventListener('change', this.handleAppStateChange);
this.saveTrackPosition();
}
handleAppStateChange = (nextAppState) => {
if (nextAppState.match(/inactive|background/) && this.state.appState === 'active') {
this.saveTrackPosition();
}
this.setState({appState: nextAppState});
}
saveTrackPosition = () => {
if (state == TrackPlayer.STATE_PLAYING) {
const ID = await TrackPlayer.getCurrentTrack()
const position = await TrackPlayer.getPosition()
await AsyncStorage.setItem(ID, String(position))
}
}
}