How to play a microphone stream in React Native? - react-native

The goal is to make a simple walkie-talkie in React Native through Wifi Direct.
I was able to capture a microphone stream on the first phone (192.168.49.1) and send it to the second phone (192.168.49.200) by using the library react-native-udp.
How do I play the transferred audio on the other phone?
Tried different audio apps, but they don't support raw audio buffer AFAIK?
Should I first save it to a file and then stream this local file?
Any ideas?
import React, {Component} from 'react';
import {ScrollView, StyleSheet, Text, View} from 'react-native';
import Recording from "react-native-recording";
import dgram from 'react-native-udp';
class App extends Component {
constructor(props) {
super(props);
}
componentDidMount() {
Recording.init({
bufferSize: 4096,
sampleRate: 44100,
bitsPerChannel: 16,
channelsPerFrame: 1,
});
Recording.start();
let a = dgram.createSocket('udp4');
let aPort = 23456;
a.bind(aPort, '192.168.49.1', function(err) {
if (err) throw err;
});
a.once('listening', function() {
const listener = Recording.addRecordingEventListener((data) => {
a.send(msg, undefined, undefined, aPort, '192.168.49.200', function(err) {
if (err) throw err;
});
}
}
}
render() {
return (
<View style={styles.container}>
</View>
);
}
}
const styles = StyleSheet.create({
container: {
flex: 1,
justifyContent: 'center',
alignItems: 'center',
backgroundColor: '#F5FCFF',
}
}
export default App;

Did you try this module? This might suit your need:
https://github.com/hqwhuang/react-native-AudioBufferPlayer
Android: https://github.com/hqwhuang/react-native-AudioBufferPlayer/blob/d6fb7cf13850055d38de27fdedaa72e2d1507092/android/src/main/java/com/reactlibrary/RNReactNativeAudioBufferPlayerModule.java#L43
But I am not sure about iOS.
Or you can write the native module on your own and share using AudioTrack class in android and AVAudioPCMBuffers in iOS.
Some helpful references:
https://stackoverflow.com/a/1977221/7978635
https://stackoverflow.com/a/53164714/7978635

Related

Getting a spinning plank screen and Error: Camera is not ready yet. Wait for 'onCameraReady' callback using expo Camera component

I'm new to web development and I'm trying to build an image recognition app using expo for testing. My code for the camera is below. On screen load, I get a black screen (not the camera) with my "capture" button. When I click on capture, I get the error:
Unhandled promise rejection: Error: Camera is not ready yet. Wait for 'onCameraReady' callback.
My code is below
import { Dimensions, Alert, StyleSheet, ActivityIndicator } from 'react-native';
// import { RNCamera } from 'react-native-camera';
import CaptureButton from './CaptureButton.js'
import { Camera } from 'expo-camera';
export default class AppCamera extends React.Component {
constructor(props){
super(props);
this.state = {
identifiedAs: '',
loading: false
}
}
takePicture = async function(){
if (this.camera) {
// Pause the camera's preview
this.camera.pausePreview();
// Set the activity indicator
this.setState((previousState, props) => ({
loading: true
}));
// Set options
const options = {
base64: true
};
// Get the base64 version of the image
const data = await this.camera.takePictureAsync(options)
// Get the identified image
this.identifyImage(data.base64);
}
}
identifyImage(imageData){
// Initialise Clarifai api
const Clarifai = require('clarifai');
const app = new Clarifai.App({
apiKey: '8d5ecc284af54894a38ba9bd7e95681b'
});
// Identify the image
app.models.predict(Clarifai.GENERAL_MODEL, {base64: imageData})
.then((response) => this.displayAnswer(response.outputs[0].data.concepts[0].name)
.catch((err) => alert(err))
);
}
displayAnswer(identifiedImage){
// Dismiss the acitivty indicator
this.setState((prevState, props) => ({
identifiedAs:identifiedImage,
loading:false
}));
// Show an alert with the answer on
Alert.alert(
this.state.identifiedAs,
'',
{ cancelable: false }
)
// Resume the preview
this.camera.resumePreview();
}
render () {
const styles = StyleSheet.create({
preview: {
flex: 1,
justifyContent: 'flex-end',
alignItems: 'center',
height: Dimensions.get('window').height,
width: Dimensions.get('window').width,
},
loadingIndicator: {
flex: 1,
alignItems: 'center',
justifyContent: 'center',
}
});
return (
<Camera ref={ref => {this.camera = ref;}}style={styles.preview}>
<ActivityIndicator size="large" style={styles.loadingIndicator} color="#fff" animating={this.state.loading}/>
<CaptureButton buttonDisabled={this.state.loading} onClick={this.takePicture.bind(this)}/>
</Camera>
)
}
}```
Could someone kindly point me in the right direction to fix this error?
https://docs.expo.dev/versions/latest/sdk/camera/#takepictureasyncoptions
Note: Make sure to wait for the onCameraReady callback before calling this method.
So, you might resolve if you add onCameraReady props to Camera component like this document.
I'm facing issue like this, and it is not resolved now... I hope my advice works well.

In react-native-webrtc in <RTCView> only video is coming, audio is not working

I am working with react-native-webrtc but I am facing one problem I am able to display video captured by the camera but the audio is not working or we can say I am getting video frames of display and not able to listen any audio. I have given microphone access too and in getusermedia audio and video both parameters are true.
Here is my code:
import React, { Component } from 'react';
import { StyleSheet, View, Button, Text } from 'react-native';
import * as mediasoupClient from 'mediasoup-client';
import {
RTCPeerConnection,
RTCIceCandidate,
RTCSessionDescription,
RTCView,
MediaStream,
MediaStreamTrack,
mediaDevices,
registerGlobals,
} from 'react-native-webrtc';
import io from 'socket.io-client/dist/socket.io';
registerGlobals();
class WebRtcScreen extends Component {
state = {
MediaStreamx: new MediaStream(),
};
componentDidMount() {
navigator.mediaDevices
.getUserMedia({ video: true, audio: true })
.then(this.handleVideo)
.catch(this.videoError);
}
handleVideo = (stream) => {
this.setState({ MediaStreamx: stream });
};
videoError = (err) => {
console.log(err.name);
};
render() {
return (
<RTCView
key={1}
zOrder={2}
objectFit="cover"
style={{ ...styles.rtcView }}
streamURL={this.state.MediaStreamx.toURL()}
/>
);
}
}
const styles = StyleSheet.create({
rtcView: {
width: 100, //dimensions.width,
height: 200, //dimensions.height / 2,
backgroundColor: 'black',
},
});
export { WebRtcScreen };
Why is the audio not working here?
The issue here is that react-native-webrtc plays audio in earphone. You need to use
https://github.com/react-native-webrtc/react-native-incall-manager
to play audio in phone speaker.
Hope this helps.

Expo Push Notifications - when app is in foreground it crashes iOS

I have an Expo app, and I'm trying to handle push notifications sent while the app is in the foreground. It works fine in Android, but iOS it's crashing the app as it's received.
I have a push notification being sent from a Rails server:
params = ({
to: token.expo_push_token,
title: user.first_name,
sound: "default",
body: msg.body,
})
puts params
puts params.class
x = Net::HTTP.post_form(URI.parse('https://exp.host/--/api/v2/push/send'), params)
puts x.body
I can see in the server it sends:
Hash
app[worker.1]: {"data":{"id":"9058abf3-7352-4181-a69d-0b5fc8a8525c","status":"ok"}}
4 TID-godk4ew98 ExpoPushWorker JID-51b823f8feeaf42c313e392e INFO: done: 2.005 sec
And if the app is closed, the push notification appears on the lock screen. If the app is open in the foreground, nothing happens.
I want to listen for notifications when the app is open, and I have this in App.js:
import React from 'react';
import { StyleSheet, Text, View } from 'react-native';
import MessagesScreenRouter from './app/components/Router/MessagesScreenRouter';
import Sentry from 'sentry-expo';
import reducers from './app/reducers';
import { Provider } from 'react-redux';
import { createStore } from 'redux';
import { Notifications } from 'expo';
export default class App extends React.Component {
constructor(props) {
super(props)
this.state = {
notification: {},
}
this._handleNotification = this._handleNotification.bind(this)
}
_handleNotification = (notification) => {
console.log(notification)
this.setState({notification: notification});
};
componentWillUnmount(){
this._notificationSubscription && this._notificationSubscription.remove()
}
componentDidMount() {
this.notificationSubscription = Notifications.addListener(
(notification) => this._handleNotification(notification),
);
}
render() {
return (
<View style={{flex:1}}>
<StatusBar hidden={true} />
<View style={{height: 50, justifyContent: 'center', alignItems: 'center'}}>
<Text>Origin: {this.state.notification.origin}</Text>
<Text>Data: {JSON.stringify(this.state.notification)}</Text>
</View>
<Provider store={createStore(reducers)}>
<MessagesScreenRouter/>
</Provider>
</View>
);
}
}
I've tried many suggestions from tutorials all day, but I can not get this to work. What am I missing here?
You can't test it on simulator as expo docs states
Note: iOS and Android simulators cannot receive push notifications. To test them out you will need to use a real-life device. Additionally, when calling Permissions.askAsync on the simulator, it will resolve immediately with "undetermined" as the status, regardless of whether you choose to allow or not.
Just run exp publish and test it on expo client. Also you have to call for permission using Permissions.askAsync in the first place.
Doc's sample work like a charm, check it out: https://docs.expo.io/versions/v28.0.0/guides/push-notifications#__next
Expo has likely been updated in that aspect since then, and now it might even be using the command you have mentioned in the comment (exp start -m tunnel). As foreground notifications are still not available on iOS so far (which might have even caused your issue in the first place), this answer is rather for people looking to implement push notifications than fixing the issue above.
I have created a file downloader and previewer that shows both internal and external notifications on both OSes without running into such issues. The code is available on GitHub and an elaboration is given in this SO answer.
The most relevant code for this post is in regard to the use of local notifications from Expo while the app is in background, and showing them in foreground using toasts from the react-native-toast package. This functionality may be replaceable by Expo Notifications once this feature gets implemented.
For completeness, here is the code for the project:
App.js
import React, { Component } from 'react';
import { View, ScrollView, StyleSheet, Button, Alert, Platform, Text, TouchableWithoutFeedback } from 'react-native';
import { FileSystem, Constants, Notifications, Permissions } from 'expo';
import Toast, {DURATION} from 'react-native-easy-toast';
async function getiOSNotificationPermission() {
const { status } = await Permissions.getAsync(
Permissions.NOTIFICATIONS
);
if (status !== 'granted') {
await Permissions.askAsync(Permissions.NOTIFICATIONS);
}
}
export default class App extends Component {
constructor(props) {
super(props);
// this.toast = null;
this.listenForNotifications = this.listenForNotifications.bind(this);
// this.openFile = this.openFile.bind(this);
this.state = {
filePreviewText: ''
}
}
_handleButtonPress = () => {
let fileName = 'document.txt';
let fileUri = FileSystem.documentDirectory + fileName;
FileSystem.downloadAsync(
"https://raw.githubusercontent.com/expo/expo/master/README.md",
fileUri
).then(({ uri }) => {
console.log('Finished downloading to ', uri);
const localnotification = {
title: 'Download has finished',
body: fileName + " has been downloaded. Tap to open file.",
android: {
sound: true,
},
ios: {
sound: true,
},
data: {
fileUri: uri
},
};
localnotification.data.title = localnotification.title;
localnotification.data.body = localnotification.body;
let sendAfterFiveSeconds = Date.now();
sendAfterFiveSeconds += 3000;
const schedulingOptions = { time: sendAfterFiveSeconds };
Notifications.scheduleLocalNotificationAsync(
localnotification,
schedulingOptions
);
})
.catch(error => {
console.error(error);
Alert.alert(error);
});
};
listenForNotifications = () => {
const _this = this;
Notifications.addListener(notification => {
if (notification.origin === 'received') {
// We could also make our own design for the toast
// _this.refs.toast.show(<View><Text>hello world!</Text></View>);
const toastDOM =
<TouchableWithoutFeedback
onPress={() => {this.openFile(notification.data.fileUri)}}
style={{padding: '10', backgroundColor: 'green'}}>
<Text style={styles.toastText}>{notification.data.body}</Text>
</TouchableWithoutFeedback>;
_this.toast.show(toastDOM, DURATION.FOREVER);
} else if (notification.origin === 'selected') {
this.openFile(notification.data.fileUri);
}
// Expo.Notifications.setBadgeNumberAsync(number);
// Notifications.setBadgeNumberAsync(10);
// Notifications.presentLocalNotificationAsync(notification);
// Alert.alert(notification.title, notification.body);
});
};
componentWillMount() {
getiOSNotificationPermission();
this.listenForNotifications();
}
componentDidMount() {
// let asset = Asset.fromModule(md);
// Toast.show('Hello World');
}
openFile = (fileUri) => {
this.toast.close(40);
console.log('Opening file ' + fileUri);
FileSystem.readAsStringAsync(fileUri)
.then((fileContents) => {
// Get file contents in binary and convert to text
// let fileTextContent = parseInt(fileContents, 2);
this.setState({filePreviewText: fileContents});
});
}
render() {
return (
<View style={styles.container}>
<View style={styles.buttonsContainer}>
<Button style={styles.button}
title={"Download text file"}
onPress={this._handleButtonPress}
/>
<Button style={styles.button}
title={"Clear File Preview"}
onPress={() => {this.setState({filePreviewText: ""})}}
/>
</View>
<ScrollView style={styles.filePreview}>
<Text>{this.state.filePreviewText}</Text>
</ScrollView>
<Toast ref={ (ref) => this.toast=ref }/>
</View>
);
// <Toast
// ref={ (ref) => this.toast=ref }
// style={{backgroundColor:'green'}}
// textStyle={{color:'white'}}
// position={'bottom'}
// positionValue={100}
// opacity={0.8}
// />
}
}
const styles = StyleSheet.create({
container: {
flex: 1,
alignItems: 'center',
justifyContent: 'center',
paddingTop: Constants.statusBarHeight,
backgroundColor: '#ecf0f1',
},
buttonsContainer: {
flexDirection: 'row',
},
button: {
flex: 1
},
filePreview: {
flex: 1,
padding: 10,
},
toastText: {
color: 'white',
padding: 5,
justifyContent: 'flex-start',
},
});
package.json: Currently using forked repo, switch back when feature becomes available.
{
"name": "local-notification-with-ios",
"version": "0.0.0",
"description": "No description",
"author": null,
"private": true,
"main": "node_modules/expo/AppEntry.js",
"dependencies": {
"expo": "^32.0.0",
"react": "16.5.0",
"react-native": "https://github.com/expo/react-native/archive/sdk-32.0.0.tar.gz",
"prop-types": "^15.5.7",
"react-native-easy-toast": "git+https://github.com/SiavasFiroozbakht/react-native-easy-toast.git#6eed52f4d64c796cb49bdafcd7b3986cf5975d62"
}
}

React native Expo TypeError: Network request failed

I'm trying to fetch data from local API that is generated by ASP.NET Web API application. This local API works fine when it is tested by Postman program but it doesn't work with my react native project. I searched for this error and I found that I should replace "localhost" by my machine IPv4 address, I did so but I still get the same error. I run my project using Expo on my mobile that has Android version 7.0 NRD90M
Here's my code:-
import React, { Component } from 'react';
import { StyleSheet, Text, View, Image} from 'react-native';
import { Icon, Button, Container, Header, Content, Left} from 'native-base';
const EXAMPLE_ENDPOINT = "http://192.168.1.5:57975/api/values/5";
class TestAPI extends Component {
constructor(props) {
super(props)
this.state = {
value: null
}
}
componentDidMount() {
this._fetchExampleAsync();
}
_fetchExampleAsync = async () => {
try {
let response = await fetch(EXAMPLE_ENDPOINT);
let result = await response.json();
this.setState({value});
} catch(e) {
this.setState({value: e});
}
};
render() {
console.log("Value: ", this.state.value);
return (
<Container>
<Content contentContainerStyle={{
flex: 1,
alignItems: 'center',
justifyContent: 'center',
}}>
<Text>Test fetch API</Text>
<Text>values: {JSON.stringify(this.state.value)}</Text>
</Content>
</Container>
);
}
}
export default TestAPI;
I had the same issue and fixed it with type: 'multipart/form-data':
var media = {uri: photo.uri ,type: 'multipart/form-data', name: filename, };

implementing redux in react native

I'm trying to implement redux to show balance in multiple screens as I update balance in single screen it should reflect in all other screens/components.
I'm pretty new to redux. As you know with complexity around redux, its even making difficult to implement it.
I followed some examples in GitHub and youtube and started implementing it .
under Actions folder I have. following two files
counteractions.js
import * as types from './actionTypes.js';
//ActionCreator methods
export function updateBalance(balanceInfo) {
return {
type: types.LEDGER_BALANCE,
payLoad: { balanceInfo }
}
}
Under Reducers folder.I have this file
balance.js
import * as types from '../actions/actionTypes.js';
const initialState = {
balance: 0
}
// reducer functions .. accepts current/initial state , actions and returns new state
const balanceReducer=(state,action)=>
{
switch (action.type) {
case types.LEDGER_BALANCE:
return {
balance: action.payload.balanceInfo
}
break;
default:
break;
}
}
export default balanceReducer;
in ConfigureStore.js
import {createStore} from 'redux';
import rootReducer from './reducers/index.js';
import balanceReducer from './reducers/balance.js';
const initailState = {
balance: 0,
}
export const store=createStore(balanceReducer,balanceReducer);
App.js
/**
* Sample React Native App
* https://github.com/facebook/react-native
* #flow
*/
import React, { Component } from 'react';
import { Provider } from 'react-redux';
//Provider - makes redux store available to connect() class in component hierarchy below
import { applyMiddleware, createStore, compose, combineReducers } from "redux";
import thunkMiddleware from 'redux-thunk';
import createLogger from 'redux-logger';
import rootReducer from './reducers/index.js';
//import store from './configureStore.js';
import {
Platform,
StyleSheet,
Text,
View,
TouchableOpacity,
TextInput
} from 'react-native';
import ReduxDemo from "./reduxDemo.js";
import { store, reducer } from './balanceDemo.js';
const instructions = Platform.select({
ios: 'Press Cmd+R to reload,\n' +
'Cmd+D or shake for dev menu',
android: 'Double tap R on your keyboard to reload,\n' +
'Shake or press menu button for dev menu',
});
export default class App extends Component<{}> {
constructor(props) {
super(props);
this.state = {
balancelocal: '',
}
}
_updateLedger = () => {
// store.dispatch({ type: 'BALANCE', payLoad: '500' });
store.dispatch({ type: 'BALANCE', payLoad: 'Your balance is 8000 MUR' });
}
render() {
store.subscribe(() => {
this.setState({
balancelocal: store.getState(),
})
//this.balanceInfo = store.getState().balance;
// alert(this.state.balancelocal);
});
return (
<View style={styles.container}>
<TouchableOpacity onPress={this._updateLedger}>
<Text>Update balance</Text>
</TouchableOpacity>
<TextInput style={{height:100,width:400}} value={this.state.balancelocal}/>
</View>
);
}
}
The styling for it
const styles = StyleSheet.create({
container: {
flex: 1,
justifyContent: 'center',
alignItems: 'center',
backgroundColor: '#F5FCFF',
},
welcome: {
fontSize: 20,
textAlign: 'center',
margin: 10,
},
instructions: {
textAlign: 'center',
color: '#333333',
marginBottom: 5,
},
});
I'm yet to complete configure store file. and. I'm wondering, where I have to subscribe and dispatch actions ...
I want to update balance with button click from app.js
I have. to update balance in another page automatically..
Please guide me to understand and implement redux .Please suggest better folder structure and better way to implement redux.
https://redux.js.org/basics/exampletodolist
The above link has a basic setting up of react with redux. It is almost similar for React Native too.
You need to wrap your App component inside Provider imported from 'react-redux', and then give that to AppRegistry. Also you do not seem to have imported any actions and haven't used the connect function either. Like the comment above, it is better for you go through a video guide on basics of redux. It'll help you understand all the complexity, and once you understand, nothing is as easy as redux. All the best.