I am new coding in Javascript.
I am creating a WebRTC connection between my iPhone and my browser.
The connection works but my code only send one candidate and I don't know if I am doing anything wrong. I would appreciate any comment or support.
Thanks
const createPeerConnection = (signaling) => {
const peerConnection = new RTCPeerConnection({
iceServers: [],
});
const offerOptions = {
offerToReceiveVideo: true, offerToReceiveAudio: true
};
peerConnection.createOffer(offerOptions);
createAndSendOffer(signaling, peerConnection);
peerConnection.onicecandidate = (iceEvent) => {
if (iceEvent && iceEvent.candidate) {
signaling.send(JSON.stringify({
type: MESSAGE_TYPE.IceCandidate,
payload: iceEvent.candidate,
}));
}
};
peerConnection.onconnectionstatechange = (state ) => {
console.log(peerConnection.connectionState);
};
return peerConnection;
};
const createAndSendOffer = async (signaling, peerConnection) => {
const offer = await peerConnection.createOffer();
await peerConnection.setLocalDescription(offer);
signaling.send(JSON.stringify({ type: MESSAGE_TYPE.SessionDescription, payload: offer }));
};
Related
The application is a Zoom-like program where users are connected P2P using PeerJS calling feature.
This is the PeerJS object set up:
var peer = new Peer(undefined, {
config: {
'iceServers': [
{
url: 'stun:relay.metered.ca:80'
},
{
url: 'turn:relay.metered.ca:80',
username: '*********',
credential: '**********',
},
{
url: 'turn:relay.metered.ca:443',
username: '*********',
credential: '*********'
}
]},
host: '/',
port: '3001'
})
Playing the current user's stream, and listening for other peers' calls:
const myVideo = document.createElement('video')
myVideo.muted = true
const peers = {}
navigator.mediaDevices.getUserMedia({
video: true,
audio: true
}).then(stream => {
addVideoStream(myVideo, stream)
// listen for external calls on peer server
peer.on('call', call => {
call.answer(stream)
const video = document.createElement('video')
call.on('stream', userVideoStream => {
addVideoStream(video, userVideoStream)
})
})
socket.on('user-connected', userId => {
const call = peer.call(userId, stream)
const video = document.createElement('video')
call.on('stream', function(videoStream) {
addVideoStream(video, videoStream)
})
call.on('close', () => {
video.remove()
})
peers[userId] = call
})
})
peer.on('open', id => {
socket.emit('join-room', ROOM_ID, id)
})
function addVideoStream(video, stream) {
video.srcObject = stream
// once the video's ready play it
video.addEventListener('loadedmetadata', () => {
video.play()
})
videoGrid.append(video)
}
The server file:
const express = require('express')
const app = express()
const server = require('http').Server(app)
const io = require('socket.io')(server)
const { v4: uuidV4 } = require('uuid')
app.set('view engine', 'ejs')
app.use(express.static('public'))
app.get('/', (req, res) => {
res.redirect(`/${uuidV4()}`)
})
app.get('/:room', (req, res) => {
res.render('room', { roomId: req.params.room })
})
io.on('connection', socket => {
socket.on('join-room', (roomId, userId) => {
socket.join(roomId)
socket.to(roomId).emit('user-connected', userId)
socket.on('disconnect', () => {
socket.to(roomId).emit('user-disconnected', userId)
})
})
})
server.listen(3000)
I connect the app successfully, and I can make calls when I open the the same room's URL in another browser window (Chrome), but the call.on('stream') does not execute. I tried to see if there is an error using peer.on(error), and console.log() in the stream event's callback function, but there were no errors. I don't know what I'm missing here! Any help would be appreciated. Thank you.
I am using simple-peer and it really worked well on server before I added APIs for my project. I already did secure with https in the beginning and the only thing that has changed is releasing the server with APIs... Here is my code and now I only can check console.log(1), (3) for initiator peer and console.log(2), (8) for requesting peer. On requestinng peer internet tab, "Uncaught ReferenceError: process is not defined" this error occurs and I don't know why this error occurs on client side. Also it worked well on both of Chrome and Edge but now I can't even get my own stream on Chrome.
const myVideo = useRef();
const userVideo = useRef();
const connectionRef = useRef();
const roomName = "123";
let userStream = null;
let creator = false;
useEffect(() => {
const socket = io("https://www.jg-jg.shop");
socket.emit("joinRoom", roomName);
socket.on("created", () => {
creator = true;
navigator.mediaDevices
.getUserMedia({ video: true, audio: true })
.then((stream) => {
userStream = stream;
myVideo.current.srcObject = stream;
console.log(1);
});
});
socket.on("joined", () => {
navigator.mediaDevices
.getUserMedia({ video: true, audio: true })
.then((stream) => {
userStream = stream;
myVideo.current.srcObject = stream;
console.log(2);
});
socket.emit("ready", roomName);
});
socket.on("ready", () => {
if (creator) {
const peer = new Peer({
initiator: true,
trickle: false,
stream: userStream,
});
peer.on("signal", (signal) => {
socket.emit("sendingSignal", { signal, roomName });
console.log(3);
});
peer.on("stream", (stream) => {
userVideo.current.srcObject = stream;
console.log(4);
});
socket.on("receivingSignal", (signal) => {
peer.signal(signal);
console.log(5);
});
connectionRef.current = peer;
}
});
socket.on("offer", (incomingSignal) => {
if (!creator) {
const peer = new Peer({
initiator: false,
trickle: false,
stream: userStream,
});
peer.on("signal", (signal) => {
socket.emit("returningSignal", { signal, roomName });
console.log(6);
});
peer.on("stream", (stream) => {
userVideo.current.srcObject = stream;
console.log(7);
});
peer.signal(incomingSignal);
console.log(8);
connectionRef.current = peer;
}
});
}, []);
I'm learning graphQL with React native and using the apollo client. I'm experimenting with some code that has a simple login screen and I'm trying to check my understanding of the cache. My graphql client code is below. By turning on the debug for the persistCache I see the line when use CMD + R to reload an iOS simulator with expo. This suggests the cache is working.
[apollo-cache-persist] Restored cache of size 29
My question is what else is needed to complete the overall process of not needing to login again? I assume I need to maintain state on whether it's logged in and not show the login screen. I'm after some examples which show this.
const retryLink = new RetryLink({
delay: {
initial: 300,
max: 5000,
jitter: true
},
attempts: {
max: Infinity,
retryIf: (error = {}) => {
return error.statusCode > 299 || !error.statusCode
}
}
});
const formatObject = data => _.isObject(data) ? JSON.stringify(data) : data;
const formatGraphQLError = err =>
`Message: ${err.message}, Location: ${formatObject(
err.locations
)}`;
const errorLink = onError(({ networkError = "", graphQLErrors = [] } = {}) => {
if (networkError)
console.log(`[Network Error]: ${networkError}`);
if (graphQLErrors.length)
graphQLErrors.map(formatGraphQLError).forEach(err => console.log(`[GraphQL Error]: ${err}`))
});
const authLink = setContext(async (_, { headers }) => {
const token = await Auth.token();
if (token)
return {
headers: {
...headers,
authorization: `Bearer ${token}`
}
};
else return { headers };
});
const httpLink = new HttpLink({
uri: Config.apiUrl
});
const cache = new InMemoryCache();
// Set up cache persistence.
persistCache({
cache,
storage: AsyncStorage,
trigger: 'background',
debug: true
});
const logLink = new ApolloLink((operation, forward) => {
console.log("Running GraphQL query or mutation");
return forward(operation);
});
//--
//-- Combine the links in your required order.
//--
let _notifications = 42;
const client = new ApolloClient({
resolvers: {
Query: {
permission: async (_, { type }) => await Permissions.askAsync(type),
token: async () => await Auth.token(),
notifications: () => _notifications
},
Mutation: {
login: async (_, { email, password }) => {
return await Auth.login(email, password)
},
updateNotifications: async (_, { notifications }) => _notifications = notifications
}
},
link: ApolloLink.from([
logLink,
retryLink,
errorLink,
authLink,
httpLink
]),
cache
});
export default client;
I am using Robin Wieruch's fullstack boilerplate but it is missing authentication for subscriptions. It uses JWT token for sessions and it is working fine for http but for ws auth is completely missing.
I need to pass user trough context for subscriptions as well, I need session info in subscriptions resolver to be able to decide weather I should fire subscription or not.
I did search Apollo docs, I saw I should use onConnect: (connectionParams, webSocket, context) function, but there is no fully functional fullstack example, I am not sure how to pass JWT from client to be able to get it in webSocket object.
Here is what I have so far:
Server:
import express from 'express';
import {
ApolloServer,
AuthenticationError,
} from 'apollo-server-express';
const app = express();
app.use(cors());
const getMe = async req => {
const token = req.headers['x-token'];
if (token) {
try {
return await jwt.verify(token, process.env.SECRET);
} catch (e) {
throw new AuthenticationError(
'Your session expired. Sign in again.',
);
}
}
};
const server = new ApolloServer({
introspection: true,
typeDefs: schema,
resolvers,
subscriptions: {
onConnect: (connectionParams, webSocket, context) => {
console.log(webSocket);
},
},
context: async ({ req, connection }) => {
// subscriptions
if (connection) {
return {
// how to pass me here as well?
models,
};
}
// mutations and queries
if (req) {
const me = await getMe(req);
return {
models,
me,
secret: process.env.SECRET,
};
}
},
});
server.applyMiddleware({ app, path: '/graphql' });
const httpServer = http.createServer(app);
server.installSubscriptionHandlers(httpServer);
const isTest = !!process.env.TEST_DATABASE_URL;
const isProduction = process.env.NODE_ENV === 'production';
const port = process.env.PORT || 8000;
httpServer.listen({ port }, () => {
console.log(`Apollo Server on http://localhost:${port}/graphql`);
});
Client:
const httpLink = createUploadLink({
uri: 'http://localhost:8000/graphql',
fetch: customFetch,
});
const wsLink = new WebSocketLink({
uri: `ws://localhost:8000/graphql`,
options: {
reconnect: true,
},
});
const terminatingLink = split(
({ query }) => {
const { kind, operation } = getMainDefinition(query);
return (
kind === 'OperationDefinition' && operation === 'subscription'
);
},
wsLink,
httpLink,
);
const authLink = new ApolloLink((operation, forward) => {
operation.setContext(({ headers = {} }) => {
const token = localStorage.getItem('token');
if (token) {
headers = { ...headers, 'x-token': token };
}
return { headers };
});
return forward(operation);
});
const errorLink = onError(({ graphQLErrors, networkError }) => {
if (graphQLErrors) {
graphQLErrors.forEach(({ message, locations, path }) => {
console.log('GraphQL error', message);
if (message === 'UNAUTHENTICATED') {
signOut(client);
}
});
}
if (networkError) {
console.log('Network error', networkError);
if (networkError.statusCode === 401) {
signOut(client);
}
}
});
const link = ApolloLink.from([authLink, errorLink, terminatingLink]);
const cache = new InMemoryCache();
const client = new ApolloClient({
link,
cache,
resolvers,
typeDefs,
});
You need to use connectionParams to set the JWT from the client-side. Below is the code snippet using the angular framework:
const WS_URI = `wss://${environment.HOST}:${environment.PORT}${
environment.WS_PATH
}`;
const wsClient = subscriptionService.getWSClient(WS_URI, {
lazy: true,
// When connectionParams is a function, it gets evaluated before each connection.
connectionParams: () => {
return {
token: `Bearer ${authService.getJwt()}`
};
},
reconnect: true,
reconnectionAttempts: 5,
connectionCallback: (error: Error[]) => {
if (error) {
console.log(error);
}
console.log('connectionCallback');
},
inactivityTimeout: 1000
});
const wsLink = new WebSocketLink(wsClient);
In your server-side, you are correct, using onConnect event handler to handle the JWT. E.g.
const server = new ApolloServer({
typeDefs,
resolvers,
context: contextFunction,
introspection: true,
subscriptions: {
onConnect: (
connectionParams: IWebSocketConnectionParams,
webSocket: WebSocket,
connectionContext: ConnectionContext,
) => {
console.log('websocket connect');
console.log('connectionParams: ', connectionParams);
if (connectionParams.token) {
const token: string = validateToken(connectionParams.token);
const userConnector = new UserConnector<IMemoryDB>(memoryDB);
let user: IUser | undefined;
try {
const userType: UserType = UserType[token];
user = userConnector.findUserByUserType(userType);
} catch (error) {
throw error;
}
const context: ISubscriptionContext = {
// pubsub: postgresPubSub,
pubsub,
subscribeUser: user,
userConnector,
locationConnector: new LocationConnector<IMemoryDB>(memoryDB),
};
return context;
}
throw new Error('Missing auth token!');
},
onDisconnect: (webSocket: WebSocket, connectionContext: ConnectionContext) => {
console.log('websocket disconnect');
},
},
});
server-side: https://github.com/mrdulin/apollo-graphql-tutorial/blob/master/src/subscriptions/server.ts#L72
client-side: https://github.com/mrdulin/angular-apollo-starter/blob/master/src/app/graphql/graphql.module.ts#L38
I would like connect 2 devices with WebRTC on localhost. All devices have no internet access. They are connected to a same local wifi.
I try this on React Native App.
In this context offline, do I need to trickle ICE candidates and addIceCandidate ? If I understund correctly, ICE candidates is for iceServer. But my case, iceServer is null (because i'm offline only, connected on same localhost wifi) :
const configuration = { iceServers: [{ urls: [] }] };
So actualty i exchange offer and answer, but after setRemoteDescription the answer, the connectionState stay on checking.
You can see my React Component :
constructor(props) {
super(props);
this.pc = new RTCPeerConnection(configuration);
}
state = initialState;
componentDidMount() {
const { pc } = this;
if (pc) {
this.setState({
peerCreated: true
});
}
this.setConnectionState();
pc.oniceconnectionstatechange = () => this.setConnectionState();
pc.onaddstream = ({ stream }) => {
if (stream) {
this.setState({
receiverVideoURL: stream.toURL()
});
}
};
pc.onnegotiationneeded = () => {
if (this.state.initiator) {
this.createOffer();
}
};
pc.onicecandidate = ({ candidate }) => {
if (candidate === null) {
const { offer } = this.state;
const field = !offer ? 'offer' : 'data';
setTimeout(() => {
alert('setTimeout started');
this.setState({
[field]: JSON.stringify(pc.localDescription)
});
}, 2000);
}
};
}
#autobind
setConnectionState() {
this.setState({
connectionState: this.pc.iceConnectionState
});
}
getUserMedia() {
MediaStreamTrack.getSources(() => {
getUserMedia(
{
audio: false,
video: true
},
this.getUserMediaSuccess,
this.getUserMediaError
);
});
}
#autobind
async getUserMediaSuccess(stream) {
const { pc } = this;
pc.addStream(stream);
await this.setState({ videoURL: stream.toURL() });
if (this.state.initiator) {
return this.createOffer();
}
return this.createAnswer();
}
getUserMediaError(error) {
console.log(error);
}
#autobind
logError(error) {
const errorArray = [...this.state.error, error];
return this.setState({
error: errorArray
});
}
/**
* Create offer
*
* #memberof HomeScreen
*/
#autobind
createOffer() {
const { pc } = this;
pc.createOffer()
.then(offer => pc.setLocalDescription(offer))
.then(() => {
this.setState({
offerCreated: true
});
})
.catch(this.logError);
}
/**
* Create anwser
*
* #memberof HomeScreen
*/
#autobind
async createAnswer() {
const { pc } = this;
const { data } = this.state;
if (data) {
const sd = new RTCSessionDescription(JSON.parse(data));
await this.setState({
offerImported: true
});
pc.setRemoteDescription(sd)
.then(() => pc.createAnswer())
.then(answer => pc.setLocalDescription(answer))
.then(() => {
this.setState({
answerCreated: true
});
})
.catch(this.logError);
}
}
#autobind
receiveAnswer() {
const { pc } = this;
const { data } = this.state;
const sd = new RTCSessionDescription(JSON.parse(data));
return pc
.setRemoteDescription(sd)
.then(() => {
this.setState({
answerImported: true
});
})
.catch(this.logError);
}
/**
* Start communication
*
* #param {boolean} [initiator=true]
* #returns
* #memberof HomeScreen
*/
#autobind
async start(initiator = true) {
if (!initiator) {
await this.setState({
initiator: false
});
}
return this.getUserMedia();
}
Anyone can help me?
No iceServers is fine on a LAN, but peers must still exchange at least one candidate: their host candidate (based on their machine's LAN IP address).
Either:
Trickle candidates using onicecandidate -> signaling -> addIceCandidate as usual, or
Out-wait the ICE process (a few seconds) before exchanging pc.localDescription.
It looks like you're attempting the latter. This approach works because...
Trickle ICE is an optimization.
The signaling (trickling) of individual ice candidates using onicecandidate, is an optimization meant to speed up negotiation. Once setLocalDescription succeeds, the browser's internal ICE Agent starts, inserting ICE candidates, as they're discovered, into the localDescription itself. Wait a few seconds to negotiate, and trickling isn't necessary at all: all ICE candidates will be in the offer and answer transmitted.
Your code
From your onicecandidate code it looks like you're already attempting to gather the localDescription after ICE completion (and you've written it to work from both ends):
pc.onicecandidate = ({ candidate }) => {
if (!candidate) {
const { offer } = this.state;
const field = !offer ? 'offer' : 'data';
this.setState({
[field]: JSON.stringify(pc.localDescription)
});
}
};
In the offerer side you've correctly commented out the equivalent code in createOffer:
pc.createOffer()
.then(offer => pc.setLocalDescription(offer))
.catch(this.logError);
// .then(() => {
// this.setState({
// offer: JSON.stringify(pc.localDescription)
// });
// });
But on the answerer side, you have not, and that's likely the problem:
createAnswer() {
const { pc } = this;
const { data } = this.state;
if (data) {
const sd = new RTCSessionDescription(JSON.parse(data));
pc.setRemoteDescription(sd)
.then(() => pc.createAnswer())
.then(answer => pc.setLocalDescription(answer))
.then(() => {
this.setState({
offer: JSON.stringify(pc.localDescription)
});
})
.catch(this.logError);
}
}
This means it sends an answer back immediately, before the answerer's ICE agent has had time to insert any candidates into the answer. This is probably why it fails.
On a side-note: Nothing appears to wait for getUserMedia to finish either, so answers likely won't contain any video either, depending on the timing of your getUserMediaSuccess function, which fails to add any tracks or streams to the connection. But assuming you're just doing data channels, this should work with my recommended fixes.