WebRTC + IOS + Freeswitch：听不到音频

我正在尝试在IOS上实现mod_verto （从iPhone调用到桌面）。 我正在使用Google的libjingle库用于RTC端，使用这个优秀的教程完成并运行。

  • 从我的iPhone拨打电话时，我使用Verto Communicator （在本地计算机上下载并运行）在桌面浏览器上接听电话。
  • 在iPhone方面，我可以听到桌面上的音频，但桌面方面没有任何声音 。

  • 如果我使用2个浏览器窗口（使用Verto Communicator）进行呼叫，则一切正常。

  • 完全披露 ，我使用ws:// unsecure websocket连接到FreeSwitch

这是我的JSONRPC日志：

发送登录请求： 

 {"jsonrpc":"2.0","method":"login","id":1,"params":{"login":"1000@MY-IP-ADDRESS","loginParams":{},"userVariables":{},"passwd":"1234","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

登录响应： 

 {"jsonrpc":"2.0","id":1,"result":{"message":"logged in","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

verto.invite（包括iPhone sdp）： 

 {"jsonrpc":"2.0","method":"verto.invite","id":2,"params":{"dialogParams":{"remote_caller_id_number":"1008","useVideo":false,"useMic":"any","useStereo":false,"tag":"webcam","login":"1000@159.203.164.7","useCamera":"any","videoParams":{"minFrameRate":30,"minWidth":"1280","minHeight":"720"},"destination_number":"1008","screenShare":false,"caller_id_name":"FreeSWITCH User","caller_id_number":"1000","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800","remote_caller_id_name":"Outbound Call","useSpeak":"any"},"sessid":"53FB0781-B586-4CDA-98C6-558680663B46","sdp":"v=0\r\no=- 8564086442942257834 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE audio video\r\na=msid-semantic: WMS\r\nm=audio 58157 UDP\/TLS\/RTP\/SAVPF 111 103 104 9 102 0 8 106 105 13 127 126\r\nc=IN IP4 82.166.93.197\r\na=rtcp:52576 IN IP4 82.166.93.197\r\na=candidate:3168280865 1 udp 2122260223 11.0.0.244 58157 typ host generation 0\r\na=candidate:1260196625 1 udp 2122194687 10.134.172.254 58951 typ host generation 0\r\na=candidate:3168280865 2 udp 2122260222 11.0.0.244 52576 typ host generation 0\r\na=candidate:1260196625 2 udp 2122194686 10.134.172.254 58945 typ host generation 0\r\na=candidate:4066106833 1 tcp 1518280447 11.0.0.244 60562 typ host tcptype passive generation 0\r\na=candidate:94302177 1 tcp 1518214911 10.134.172.254 60563 typ host tcptype passive generation 0\r\na=candidate:4066106833 2 tcp 1518280446 11.0.0.244 60564 typ host tcptype passive generation 0\r\na=candidate:94302177 2 tcp 1518214910 10.134.172.254 60565 typ host tcptype passive generation 0\r\na=candidate:1610196941 1 udp 1686052607 82.166.93.197 58157 typ srflx raddr 11.0.0.244 rport 58157 generation 0\r\na=candidate:1610196941 2 udp 1686052606 82.166.93.197 52576 typ srflx raddr 11.0.0.244 rport 52576 generation 0\r\na=candidate:2274372738 2 udp 1685987070 176.13.15.205 5834 typ srflx raddr 10.134.172.254 rport 58945 generation 0\r\na=candidate:2274372738 1 udp 1685987071 176.13.15.205 5840 typ srflx raddr 10.134.172.254 rport 58951 generation 0\r\na=ice-ufrag:g8lHDtPwH7m5xRex\r\na=ice-pwd:Q6jcBJNTWAyu0JTuIaQAeNI3\r\na=fingerprint:sha-256 0F:A1:68:51:87:3E:B4:C1:0D:33:97:40:78:22:2A:8C:D2:B6:46:23:F5:99:C9:88:5D:34:DB:E2:C5:94:B3:DD\r\na=setup:actpass\r\na=mid:audio\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:3 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus\/48000\/2\r\na=fmtp:111 minptime=10; useinbandfec=1\r\na=rtpmap:103 ISAC\/16000\r\na=rtpmap:104 ISAC\/32000\r\na=rtpmap:9 G722\/8000\r\na=rtpmap:102 ILBC\/8000\r\na=rtpmap:0 PCMU\/8000\r\na=rtpmap:8 PCMA\/8000\r\na=rtpmap:106 CN\/32000\r\na=rtpmap:105 CN\/16000\r\na=rtpmap:13 CN\/8000\r\na=rtpmap:127 red\/8000\r\na=rtpmap:126 telephone-event\/8000\r\na=maxptime:60\r\nm=video 61966 UDP\/TLS\/RTP\/SAVPF 100 101 116 117 96\r\nc=IN IP4 82.166.93.197\r\na=rtcp:63816 IN IP4 82.166.93.197\r\na=candidate:3168280865 1 udp 2122260223 11.0.0.244 61966 typ host generation 0\r\na=candidate:1260196625 1 udp 2122194687 10.134.172.254 50435 typ host generation 0\r\na=candidate:3168280865 2 udp 2122260222 11.0.0.244 63816 typ host generation 0\r\na=candidate:1260196625 2 udp 2122194686 10.134.172.254 63396 typ host generation 0\r\na=candidate:4066106833 1 tcp 1518280447 11.0.0.244 60566 typ host tcptype passive generation 0\r\na=candidate:94302177 1 tcp 1518214911 10.134.172.254 60567 typ host tcptype passive generation 0\r\na=candidate:4066106833 2 tcp 1518280446 11.0.0.244 60568 typ host tcptype passive generation 0\r\na=candidate:94302177 2 tcp 1518214910 10.134.172.254 60569 typ host tcptype passive generation 0\r\na=candidate:1610196941 1 udp 1686052607 82.166.93.197 61966 typ srflx raddr 11.0.0.244 rport 61966 generation 0\r\na=candidate:1610196941 2 udp 1686052606 82.166.93.197 63816 typ srflx raddr 11.0.0.244 rport 63816 generation 0\r\na=candidate:2274372738 1 udp 1685987071 176.13.15.205 5879 typ srflx raddr 10.134.172.254 rport 50435 generation 0\r\na=candidate:2274372738 2 udp 1685987070 176.13.15.205 5860 typ srflx raddr 10.134.172.254 rport 63396 generation 0\r\na=ice-ufrag:g8lHDtPwH7m5xRex\r\na=ice-pwd:Q6jcBJNTWAyu0JTuIaQAeNI3\r\na=fingerprint:sha-256 0F:A1:68:51:87:3E:B4:C1:0D:33:97:40:78:22:2A:8C:D2:B6:46:23:F5:99:C9:88:5D:34:DB:E2:C5:94:B3:DD\r\na=setup:actpass\r\na=mid:video\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:100 VP8\/90000\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtpmap:101 VP9\/90000\r\na=rtcp-fb:101 ccm fir\r\na=rtcp-fb:101 nack\r\na=rtcp-fb:101 nack pli\r\na=rtcp-fb:101 goog-remb\r\na=rtcp-fb:101 transport-cc\r\na=rtpmap:116 red\/90000\r\na=rtpmap:117 ulpfec\/90000\r\na=rtpmap:96 rtx\/90000\r\na=fmtp:96 apt=100\r\n"}}

调用创建的响应： 

 {"jsonrpc":"2.0","id":2,"result":{"message":"CALL CREATED","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

verto.media被调用： 

 {"jsonrpc":"2.0","method":"verto.media","id":637,"params":{"sdp":"v=0\no=FreeSWITCH 1457232832 1457232833 IN IP4 159.203.164.7\ns=FreeSWITCH\nc=IN IP4 159.203.164.7\nt=0 0\na=msid-semantic: WMS TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\nm=audio 30784 UDP/TLS/RTP/SAVPF 111 126\na=rtpmap:111 opus/48000/2\na=fmtp:111 useinbandfec=1; minptime=10\na=rtpmap:126 telephone-event/8000\na=silenceSupp:off - - - -\na=ptime:20\na=sendonly\na=fingerprint:sha-256 FE:CD:54:3E:2A:D7:DB:00:57:B7:D4:55:A8:EB:79:08:16:BB:B0:EA:43:44:42:9A:90:01:49:37:7B:31:48:F8\na=setup:active\na=rtcp-mux\na=rtcp:30784 IN IP4 159.203.164.7\na=ice-ufrag:qLh1zzclxONPNyQO\na=ice-pwd:G7g4Drkist37beYsP5jfvlqS\na=candidate:9922185636 1 udp 659136 159.203.164.7 30784 typ host generation 0\na=ssrc:1323504502 cname:bhqCyFkpPbjUPSk0\na=ssrc:1323504502 msid:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub a0\na=ssrc:1323504502 mslabel:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\na=ssrc:1323504502 label:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Uba0\nm=video 31380 UDP/TLS/RTP/SAVPF 100\na=rtpmap:100 VP8/90000\na=sendonly\na=fingerprint:sha-256 FE:CD:54:3E:2A:D7:DB:00:57:B7:D4:55:A8:EB:79:08:16:BB:B0:EA:43:44:42:9A:90:01:49:37:7B:31:48:F8\na=setup:active\na=rtcp-mux\na=rtcp:31380 IN IP4 159.203.164.7\nb=AS:1024\na=rtcp-fb:100 ccm fir\na=rtcp-fb:100 nack\na=rtcp-fb:100 nack pli\na=ssrc:594893571 cname:bhqCyFkpPbjUPSk0\na=ssrc:594893571 msid:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub v0\na=ssrc:594893571 mslabel:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\na=ssrc:594893571 label:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ubv0\na=ice-ufrag:2KDK4wDMYuAuVdAZ\na=ice-pwd:YTpxObqpLuBEfig7TKHN6bqU\na=candidate:7508673635 1 udp 659136 159.203.164.7 31380 typ host generation 0\n","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800"}}

verto.answer被调用： 

 {"jsonrpc":"2.0","method":"verto.answer","id":638,"params":{"callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800"}}

问：为了在浏览器端听到音频，我错过了什么？
感谢任何信息:)

更新，添加了freeswitch日志

更新2 IOS：音频流代码 

 ... let audioTrack = self.factory.audioTrackWithID("Local-Audio") self.localMediaStream?.addAudioTrack(audioTrack); self.peerConnection!.addStream(self.localMediaStream) ...

更新3 – 部分解决方案在检查我的代码时，我发现用于向我的本地媒体流添加video轨道的旧代码，禁用此部分解决了音频问题， 但为什么？ 该代码有什么问题？

PS Promise类是由朋友创建的，模仿JS Promise方法。 

 func getUserMedia(mediaOptions:Dictionary? = nil) -> Promise{ return Promise(executor: { (resolve, reject) -> () in var cameraID:String? self.localMediaStream = self.factory.mediaStreamWithLabel("Local-Meida") //if video option is enabled (default true) //-------------- Disabling this section solves the audio issues -------------- if(mediaOptions?["video"] as? Bool ?? true){ for captureDevice in AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo){ if (captureDevice.position == mediaOptions?["devicePosition"] as? AVCaptureDevicePosition ?? AVCaptureDevicePosition.Front){ cameraID = captureDevice.localizedName break } } if(cameraID == nil){ reject(NSError(domain: "No cammera detected", code: 0, userInfo: nil)) } let capturer = RTCVideoCapturer.init(deviceName: cameraID) let videoSource = self.factory.videoSourceWithCapturer(capturer, constraints: mediaOptions?["constraints"] as? RTCMediaConstraints ?? nil) if let localVideoTrack = self.factory.videoTrackWithID("Local-Video", source: videoSource){ //!!!! THIS IS THE PROBLEMATIC LINE !!!! self.localMediaStream?.addVideoTrack(localVideoTrack) }else{ reject(NSError(domain: "No Video track", code: 0, userInfo: nil)) } } //-------------- Disabling this section solves the audio issues -------------- if(mediaOptions?["audio"] as? Bool ?? true){ let audioTrack = self.factory.audioTrackWithID("Local-Audio") self.localMediaStream?.addAudioTrack(audioTrack); } self.peerConnection!.addStream(self.localMediaStream) resolve(self.localMediaStream!) }) }

调试有问题的行 在此处输入图像描述

更新

除非您在我们的案例中检查过媒体服务器 ， Web客户端和iOS客户端 ，否则很难理解WebRTC实现的问题。

您的案例是音频呼叫 ，因此您的localStream不需要包含任何video流，但如果仔细观察， 您会发现您实际上是在移动流中添加了videoTrack ： 

  if(mediaOptions?["video"] as? Bool ?? true){ for captureDevice in AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo){ if (captureDevice.position == mediaOptions?["devicePosition"] as? AVCaptureDevicePosition ?? AVCaptureDevicePosition.Front){ cameraID = captureDevice.localizedName break } } if(cameraID == nil){ reject(NSError(domain: "No cammera detected", code: 0, userInfo: nil)) } let capturer = RTCVideoCapturer.init(deviceName: cameraID) let videoSource = self.factory.videoSourceWithCapturer(capturer, constraints: mediaOptions?["constraints"] as? RTCMediaConstraints ?? nil) if let localVideoTrack = self.factory.videoTrackWithID("Local-Video", source: videoSource){ //!!!! THIS IS THE PROBLEMATIC LINE !!!! self.localMediaStream?.addVideoTrack(localVideoTrack) }else{ reject(NSError(domain: "No Video track", code: 0, userInfo: nil)) } }

因此导致问题的一行是： self.localMediaStream?.addVideoTrack(localVideoTrack) ，因为您将video附加到localStream 。

意见

正如我所提到的，我们可能有不同的麻烦情况，在这里我根据我在构建类似系统时的经验列出了一些意见：

  1. 您的MediaServer可能没有可以在成功状态下重定向和处理您的呼叫的实现，因为在您附加video时会添加其他内容（请参阅您的会话描述您实际发送的内容），并且它只是拒绝创建呼叫。
  2. 即使您的MediaServer处理该场景，这将包括Client （ 桌面和移动 ）的正确实现，以符合其协议的信令。
  3. 您通过了所有测试，现在正在添加video和音频，因此您从移动设备启动localStream，同样需要以其他方式创建。 然后，当您通过websockets添加流，删除流和其他内容时，您需要处理事件。

在这种情况下解决方案

删除在localStream中添加localTrack的部分，然后即使您有错误，也不是由创建localStream引起的，所以此步骤当前已解决。

原始答案

在这里，我有一个我的工作版本，但只适合您的需要，因为您只使用音频。

创建和设置peerConnection（localSide） 

 // Connecting to the socket ......... // Create PeerConnectionFactory self.peerConnectionFactory = [[RTCPeerConnectionFactory alloc] init]; RTCMediaConstraints *constraints = [self defaultPeerConnectionConstraints]; // Initialize peerConnection based on a list of ICE Servers self.peerConnection = [self.peerConnectionFactory peerConnectionWithICEServers:[self getICEServers] constraints:constraints delegate:self]; // Create the localStram which contains the audioTrack RTCMediaStream *localStream = [self createLocalMediaStream]; // Add this stream to the peerConnection [self.peerConnection addStream:localStream]; // Please be aware here that I am using blocks, as I created a wrapper for easier maintenance, but you can use createOfferWithDelegate: which will go back at your delegation NSLog(@"Creating peer offer"); RTCManager *strongSelf = self; [strongSelf.peerConnection createOfferWithCallback:^(RTCSessionDescription *sdp, NSError *error) { if (!error) { dispatch_async(dispatch_get_main_queue(), ^{ NSLog(@"Success at creating offer, now setting local description"); [strongSelf.peerConnection setLocalDescriptionWithCallback:^(NSError *error) { if (!error) { dispatch_async(dispatch_get_main_queue(), ^{ NSLog(@"Success at setting local description"); // On my type of signalization here I am connected, but yours is based on what type of signalization requires }); } } sessionDescription:sdp]; }); } } constraints:[strongSelf defaultPeerConnectionConstraints]];

助手 

 // Now here we create the stream which contains the audio (Please note the ID) - (RTCMediaStream *)createLocalMediaStream { RTCMediaStream *localStream = [self.peerConnectionFactory mediaStreamWithLabel:@"ARDAMS"]; [localStream addAudioTrack:[self.peerConnectionFactory audioTrackWithID:@"ARDAMSa0"]]; return localStream; } - (RTCMediaConstraints *)defaultPeerConnectionConstraints { // DtlsSrtpKeyAgreement is required for Chrome and Firefox to interoperate. NSArray *optionalConstraints = @[[[RTCPair alloc] initWithKey:@"DtlsSrtpKeyAgreement" value:@"true"]]; RTCMediaConstraints *constraints = [[RTCMediaConstraints alloc] initWithMandatoryConstraints:nil optionalConstraints:optionalConstraints]; return constraints; }

请注意，您的问题可能也是因为没有在主线程上调用东西。

Interesting Posts

无法运行bms-samples-cordova-hellopush – 找不到bms_samples_cordova_push-Swift.h文件

cordovaiPhone上的位置权限提醒

如何使用ALAsset获取图像的裁剪版本？

将私钥添加到iOS Keychain中

Xcode的uitesting：staticTexts开始

哪里可以findiOS模拟器的GPX文件

Swift中的App架构和对象组成

Swift不能使用函数指针吗？

不能在其他文件的扩展名中使用私有属性

IOS 5如何更改导航栏中后退按钮的颜色？