I am using SpeechRecognitionEngine to recognize information being spoken by a user. The method will be running on the client's computer and it is working just fine and recognizing text almost like I want it to. So I'm happy.
However, I want to be able to do some processing of the wave file on my server. Right now I am testing on my local machine and when I use the SetInputToWaveFile method on the Recognizer, and pass the same audio clip back through (the one recorded by the engine originally) it does not give anything close to the original matches (or alternates).
For example:
User speaks and recognizer returns the phrase: "Hello how are you today" with 10 alternates.
Wave file is saved and then passed in through using SetInputToWaveFile (or SetInputToAudioStream). The recognizer will return a phrase (usually one word) that is nothing like the spoken text, example "Moon" along with ZERO alternates.
Often, when doing this, the recognizer will not raise the RecognizeCompleted event. It will however sometimes raise the SpeechHypothesized event, other times the AudioSignalProblem occured.
Shouldn't passing the audio clip that was captured from the recognizer results, back through the same recognizer, return the same matches?
Original:
Private _recognizer As New SpeechRecognitionEngine(New CultureInfo("en-US"))
_recognizer.UnloadAllGrammars()
_recognizer.LoadGrammar(New DictationGrammar())
_recognizer.SetInputToDefaultAudioDevice()
_recognizer.InitialSilenceTimeout = TimeSpan.FromSeconds(2)
_recognizer.MaxAlternates = 10
_recognizer.BabbleTimeout = TimeSpan.FromSeconds(1)
Dim result As RecognitionResult = _recognizer.Recognize()
Dim aud As RecognizedAudio = _result.Audio 'This is the audio that gets saved
aud.WriteToWaveStream("mypath")
(I've removed some of the logic code in between that pulls out the results, and does some processing)
Now trying to pull out from the Audio file:
_recognizer.SetInputToWaveFile("mypath")
'Doesn't work either
'_recognizer.SetInputToAudioStream(File.OpenRead("mypath"), New SpeechAudioFormatInfo(44100, AudioBitsPerSample.Sixteen, AudioChannel.Mono))
Dim result2 As RecognitionResult = _recognizer.Recognize()
The recognition/matches from result and result2 are not even close.
I manually set the speech audio format info, and now it works perfectly.
_recognizer.SetInputToAudioStream(File.OpenRead("mypath"), New SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, Nothing))
Related
Im new to Lua and the gmod gamemode creation, and I'm having a bit of trouble. I want to deactivate a HUD when the game starts. I have 2 files, one the init.lua file, where a function is called that the game starts (there I want to change the value of HUD.lua) and a HUD.lua file, where the HUD is drawn and it contains the variable I want to change.
I tried multiple approaches, like referencing the script like:
hud.gameBegan = true
, but that didn't worked, so I tried also this putting into my init.lua:
SetNWBool("gameBegan", true)
and then I put this into the HUD.lua:
gameBegan = GetNWBool("gameBegan")
Lastly I tried this:
hud = AddCSLuaFile("hud.lua")
hud:gameChanged(true)
Unfortunatly, neither of these approaches worked for me, can somebody help me?
I would suggest keeping a table on the GM table for your gamemode to hold game states. This would then be synced between the server and the client using network messages.
Essentially how it will work is once the game starts, the server will send a network message to every player telling them that game started is true. When a new player joins, they will also need to know whether the game has started yet or not, and so we will also have to send a network message from the server to any new player that joins, telling them whether game started is true or false.
Once the game ends we need to also inform every player that the game has ended.
To start we need to store the states somewhere, and since whether a game has started or not relates to the gamemode it may be best to store it on the GAMEMODE table, and it also needs to be defined for the server and each player, so we should use the GAMEMODE table in gamemode/shared.lua:
GAMEMODE.States = GAMEMODE.States or {}
GAMEMODE.States.GameStarted = false
In your gamemode/init.lua file (which runs on the server) you may then add something like this:
util.AddNetworkString("MyGamemode.GameStartedSync")
function GM:SetGameStarted(hasStarted)
GAMEMODE.States.GameStarted = hasStarted
-- We have updated the state on the server, now update it
-- for each player on their client
net.Start("MyGamemode.GameStartedSync")
net.WriteBool(hasStarted)
net.Broadcast()
end
function GM:PlayerInitialSpawn(ply, transition)
BaseClass.PlayerInitialSpawn(self, ply, transition)
-- Player has joined, so send them the current game state
net.Start("MyGamemode.GameStartedSync")
net.WriteBool(GAMEMODE.States.GameStarted)
net.Send(ply)
end
If you already have a GM:PlayerInitialSpawn(ply) function then simply merge the contents of that one with yours.
(Note you will need DEFINE_BASECLASS("gamemode_base") at the top of your init.lua file to make BaseClass available if you don't already.)
In you gamemode/cl_init.lua file you need to now write the code on the player's client that can receive the network message:
net.Receive("MyGamemode.GameStartedSync", function()
local hasStarted = net.ReadBool()
GAMEMODE.States.GameStarted = hasStarted
end)
You can then set the sate of whether the game has started using GAMEMODE:SetGameStarted(true) or GAMEMODE:SetGameStarted(false) in any serverside script. And its value can be used with GAMEMODE.States.GameStarted on both the client and the server.
e.g.
if GAMEMODE.States.GameStarted then
DrawMyHud()
end
I am not familiar with GMod but maybe this can work?
In your init.lua file, you can use the include function to include the HUD.lua file and then set the variable to deactivate the HUD.
Here is an example:
include("HUD.lua") -- include the HUD file
HUDEnabled = false -- set the variable to false
In your HUD.lua file, you can then check the value of the variable before drawing the HUD:
if HUDEnabled then
-- code to draw HUD
end
You can also move the variable in the HUD.lua file so you can use it in the HUD.lua file and init.lua file.
-- HUD.lua
local HUDEnabled = true
if HUDEnabled then
-- code to draw HUD
end
-- init.lua
include("HUD.lua")
HUDEnabled = false
According to the Discord.NET documentation page for the EmbedBuilder class, the syntax (converted to VB) to add a local image to an EmbedBuilder object should look something like this:
Dim fileName = "image.png"
Dim embed = New EmbedBuilder() With {
.ImageUrl = $"attachment://{fileName}"
}.Build()
I'm trying to use something like this to add a dynamically created image to the EmbedBuilder, but I can't seem to get it to work properly. Here's basically what I've got:
Dim TweetBuilder As New Discord.EmbedBuilder
Dim DynamicImagePath As String = CreateDynamicImage()
Dim AttachURI As String = $"attachment:///" & DynamicImagePath.Replace("\", "/").Replace(" ", "%20")
With Builder
.Description = "SAMPLE DESCRIPTION"
.ImageUrl = AttachURI
End With
MyClient.GetGuild(ServerID).GetTextChannel(PostChannelID).SendMessageAsync("THIS IS A TEST", False, Builder.Build)
My CreateDynamicImage method returns the full path to the locally created image (e.g., C:\Folder\Another Folder\image.png). I've done a fair amount of "fighting"/testing with this to get past the Url must be a well-formed URI exception I was initially getting because of the [SPACE] in the path.
MyClient is a Discord.WebSocket.SocketClient object set elsewhere.
The SendMessageAsync method does send the Embed to Discord on the correct channel, but without the embedded image.
If I instead send the image using the SendFileAsync method (like so):
MyClient.GetGuild(ServerID).GetTextChannel(PostChannelID).SendFileAsync(DynamicImagePath, "THIS IS A TEST", False, Builder.Build)
the image is sent, but as a part of the message, rather than included as a part of the Embed (this is expected behavior - I only bring it up b/c it was a part of my testing to ensure that there wasn't a problem with actually sending the image to Discord).
I've tried using the file:/// scheme instead of the attachment:/// scheme, but that results in the entire post never making it to Discord at all.
Additionally, I've tried setting the ImageUrl property to a Web resource (e.g., https://www.somesite.com/someimage.png) and the Embed looks exactly as expected with the image and everything when it successfully posts to Discord.
So, I'm just wondering at this point if I'm just missing something, or if I'm just doing it completely wrong?
I cross-posted this to issue #1609 in the Discord.Net GitHub project to get a better idea of what options are available for this and received a good explanation of the issue:
The Embed (and EmbedImage) objects don't do anything with files. They simply pass the URI as configured straight into Discord. Discord then expects a URI in the form attachment://filename.ext if you want to refer to an attached image.
What you need to do is use SendFileAsync with the embed. You have two options here:
Use SendFileAsync with the Stream stream, string filename overload. I think this makes it clear what you need to do: you provide a file stream (via File.OpenRead or similar) and a filename. The provided filename does not have to match any file on disk. > So, for example:
var embed = new EmbedBuilder()
.WithImageUrl("attachment://myimage.png")
.Build();
await channel.SendFileAsync(stream, "myimage.png", embed: embed);
Alternatively, you can use SendFileAsync with the string filePath overload. Internally, this gets a stream of the file at the path, and sets filename (as sent to Discord) to the last part of the path. So it's equivalent to:
using var stream = File.OpenRead(filePath);
var filename = Path.GetFileName(filePath);
await channel.SendFileAsync(stream, filename);
From here, you can see that if you want to use the string filePath overload, you need to set embed image URI to something like $"attachment://{Path.GetFileName(filePath)}", because the attachment filename must match the one sent to Discord.
I almost had it with my code above, but I misunderstood the intention and usage of the method and property. I guess I thought the .ImageUrl property somehow "automatically" initiated a Stream in the background. Additionally, I missed one very important piece:
As it's an async method, you must await (or whatever the VB.NET equivalent is) on SendFileAsync.
So, after making my calling method into an async method, my code now looks like this:
Private Async Sub TestMessageToDiscord()
Dim Builder As New Discord.EmbedBuilder
Dim AttachmentPath As String = CreateDynamicImage() '<-- Returns the full, local path to the created file
With Builder
.Description = "SAMPLE DESCRIPTION"
.ImageUrl = $"attachment://{IO.Path.GetFileName(AttachmentPath)}"
End With
Using AttachmentStream As IO.Stream = IO.File.OpenRead(AttachmentPath)
Await MyClient.GetGuild(ServerID).GetTextChannel(PostChannelID).SendFileAsync(AttachmentStream, IO.Path.GetFileName(AttachmentPath), "THIS IS A TEST", False, Builder.Build)
End Using
End Sub
Now, everything works exactly as expected and I didn't have to resort to uploading the image to a hosting site and using the new URL (I actually had that working before I got the response on GitHub. I'm sure that code won't go to waste).
EDIT
Okay, so I still ended up going back to my separately hosted image option for one reason: I have a separate event method that modifies the original Embed object during which I want to remove the image and replace the text. However, when that event fired, while the text was replaced, the image was "moved" to the body of the Discord message. While I may have been able to figure out how to get rid of the image entirely, I decided to "drop back and punt" since I had already worked out the hosted image solution.
I've tried everyting I could, but I got stuck at the same point at where you are now.
My guesses are that Discord doesn't like the embedded images from https://cdn.discordapp.com/attachments, and only accepts the new files from https://media.discordapp.net. I might be wrong though, this is the way it worked for me.
I believe it's only a visual glitch, as I found if you send a link for an image from cdn.discordapp.com/attchments in your regular Discord client, it bugs out and shows an empty embed for some reason.
That would make sense since the default link used in an embedded image actually starts with https://cdn.discordapp.com/attachments/...
You could solve this issue by using https://media.discordapp.net, but it seems like Discord.net is configured to use the old domain.
I am trying to implement word level matches in Google Diff Match Patch, but it is beating me up.
The result I get is:
=I've never been =|-a-|=t=|= th=|-e-|=se places=|
=I've never been =|=t=|+o+|= th=|+o+|=se places=|
The result I want is:
=I've never been =|-at these-|= places=|
=I've never been =|+to those+|= places=|
The documentation says:
make a copy of diff_linesToChars and call it diff_linesToWords. Look
for the line that identifies the next line boundary: lineEnd =
text.indexOf('\n', lineStart);
In the c# version, I found the line to change in diff_linesToCharsMunge, which I changed to:
lineEnd = text.Replace(#"/[\n\.,;:]/ g"," ").IndexOf(" ", lineStart);
However, there is no change in granularity -it still finds differences at character level.
I am calling:
List<Diff> differences = diffs.diff_main(linepair.Original, linepair.Corrected, true);
diffs.diff_cleanupSemantic(differences);
I have stepped through to make sure that it is hitting the change I made (incidently, there is a hardcoded minimum of 100 characters before it kicks in).
I have created a sample dotnet project with diffmatch program. Its probably older version of DiffMatchPatch file but the word and lines work.
DiffMatchPatchSample
For your above sample text ,I get below output.
at these | to those
How would I send a message to all of the players on the server? (When trying to do so, it only outputs to the console, and I believe that is because the mod is not installed on the client side.)
I have been trying to make a mod for a 1.7.10 server (To put in a 1.7.10 mod pack) that can message all of the players online. I have looked this question up, and have not found any answers.
#SideOnly(Side.SERVER)
#SubscribeEvent
public void onDeath(PlayerEvent.Clone event)
{
if (event.wasDeath) {
final String[] messages = {"Oh boiss we got a respawner O_O", "How dare ye respawn on me?", "GAAH! You died again!", "._. Just why...", "Was taht me or waas that you? -.-","Why isn't this in hardcore mode? It should be..."};
Random random = new Random();
int index = random.nextInt(messages.length);
ChatComponentText text = new ChatComponentText(messages[index]);
ChatStyle style = new ChatStyle();
style.setColor(EnumChatFormatting.LIGHT_PURPLE);
text.setChatStyle(style);
FMLCommonHandler.instance().getMinecraftServerInstance().getConfigurationManager().sendChatMsg(text);
System.out.println("Respawned");
}
}
I expect that the server will send a message to all, but only outputs to the console.
A really quick way this could be handled is to create an EventHandler for when a player joins. Then add them to an ArrayList. Then when they leave (Check for kick / quit event). Remove them from the ArrayList. By having an arraylist you can run through this and message every player.
Your 'System.out.println("Respawned");` line will only print to the console.
I believe what you are looking to do is the following:
1) Loop through all the players on the server.
2) Send each player the calculated message.
I'm not 100% sure how to access the player-list off the top of my head, but you need to access the FMLServerHandler and get the player-list, OR (the better way) access the EntityPlayer objects connected to the player's current world and do the above steps. The second method would only work for the current world, so if you wanted to access all the connections to the server, the first method is the way to go.
I want to use the Microsoft.Speech namespace in VB.NET to create a telephony application. I need to be able to set the recognizer input to any audio device installed on the system. Microsoft has the recognizer.SetInputToDefaultAudioDevice() method, but I need something like .SetInputToAudioDeviceID. How can I choose another wave audio input from the list of devices installed on my system? In SAPI, I would use MMSystem and SpVoice:
Set MMSysAudioIn1 = New SpMMAudioIn
MMSysAudioIn1.DeviceId = WindowsAudioDeviceID 'set audio input to audio device Id
MMSysAudioIn1.Format.Type = SAFT11kHz8BitMono 'set wave format, change to 8kHz, 16bit mono for other devices
Dim fmt As New SpeechAudioFormatInfo(1000, AudioBitsPerSample.Eight, AudioChannel.Mono)
Recognizer.SetInputToAudioStream(MMSysAudioIN1, fmt)
How do I do this with Microsoft.Speech?
MORE INFO: I want to take any wave input device in the Windows list of wave drivers and us that as input to speech recognition. Specifically, I may have a Dialogic card with wave input reported by TAPI as deviceID 1-4. In SAPI, I can use the SpMMAudioIn class to create a stream and set which device ID is associated with that stream. You can see some of that code above. Can I directly set Recognizer1.SetInputToAudioStream by the device ID of the device like I can in SAPI? Or do I have to create code that reads bytes and uses buffers, etc. Do I have to create a MemoryStream Object? I can't find any example code anywhere. What do I have to check in .NET to get access to ISpeechMMSysAudio/spMMAudioIn in case something like this would work? But hopefully, there is a way to use MemoryStream or something like it that takes a device ID and lets me pass that stream to the recognizer.
NOTE 2: I added "imports Speechlib" to the VB project and then tried to run the following code. It gives the error listed in the comments below about not being able to set the audio stream to a COM object.
Dim sre As New SpeechRecognitionEngine
Dim fmt As New SpeechAudioFormatInfo(8000, AudioBitsPerSample.Sixteen, AudioChannel.Mono)
Dim audiosource As ISpeechMMSysAudio
audiosource = New SpMMAudioIn
audiosource.DeviceId = WindowsAudioDeviceID 'set audio input to audio device Id
' audiosource.Format.Type = SpeechAudioFormatType.SAFT11kHz16BitMono
sre.SetInputToAudioStream(audiosource, fmt) <----- Invalid Cast with COM here
It also appears the SpeechAudioFormatType does not support 8kHz formats. This just gets more and more complicated.
You would use SpeechRecognitionEngine.SetInputToAudioStream. Note that if you're having problems with streaming input, you may need to wrap the stream, as illustrated here.