I'm trying to design a script that requires very fast and precise key events. It needs to time key presses ideally to within 2 or 3 milliseconds max and needs to run the same way every time. I'm wondering what the best way to implement this project is. I'm open to suggestions on languages, project structures, and anything else that might help. Ideally, the script would be able to send key press events to a specific application so I can use other apps while it is running, but I am working on Mac OS which I know is a more challenging environment to implement that sort of functionality.
I know how to implement the basic idea in python and java, but I'm looking for ideas on optimizing speed and accuracy. And the thing about sending the events to a specific application.
Thanks for your help!
Related
I was recently using a python package called OpenPyXl. It's a package that let me control and write to Excel and I'm trying to understand how something like that even works. How can Python and Excel "talk" to each other? I had a few questions regrading this topic, I hadn't been able to find definitive answers to.
From what I understand after reading a bit, Python must be talking to Excel through some API that the developer (in this case Microsoft I guess) made available for use. Is that generally how software which controls/interacts with other software works?
Without an API are you generally unable to interact with software through your own code? Let's say I wanted to write a program that could play the game solitaire on my windows machine, without Microsoft creating a specific API that makes it possible so some code I write can "talk" to Solitaire, would my only option be some sort of program that attempts to actually visually read and comprehend my screen, and control the mouse to implement moves?
I'm just trying to understand how various pieces of software can interact with each other, thus far my only experience has really only been in self contained scripts, and I don't fully understand if every program has some sort of.
If for instance, I had to write a bot to play solitaire, I don't even know where I would begin, and I just want to have a better understanding of how the pieces fit together.
I´m working on a platform to teach how to read/write sheet music.
So far, It´s been "easy"* to figure out a system in which people select the minimal rhytmic units to WRITE some music, but I dont know even how to start to other direction of this process: how to create an input system to provide activities in which students could READ and tap some rhytms on the keyboard. Of course, my objective is to capture the time interval between taps in relation to beats. In other words....how to measure the time between taps.
I know softwares like EarMaster or GNUSolfege figured this out.
Any help would be much appreciated.
Thanks people :)
*"easy" because I´m a pianist/psychologist. All the PHP and JQuery and MySQL recently added to my life are still some hard/exciting things to understand.
Say you have a button. That button will have some method that runs when you tap it. What you will want to do is get the current time, it will depend on your language/platform but there is typically some built-in method. This will be in Unix time, which is the number of seconds that have elapsed since jan 1 1970 (or milliseconds). So each time a user taps the button, get that time (an integer of milliseconds) and store it / match it against the tempo.
This sounds easier if you have a pre-defined tempo. Just convert beats-per-minute into milliseconds-per-beat, and you would have a repeating timer in the app that plays a sound or flashes a color after that many milliseconds.
This is probably an unsatisfying answer, and generally, stack overflow should be used for questions that specifically relate to code. So when you know what language you are using and you start to implement this feature, if you run into problems, try asking this question again with your code / a minimal version of it and you will likely get a better answer.
I have been learning coding for about a month now. I have some good experience with Python, and additionally I have completed this web development course on Udacity.
Now, I have a good foundation for programming, but one thing that confused me a lot is how to interact with various websites and APIs. The course I did briefly touched on this in terms of XML and JSON files and how some webpages offer their pages in these formats for easier reading by other machines.
But there are still a bunch of tasks which I have no idea how to approach whatsoever, but want to eventually do. I have constructed some hypothetical examples for the purpose of this question. I will post my current rough understanding of how I would do them below each one, and would appreciate feedback (on the API interaction, not on the front-end or on any back-end algorithms/AI/parsing):
Creating a phone application (disregarding the front-end part) which can then communicate with and perform rudimentary tasks on my computer.
I have no idea how to do this, and my guess would be that I would have to look into some external application/API meant for this process and implement this on both-ends of the system.
Being able to write a bot which goes on to a game website and controls the object via script. (e.g going onto a pacman game website written in flash and automatically controlling the character to avoid the ghosts)
I don't even know if this is possible, or how browser flash games interact handle the user-server interaction. Would I have to post some data via HTTP manually in the same way that playing in the keyboard would do? Or is everything done client side, in which case how would I fake user input? How would I get information on the ghost's position to work the AI?
Creating a mobile app for my school by allowing users to put their username and password into the app and then having the app automatically log in to the school and fetch certain data (e.g timetable) and return back in a readable form.
I'm guessing that I would take the input from the user on my mobile-app, and then navigate to the school's login page and POST this data in the relevant forms to log in. And then that I would (somehow, not sure), navigate to the timetable URL through my code while still managing to stay logged in, and then parse the html there?
I would appreciate some detail on how these kind of things are done, preferably with reference to these examples, so that I can get a better conceptual understanding.
Thanks!
Note: I have asked all those various questions mostly rhetorically, just so that those reading can get a better understanding of what my current programming level and understanding of APIs is at. I do not expect nor require specific answers for each and every question (so I hope this doesn't get flagged as being too vague or requiring too much detail!), I just would appreciate some responses telling me roughly how each of these APIs work approximately and how I would even start at looking at how to do these things.
You asked too many questions and honestly speaking I am not able to read and grasp entire text posted by you.
So, I am focusing only the title of your question:
"A conceptual understanding of API"
API (Application Programming Interface) means a set of functions which you can directly use by simply passing parameters to them.
Actually, in application development there are many common functions which every application programmer have to use. So, instead of coding them every time by every programmer, they are already coded in functions which you can use simply by passing parameters to them (if they need any external parameter).
Example:
I am offering you a maths API, set of functions {add, sub, mul, div}. You can pass two numbers to any of these four functions and get desired result instead of coding every time for ever operation like add, sub, mul and div.
Hope it helps...
I know this is a pretty open question but I was wondering how people go about writing scripts that will "play" a game, or manipulate it in some way. I had been thinking that I'd try to get a working AI to play a game for fun, but don't even really know where to start. Are there any good resources to learn this? What are good languages to use? Once I have the language, how do I get my script hooked into the game? I was thinking of just trying simple flash games, if that helps.
Thanks a bunch!
Are you looking to "play" an existing game, or write some sort of a game that can then be scripted?
If the former, you find that game's API and start writing code in the appropriate language. E.g. Unreal-powered games use UnrealScript and ship with the tools so you can embed new functionality. You're pretty much dependent on the game creators leaving an "in" such as this...
...unless you want to get totally nutty and hack the executable itself to execute your own code somehow. For information on how to do this, search for information on game "trainers". I'm not sure what a great set of search terms would be, but I know a lot of games do have trainers released, so type in "trainer <my_favorite_game>" and see what you get?
You can do this quite simply in Java 6.
expose your objects/functions in your Java app into the scripting environment
For example you can expose your character or the game world into the scripting environment and allow the script to control it.
invoke script functions/methods from Java environment
Allow the script to customize behaviours, add new actions, etc. Another use of this is to override the default build in game rules and load your own house rules. Another use is as a macro facility. Typically this would be part of the main game loop
use the script as a data/level file
For example, in a game likes SpaceHulk where it is mission based, you can use a script to define a game specific DSL, and us this DSL to define new missions. When the game starts up, it evaluates the data file and create a model of the game.
To integrate a scripting engine into Java, simply do this
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine jsEngine = mgr.getEngineByExtension("js");
jsEngine.eval("print('Hello world!');
To expose an object into the script environment
jsEngine.put("avatar", avatarObject);
//An avatar object is now available in myscript.js
jsEngine.eval(new FileReader("myscript.js");
To invoke a function in the script
jsEngine.eval("function hello(name) ... ");
Invocable inv = (Invocable)jsEngine;
inv.invokeFunction("hello", "Fred");
One of the problem with scripting is that, the script has access to your entire application. You can restrict what the script can 'see' by setting a classloader when you create the ScriptEngineManager instance.
Java 6 comes with a bundled JavaScript engine. However I prefer Groovy and loading Groovy basically mean changing 'js' in getEngineByExtension() to 'groovy'.
I know that a lot of people say that Java is not fast for games, but if you are writing turn based games or social games like Farmville then I think it is more than adequate.
For C/C++ based games, I think the preferred script engine is Lua. I've not used it so I cannot comment on it.
The best way to do that is to use the API provided by the game, if there is one. The Civilization games, for example, come with a very complete modding engine that allows you to script whatever you want. The same can be said for Age of Empires and a variety of other games.
Simple flash games, though? That'll be tough. Those don't generally provide any supported way to script, so you'll be left with simulating input. It'll be tough.
The easy way is to control mouse and keyboard functions with autoit. You can also read out information from your screen, but it is a bit limited It is good to just check if things changed or check whats on a certain area of the screen if you know there are a limited amount of possibilities. Not so easy to read and interpret text. Autoit is excelent to automate games like farmvile. I have also made blackjack bots for online casinos. And use it for small but mundane and repeating (manufacturing) tasks in RPGs. Dont expect to build an AI for an RTS or FPS though.
Autoit is easy to find with google. A fast and small download. Excelent documentation and you can learn it in days.
The hard way is what people wrote about above here i guess. Interacting with the game data. Searching information on that is what brought me here. I see its with C++ or Java. While the languages are all similar and you can use them all after learning 1, these come with complicated software environments that take lots of time to learn. The possibilities and chances to make something really reliable are much much greater though.
One of our next projects is supposed to be a MS Windows based game (written in C#, with a winform GUI and an integrated DirectX display-control) for a customer who wants to give away prizes to the best players. This project is meant to run for a couple of years, with championships, ladders, tournaments, player vs. player-action and so on.
One of the main concerns here is cheating, as a player would benefit dramatically if he was able to - for instance - let a custom made bot play the game for him (more in terms of strategy-decisions than in terms of playing many hours).
So my question is: what technical possibilites do we have to detect bot activity? We can of course track the number of hours played, analyze strategies to detect anomalies and so on, but as far as this question is concerned, I would be more interested in knowing details like
how to detect if another application makes periodical screenshots?
how to detect if another application scans our process memory?
what are good ways to determine whether user input (mouse movement, keyboard input) is human-generated and not automated?
is it possible to detect if another application requests informations about controls in our application (position of controls etc)?
what other ways exist in which a cheater could gather informations about the current game state, feed those to a bot and send the determined actions back to the client?
Your feedback is highly appreciated!
I wrote d2botnet, a .net diablo 2 automation engine a while back, and something you can add to your list of things to watch out for are malformed /invalid/forged packets. I assume this game will communicate over TCP. Packet sniffing and forging are usually the first way games (online anyways) are automated. I know blizzard would detect malformed packets, somehting i tried to stay away from doing in d2botnet.
So make sure you detect invalid packets. Encrypt them. Hash them. do somethign to make sure they are valid. If you think about it, if someone can know exactly what every packet means that is sent back and forth they dont even need to run the client software, which then makes any process based detection a moot point. So you can also add in some sort of packet based challenge response that your cleint must know how to respond to.
Just an idea what if the 'cheater' runs your software in a virtual machine (like vmware) and makes screenshots of that window? I doubt you can defend against that.
You obviously can't defend against the 'analog gap', e.g. the cheater's system makes external screenshots with a high quality camera - I guess it's only a theoretical issue.
Maybe you should investigate chess sites. There is a lot of money in chess, they don't like bots either - maybe they have come up with a solution already.
The best protection against automation is to not have tasks that require grinding.
That being said, the best way to detect automation is to actively engage the user and require periodic CAPTCHA-like tests (except without the image and so forth). I'd recommend utilizing a database of several thousand simple one-off questions that get posed to the user every so often.
However, based on your question, I'd say your best bet is to not implement the anti-automation features in C#. You stand very little chance of detecting well-written hacks/bots from within managed code, especially when all the hacker has to do is simply go into ring0 to avoid detection via any standard method. I'd recommend a Warden-like approach (download-able module that you can update whenever you feel like) combined with a Kernel-Mode Driver that hooks all of the windows API functions and watches them for "inappropriate" calls. Note, however, that you're going to run into a lot of false positives, so you need to not base your banning system on your automated data. Always have a human look over it before banning.
A common method of listening to keyboard and mouse input in an application is setting a windows hook using SetWindowsHookEx.
Vendors usually try to protect their software during installation so that hacker won't automate and crack/find a serial for their application.
Google the term: "Key Loggers"...
Here's an article that describes the problem and methods to prevent it.
I have no deeper understanding on how PunkBuster and such softwar works, but this is the way I'd go:
Iintercept calls to the API functions that handle the memory stuff like ReadProcessMemory, WriteProcessMemory and so on.
You'd detect if your process is involved in the call, log it, and trampoline the call back to the original function.
This should work for the screenshot taking too, but you might want to intercept the BitBlt function.
Here's a basic tutorial concerning the function interception:
Intercepting System API Calls
You should look into what goes into Punkbuster, Valve Anti-Cheat, and some other anti-cheat stuff for some pointers.
Edit: What I mean is, look into how they do it; how they detect that stuff.
I don't know the technical details, but Intenet Chess Club's BlitzIn program seems to have integrated program switching detection. That's of course for detecting people running a chess engine on the side and not directly applicable to your case, but you may be able to extrapolate the apporach to something like if process X takes more than Z% CPU time the next Y cycles, it's probably a bot running.
That in addition to a "you must not run anything else while playing the game to be eligible for prizes" as part of the contest rules might work.
Also, a draconian "we might decide in any time for any reason that you have been using a bot and disqualify you" rule also helps with the heuristic approach above (used in prized ICC chess tournaments).
All these questions are easily solved by the rule 1 above:
* how to detect if another application makes periodical screenshots?
* how to detect if another application scans our process memory?
* what are good ways to determine whether user input (mouse movement, keyboard input) is human-generated and not automated?
* is it possible to detect if another application requests informations about controls in our application (position of controls etc)?
I think a good way to make harder the problem to the crackers is to have the only authoritative copies of the game state in your servers, only sending to and receiving updates from the clients, that way you can embed in the communication protocol itself client validation (that it hasn't been cracked and thus the detection rules are still in place). That, and actively monitoring for new weird behavior found might get you close to where you want to be.