I like to make an overlay with the following properties:
should work at least on Windows 8.1
should be on top on everything, like a mouse cursor
should incorporate the pixels which are already on the background, like a blur filter
no flickering
Details to each of this points:
1) I assume that WDM is activated and DirectX 11.2 is used. Sure it would be nice to have it working on other Windows versions but this has no priority.
2) The problem is that with simply using the WS_EX_TOPMOST, menus from applications are over my overlay. In my case this really hurts as I like to display something with the same properties as a cursor. Imagine that a cursor suddenly is hidden if you open a menu -> unacceptable.
3) I like to read the pixels from the Windows desktop, including any effect Windows applies (like blur), and use this information for my filter. If I add my overlay, as described in 2, I should be able to get a fresh unobstructed copy of the background in the next frame and not read out my own overlay.
4) If I just write something into the Windows desktop directly, it gets overwritten immediately on the next frame by Windows itself. This is not acceptable.
One example of such an application is a magnifying glass, which exactly has all the properties I need. But for this case Windows 8.1 has an API. In contrast I like to write a program which displays a hand on the desktop (which is controlled by Leap Motion) which influences the Windows desktop, so you almost "feel" how you move your hand over the desktop.
If I write a tiny DirectX and/or OpenGL application for myself this is all very easy:
render all the regular stuff to a texture
use this texture for a post processing filter and add all my stuff on top of it
render just a quad to the back buffer
But I like to do that for the whole Windows desktop.
I found many different application, but they are to no use for me:
application which claim to be on top, are still behind menus. This normally doesn't really hurt, but is unacceptable for a cursor-alike thing
screen capturing programs which hook them self in all running programs are nice, but I want to hook myself into WDM
normally screen capturing programs do not draw anything into the back buffer, so they get every frame a new unobstructed back buffer
My questions can be boiled down to: How can I write my own magnifying glass for Windows 8.1.
I fear that my only serious option is to hook myself into WDM, what I try to avoid.
I'm happy to hear any idea how to achieve this, or hints to application which are doing what I describe.
Related
When screen sharing a specific window on macOS with Zoom or Skype/Teams, they draw a red or green highlight border around that window (which belongs to a different application) to indicate it is being shared. The border is following the target window in real time, with resizing, z-order changes etc.
See example:
What macOS APIs and techniques might be used to achieve this effect?
You can find the location of windows using CGWindowListCopyWindowInfo and related API, which is available to Sandboxed apps.
This is a very fast and efficient API, fast enough to be polled. The SonOfGrab sample code is great platform to try out this stuff.
You can also install a global event tap using +[NSEvent addGlobalMonitorForEventsMatchingMask:handler:] (available in sandbox) to track mouse down, drag and mouse up events and then you can respond immediately whenever the user starts or releases a drag. This way your response will be snappy.
(Drawing a border would be done by creating your own transparent window, slightly larger than, and at the same window layer as, the window you are tracking. And then simply draw a pretty green box into it. I'm not exactly sure about setting the z-order. The details of this part would be best as a separate question.)
As the title said, I want to move the screen up, so that my app's window can come in from the bottom. What I currently have now is a floating window, whose background has been set to a screenshot of the screen. Like this:
It looks fine to the user, except now any objects that work in the background don't appear above the window, it's basically frozen.
Can I do this? This effect is similar to what Notification Center does in 10.8.
You can do this, for a certain definition of "can" - there's no API for it, but you can likely use the same SPI that the Finder/Dock/etc uses. The only complication may lie in needing special privileges, or needing your code to be specially signed - I'm not sure what checks are in place.
It's not too tricky to figure this out; you can use tools like nm, otool and even class-dump.
In the Windows 8 Consumer Preview, moving the mouse towards the left or right edge in the start screen causes the content to scroll.
The standard controls (and currently released preview apps) does not seem to support this.
Is there a way to make this work?
I asked this question at TechEd North America this year, after one of the sessions given by Paul Gusmorino, a lead program manager for the UI platform.
His answer was that no, apps can't do push-against-the-edge-to-scroll. WinJS and WinRT/XAML apps don't even get the events you would need to implement it yourself. Apps get events at the level of the mouse pointer, and once the mouse pointer hits the edge of the screen, it can't move any farther and you don't get any more events. (Well, it might wiggle up and down a little bit, but not if it hit a corner. At any rate, it's not good enough to scroll the way the Start screen does.)
He mentioned that, if you were writing a C++/DirectX app, you would be able to get the raw mouse input you needed to do this yourself -- you can get low-level "device moved by DX,DY" rather than the high-level "pointer moved to X,Y". I'm guessing this is how the Start screen does it, though I didn't think to ask.
But no, it's not built-in, it's not something you can implement yourself (unless you write your app in low-level C++/DirectX), and it sounds like they have no plans to add it before Windows 8 ships.
Personally, I think it's pretty short-sighted of them to have apps feel cripped compared to the Start screen, but evidently they're not concerned about little things like usability. </rant>
You can do the following to get information on mouse moving beyond the screen and use the delta information to scroll your content.
using Windows.Devices.Input;
var mouse = MouseDevice.GetForCurrentView();
mouse.MouseMoved += mouse_MouseMoved;
private void mouse_MouseMoved(MouseDevice sender, MouseEventArgs args)
{
tb.Text = args.MouseDelta.X.ToString();
}
I'm making... well, it's sort of like a game in VB.net (using VS2008, if it matters). Being sort of like a game, it's dependent on timing, but it also has a large area (have to scroll the window on pretty much any resolution).
The problem I'm having is that whenever I scroll the window (or move any of the smaller, additional windows) the program pauses what it's doing, and doesn't start again until I stop scrolling (or moving windows, or whatever).
I hate to sound picky, but the program keeping track of how much time it's lost and making it up for it when you're finished scrolling isn't going to cut it.
So is there any way to stop it pausing when you scroll or move windows? Thanks for your answers.
Essentially, you can’t (reliably). Don’t use scroll controls, or in fact any controls at all. It’s a common beginners’ error to use multiple controls in a game.
Render all your game in one PictureBox control and scroll that using e.g. the ScrollDC Windows API function, or by building up the graphic incrementally as the user scrolls. This is more difficult but also usually more efficient since you only need to draw a smaller area (i.e. the visible area, and nothing more).
If that’s not fast enough, chances are that GDI+ graphics won’t cut it and you need to resort to DirectX/XNA.
How would I go about programming a HUD type overlay in OSX.
I want to be able to have an application that will display text at a certain point over a different application's window.
And thus if the (other applications) window moves the HUD part will stay at the same coordinate of the other window.
For the window itself, use a borderless, transparent window (plenty of examples) with your own custom view into which to draw your overlaid elements.
For the "other applications' windows" part, there's no public API that's going to let you do this smoothly. You use Universal Access and its window location/navigation API, but it requires your users to turn on "Enable access for assistive devices" (I think it still can't be done programmatically). I don't believe it "lets you know" when a window moves, but I could be wrong. If it does, it'd likely be a one-shot "here's where I am now", so your overlay would likely not keep up. I also don't think it gives you the "window level" to allow you to make sure you're "above" any given window/sheet/palette.
The only other option (to move with other apps' windows) is a system-wide, invasive hack a la Application Enhancer (which is quite controversial). It's easy to get this wrong and destabilize a user's system (hence the controversy).
You could use undocumented CoreGraphics Functions in order to track a window, see http://code.google.com/p/undocumented-goodness/source/browse/trunk/CoreGraphics/CGSPrivate.h