I'm interested in PC firmware programming, and am just studying the UEFI spec. To my surprise, it seems like a spec for an entire OS which is embedded in firmware. You can even write UEFI "applications", which run directly using the UEFI boot services, without any other OS present.
I've found blog posts which show how to create a "Hello, world!" application which can run in the UEFI preboot environment. This is... interesting, and bizarre at the same time. I'll run my "Hello, world" programs on a regular OS, thank you.
What kind of use cases are UEFI applications actually good for? Fancy boot configuration screens? Does any "real", commercially available PC firmware use UEFI applications to implement anything more than just boot loaders and boot configuration utilities?
Anything that isn't PEI/DXE/SMM core or driver is an application, so any "real" PC have them, because BIOS Setup is actually an UEFI application. Some vendors include various other apps like firmware updaters, diagnostic and troubleshooting utilities, etc. UEFI 2.4 makes possible to add your own application with a properly filled BootXXXX/KeyXXXX variable pair and then run it by pressing a key combination during POST.
Most console applications written in C can be compiled as UEFI application by using StdLib package of current EFI Development Kit and then run in UEFI shell.
Major examples of useful UEFI apps (besides bootloaders, shell and Linux kernel, of course) are Intel ME System Tools, Read Universal, Python 2.7 and many more.
Eventually, when legacy boot will not be available anymore, all currently useful DOS utilities must either be made UEFI applications or go extinct.
Despite many valuable answers here, because I wrote couple UEFI applications myself I will try to add my 2 cents. First, what is UEFI application to just give ground what we talking about:
UEFI Specificatin v2.5:
Section 2.1.1
The major differences between image types are the memory type that the firmware
will load the image into, and the action taken when the image’s entry point exits or returns. An
application image is always unloaded when control is returned from the image’s entry point.
Section 2.1.2
When the application returns from
its entry point, or when it calls the Boot Service EFI_BOOT_SERVICES.Exit(), the application
is unloaded from memory and control is returned to the UEFI component that loaded the application.
Groups of applications that make sense in UEFI:
Configuration tools - Configuration interface for Option ROMs (ie. for storage controllers), out of band management (ie. AMT configuration tools), manufacturer performance tweaking tools
Provisioning tools - used by administrators to preload specific BIOS setting, manually setting all options in BIOS setup would be inefficient
Diagnostics tools - mostly for tests that cannot be performed in OS (DRAM tests, full storage scan, storage R/W tests, etc.). In some districts specific diagnostics tools are required in UEFI BIOS, so those can be sold to government.
Security applications - HDD encryption/decryption, antivirus scanner and anti thief applications
BIOS capability enhancement - Power Over Ethernet extensions, DRAM discovery, patching and modification of system tables (SMBIOS, ACPI)
Display tools - for displaying complex animations while running, splash screen displaying
Bootloaders - this is special type of application, which can call EFI_BOOT_SERVICES.ExitBootServices() causing termination of all memory management and passing control to Operating System.
Note that very important feature of UEFI application is that it can be added to boot order and be executed each boot time. Also UEFI application do not have to be delivered with BIOS image it can be stored in connected device memory, which is common for Option ROM configuration tools.
Here it is an example of a full blown UEFI Pre-boot Application;
There are SED SSD/HDD drives. As soon as SSD/HDD loses its power it goes into locked state (hardware-based encryption) There is no way you can get access to drive's data and all partitions on the drive are no longer even visible. Only small read-only partition (ShadowMBR) is available. UEFI firmware boots an UEFI application from that only available partition (UEFI app is written on that partition during the initialization process and when the ownership of an SED is taken). It securely authenticate user and if credentials are valid it unlocks the drive. When the drive is unlocked Shadow MBR disappears and all partitions on the drive becomes available. Then the App chain-boots the installed OS.
So if you don't have credentials you cannot even boot the OS and you cannot access the data on the drive by any means.
Here's a couple of examples:
https://github.com/NikolajSchlej/CrScreenshotDxe
UEFI DXE driver to take screenshots from GOP-compatible graphic console (yes, you can make PNG screenshots of your BIOS and save them)
http://ruexe.blogspot.com/
RU.EFI is quite an advanced tool for debugging the BIOS
Well, there are the OS loaders - both the more heavyweight ones (Windows, GRUB, BSD Loader) and the "present a menu" ones (rEFInd, Gummiboot). Shim, which enables UEFI Secure Boot for Linux platforms, consists of an application as well as installing a protocol for use by other applications.
Then you have things like the Linux kernel, which when compiled with CONFIG_EFI_STUB becomes a valid UEFI application, with the awareness of booting itself.
And firmware updates can also be shipped as UEFI applications.
The UEFI shell itself is an application.
Then there are things like factory production testing utilities, development diagnosis tools, ...
Windows 7 - 8 have UEFI installer. I'm not fully aware of the details, but I'm pretty sure this new environment gives a lot more flexibility to the developers than traditional boot environment on DVD.
Some motherboards have "instant on" features that allows you to get to a desktop screen within a few seconds. This is usually a stripped down flavor of some linux that allows you to access a web browser and play music/video. ASUS have such boards.
Related
due to the discontinuation of the SL-Thread stack by Silicon Labs, we are considering migrating to OT for a new device in a system that is already based on EFR32s.
The device would be a relatively simple, mains powered FTD (think "range extender").
I am trying to estimate what the effort would be and in particular I am a bit worried about OTA firmware updates.
Looking at the EFR directory in the GitHub repo, I see:
no trace of the Gecko bootloader. Does this imply that we should use a regular Gecko bootloader build from the SL SDK? Or is there a OT-specific bootloader that I'm missing?
no trace of an OTA protocol (in SL'Thread there used to be a TFTP implementation and an dotdot implementation)
Is there any plan to have a OpenThread specific OTA method? Or is the official advice to use the GeckoBootloader and implement your own transfer protocol?
Thanks in advance,
Matteo
Jonathan Hui, tech lead for OpenThread, replied in the Google Group.
Quoting him: "The primary goal of the OpenThread project is to implement the Thread protocol. Given that Thread is a network-layer technology, it does not specify a protocol for OTA. Similarly, the OpenThread project does not include bootloaders and OTA in its scope."
I have lots of experience with the efr32 line of microcontrollers and, incidentally, I'm also attempting to port our Silicon Labs Thread implementations to OpenThread. So I think I can help you here...
If you only need a powered (assuming non-sleepy) full thread device this may not be too much of a problem. You can build the demo projects from the command line (either in Linux or Windows Subsystem for Linux) and just change one of the demo projects to meet your needs.
If you need an IDE build for advanced debugging functionality or just general usability this is much more of a challenge.
The bootloader may be a bit of an issue though. OpenThread is only designed to implement the Thread protocol and is generic so it can be used in a myriad of applications. You won't see any of the Gecko Bootloader stuff there, or any bootloader stuff for that matter.
The Gecko bootloader is essentially an application that runs after a microcontroller reset which looks for a Gecko bootloader image (.gbl) either in external flash or internal flash. If it finds an image it will overwrite the existing application and boot into it. If it doesn't find an image it will just jump to the application. So you're bootloader challenge is 2-fold.
1) The application has to put the .gbl image into the memory location the Gecko bootloader expects. There isn't a standard way to to this as far as I know, it's up to the application. My application, for instance, repeatedly polls a CoAP endpoint to grab chunks of the .gbl image to put in external flash (which is where my bootloader expects the image). Then when it gets the whole image it reboots.
The demo project for the efr32 boards in openthread use a linker script that doesn't include space for the Gecko Bootloader. You'll have to reverse engineer the linking process used in the Simplicity Studio build. It uses 2 linker files as well as some #defines to put the application in the correct location. Depending on your comfort level with the build process this may or may not be difficult.
A note of caution: If you need an migrate an existing Silicon Labs Thread stack based project to an OpenThread project you're in for a beast of a project. The Silicon Labs Thread stack integrates a simple non-rtos based scheduler and sleep functionality. OpenThread, even the efr32 demo projects, contain none of this. Because the Silicon Labs Thread stack is closed source, it makes this much much more difficult. I'm attempting to do this right now an it is very unpleasant. I wouldn't wish it on my worse enemy.
I know that executable files (eg: .exe for Windows) are binaries. I know about hex files and assembly and so on. I also know about OS API's. Theoretically, I could write a web browser entirely in assembly code that uses the OS API's, assemble it with NASM and get an executable. But my question is, how do operating systems control the applications? For example, I could have a executable on Windows that writes to the video memory, and fills the screen with random stuff. I've tried this (actually) and Windows halts the application. How does it control the application? Moreover, if I have a linux executable and I attempt to run it on a Windows machine with the exact same hardware, theoretically, it should work (though it won't use any Windows API's) since the processor architecture is the same, but yet It cant. How exactly does the OS 'control' the binaries?
The operating system, in particular windows and linux protect other applications and the operating system itself from applications. So there is a protection layer that an application ideally cannot punch through running as a user.
You want to put pixels on the screen, you have to ask the operating system to do it for you. If the operating system allows a way to punch through (calling mmap perhaps having to run as root) then you can trash the computer at will, yes. The plan/design is to not let you have direct access, you are given a virtual space, a sandbox, for your application to play in, you have a virtual address space with some ram you can read and write at will, but you go outside that, you need to be shut down or at least handled in some way.
As far as what language you write applications in yes, if it is a normal windows program or linux program, then anything you can do to generate machine code for that target, asm, ada, C, etc. If you are talking specific virtual machines (java, python) then your choices get more limited. If you want to do more than play in your space, like have some sort of output other than a return value, then you have to make system calls in the way that the system requires them, which is target and operating system specific. And again, you get the right instructions or registers or memory structure or whatever is required by the operating system for that call in whatever language you are using or libraries you can link with then the rest of your program can be in whatever language, yes absolutely you want to write a web browser in assembly language using nasm, completely doable. have to create a binary format the os supports, and make the system calls as needed in the way that the system requires.
Currently, IntelliJ IDEA does not have a "Remote Development" feature.
Lets say I have two machines: Machine 1 (very good configuration ex 64GB ram with Intel Xeon processors) and Machine 2 (Macbook Pro with 8GB RAM).
Lets say I have IntelliJ IDEA installed on both machines. The problem now is, there is no client-server mode for the IDE. The closes thing I have is to use OpenNX.
What I'm looking for is a plugin/feature that enable remote development. What I mean by this is: On my macbook, I should be able to add Machine 1 as a "server". And once that is done, the IntelliJ IDEA on my macbook will only act as a client for the IntelliJ IDEA on my Linux box. Basically it would be replicating the UI. However, the catch here is that, it shouldn't do so by sending images (the way any VNC or NX client would). Instead, since it is for a specific application most of the data can be managed through text data only.
Since OpenNX uses images, even with compression it wouldn't match up the performance of text only transmission.
Basically I'm looking for IDEA on one machine to be a client (Remote GUI) for IDEA on another machine.
UPDATE
The eventual answer is: This is not possible (As of now). While I was aware of other options, that wasn't what I really wanted. However, it appears there is no such option.
The main reason why I wanted the option was because my desktop (remote Linux box) has a much higher configuration (Intel Xeon 2GHz processor and 64GB RAM) and my client was an Macbook Pro with Intel Core i7 and 8GB RAM. (By no means any less). However, due to the size of my codebase etc, the indexing of the code etc by the IDE slows it down.
Both client and server are perfectly capable of running an IDE by themselves. However, due the size of the code base it would be better to have the build of the work being done by the IDE on the server and the client being just the front end to it.
The other solutions like VNC, Nomachine - OpenNx all use image compression. And when your client is a Mac, you run into keyboard mapping problems. A client-server mode in the IDE itself would use text compression instead and would be much faster. It would also solve the keyboard mapping problems.
While to me, it sounds like a good idea, it probably doesn't get used by enough people for it to be a feature of the IDE.
Note: I would also be open to considering Eclipse as the IDE if this feature is available. Any answers will always be apreciated.
You'd probably be better off switching instead to a remote code repository that you keep in sync. While the concept of doing this in an IDE plugin is interesting, it has some fundamental flaws. What happens when the machines can't talk to each other? Are you unable to work at that point, or can you work offline. If you work offline on both machines, how do you reconcile changes...
I suggest looking into using "git". You can set up a remote repository very easily. If you have ssh access to either machine or some other shared machine, you can create a remote repository on that machine, and your "client" machines can easily push files/changes around.
There are plenty of other code repository options, but I've found git the easiest to set up.
I know how to change boot order through bios settings, but I have a unique situation where doing it programatically would be better. The company I work for sells and supports software remotely to thousands of non tech savvy customers. We can't touch their hardware settings, we are only the software vendor.
Recently we rolled out a option for their PCI compliancy that requires a separate removable drive to store a private encryption key. Customers that use this option have to leave a usb drive with a .dat file containing the RSA key at all times. Currently this presents an issue when customers reboot. Sometimes we can walk them through over the phone how to change their bios settings to skip the USB drive, but in many circumstances we cannot, because of the caller and the other end of the phone not being tech savvy enough to change bios settings, and different PC's having different BIOS setups.
So my question is, is there any kind of ini file I can create or boot record on the disk itself than can be added or changed to cause the system to see that there is no OS on the USB, keep going down the list of boot drives. Instead, with no OS many PC's hang on Missing OS screen until we have customer remove drive reboot and plug back in after Windows starts to load. All PC's are Windows, all XP or newer.
You're talking about manipulating BIOS Setup data. Unfortunately there is no industry-standard for computers to manipulate Setup fields, like the boot sequence, so any solution is likely to be vendor-specific.
An example: Dell Inc. provides customers OpenManage Client Instrumentation (OMCI) that allows admins to remotely change settings, like boot sequence, via standard interfaces like CIM/WMI. See this whitepaper:
http://www.dell.com/downloads/global/solutions/omci_info.pdf
Especially:
OMCI is the Dell instrumentation package that enables OptiPlex™, Dell
Precision™, and Latitude™ systems to be managed remotely. OMCI
contains the underlying driver set that collects system information
from a number of different sources on the client computer, including
the BIOS, CMOS, System Management BIOS (SMBIOS), System Management
Interface (SMI), operating system, APIs, DLLs, and registry settings.
OMCI exposes that information through the CIMOM interface of the WMI
stack. Thus, OMCI enables IT administrators to remotely collect asset
information, modify CMOS settings, ...
OMCI is specific to the Dell BIOS, so it won't work with other vendors' machines. Other enterprise hardware vendors (e.g. HP, IBM) provide similar software. If you can live with a vendor-specific solution, this may work for you.
May I ask if your USB is actually non-bootable?
How did you format it? FAT32, NTFS etc?
Why can't Windows bypass this usb when booting normally when (i assume) it is not bootable. Normally my system boots to Windows OS even if there is a non-bootable usb plugged in.
BTW have you tried keeping the DAT file on usb as hidden, read only, it's worth a try.
Is there anyway that you can expose local partition or disk image through your computer usb to another computer to appear like external drive on mac/linux/bsd system ?
I'm trying to play with something like kernel development and I need one system for compiling and other for restarting/testing.
With USB: Not a chance. USB is unidirectional, and your development system has no way of emulating a mass storage device, or any kind of other USB device.
With Firewire: Theoretically. (This is what Apple's target disk mode is using.) However, I can't find a readily available solution for that.
I'd advice you to try either virtualization or network boot. VirtualBox is free and open software, and has a variety of command line options, which means it can be scripted. Network boot takes a little effort to set up, but can work really well.
Yet another option, is to use a minimal Linux distribution as a bootstrap which sets up the environment you want, and then uses kexec to launch your kernel, possibly with GRUB as an intermediary step.
What kind of kernel are you fiddling with? If it's your own code, will the kernel operate in real or protected mode? Do you strictly need disk access, or do you just want to boot the actual kernel?