What are the factors that decide to use the camera api version, is it related to the hardware, and if it is related to the hardware, what is it?
Related
Being a novice I need an advice how to solve the following problem.
Say, with photogrammetry I have obtained a point cloud of the part of my room. Then I upload this point cloud to an android phone and I want it to track its camera pose relatively to this point cloud in real time.
As far as I know there can be problems with different cameras' (simple camera or another phone camera VS my phone camera) intrinsics that can affect the presision of localisation, right?
Actually, it's supposed to be an AR-app, so I've tried existing SDKs - vuforia, wikitude, placenote (haven't tried arcore yet cause my device highly likely won't support it). The problem is they all use their own clouds for their services and I don't want to depend on them. Ideally, it's my own PC where I perform 3d reconstruction and from where my phone downloads a point cloud.
Do I need a SLAM (with IMU fusion) or VIO on my phone, don't I? Are there any ready-to-go implementations within libs like ARtoolKit or, maybe, PCL? Will any existing SLAM catch up a map, reconstructed with other algorithms or should I use one and only SLAM for both mapping and localization?
So, the main question is how to do everything arcore and vuforia does without using third party servers. (I suspect the answer is to device the same underlay which vuforia and other SDKs use to employ all available hardware..)
As the question says ,I want to monitor the value of power(watts) that some components consumption .especially the value of CPU , Memory and disk .
when I use aida64，I found that in computer/sensor ,there are some data about power consumption . I want to know how did it can get these data ?
I already have some idea ,but not sure which is the best way to solve this question :
there are some sensors on the motherboard ,we can use values of those sensors to calculate the real-time power.
according to different OS, we have some APIs that can get the utilization of cpu,memory throughout rate and disk I/O rate . Using this data ,we can build mode of power consumption about PC.if there are those APIs,where can I find them ?
maybe the hardware manufacturer like intel has already record the value of power in real-time ,they put the value into some special register in hardware .we can get the value through mapping into special memory location .
In my opinion ,the second way maybe the solution that most monitor software using .but I just don't know where can I get those API.
whats more ,our aim is to design an OS-independent real-time power monitor software. So, if there are any better solutions about this question ,I will appreciate your help .
Hmmm. I wasn't sure if I should post this as a comment or an answer. It is an answer but in the negative.
At this time, you can't create an OS independent software-based non-intrusive power monitor. By non-intrusive, I mean that you are not putting special instrumentation on the motherboard and other hardware. This is because the power technology being used by modern processors is in rapid flux, each new generation making significant advances. Additionally, the amount of power related information available to software from the hardware (via PMU events and the like) is continually increasing as more silicon real estate becomes available. For example, I believe that in the most current processors, you can get direct thermal information for key parts of the processor silicon, and temperature, power and current readings from various parts of the core and uncore.
The best you can do is to abstract the top layer of your monitor from the lower layers. Then the top becomes OS / HW independent while the lower levels need to be platform dependent.
Check out the PAPI APIs. Note that the APIs appear to give you the world, but are really just an API set. Someone still has to implement what's on the other side of the API.
Now if you can do your own special instrumentation, many (most?) motherboards and other hardware have measurement points (some undocumented) that provide thermal, current (and so power) information. This information is important for debugging devices and platforms.
I try to understand how they both relate to each other. As far as I know, they both can be a part of the HAL. In case of a communication between an application and a graphics card - can an API get the job done on its own or do we have to rely on them both? Can an API directly communicate with the hardware or do we always need a driver in-between, which translates the command of the API?
TL;DR
Think of an API as a specification that describes what to do, while a driver is an implementation that describes how to do it.
Details
As a contrived example, imagine we have three different audio cards that we want to play nicely with multiple operating systems. We can define an API for the card manufacturers that says, "Each card must support four methods: mute(), playsound(sound), volumeup() and volumedown()". By defining the API, we get a common interface that allows the operating system designers to support the audio devices without worrying about the hardware details. They know that if they want to mute the sound card, they can call mute(), or if they want to turn the volume up, they call volumeup().
It is then up to the device manufacturers to implement a driver that actually performs those actions. The driver will vary between the three different audio cards because they are different at the hardware level, but the API is consistent so the next higher abstraction level (the OS) doesn't need to know how to deal with the hardware.
For a more concrete example, consider the Advanced Control & Power Interface (ACPI) specification. It defines a common interface for operating systems to manage power consumption and thermal characteristics of hardware devices. There are methods that a device driver or firmware must implement in order to be "ACPI Compliant". This allows Windows operating systems and Linux variants to both perform the same actions on hardware devices without needing to implement their own drivers for the hardware
Note: Windows performs ACPI actions through acpi.sys, which they call an "ACPI Driver". Don't let the terminology confuse you; even though they call it a driver, it is really a window into the ACPI interface. Linux uses the acpi kernel module to do the same thing, and Linux doesn't call it a driver. Perhaps ACPI wasn't the best example, but I don't have anything better at the moment.
As firewire cameras are becoming obsolete because of their bandwidth limitations, it seems as though camera manufacturers are switching to USB 3.0 or Gigabit Ethernet interfaces. Both have standards USB3 Vision and GigE Vision, which many manufacturers are adhering to.
However, it seems as though each manufacturer - Basler, Pointgrey, Ximia, and others - has its own SDK for interfacing with their cameras. When developing an application, developers would need to learn and interface with each API which is a pain, or stick to one manufacturer. I may be misunderstood but, in that case, what is the point of an industry standard if developers need to use manufacturer dependent APIs?
For firewire cameras, developers have access to libdc1394 cross-platform, high level API. They do not need to worry about who manufactures the camera and do not have to write separate drivers. Is something like this even possible for USB3 Vision and GigE Vision? If so, who would develop it?
At least for GigEVision, let me mention the Aravis project is available for linux. It is meant to be a GenTL/GenICam library but only supports GigE right now due to the driver-constraint problems outlined below.
First of all, I agree with Martin's point that creating a general SDK is not in the interest of the camera manufacturers themselves for competitive and support reasons. The manufacturers develop proprietary usb drivers (for USB3Vision) and NIC filter drivers (optional for GigE but highly recommended) in conjunction with their SDKs. It incentivizes them to lock in users to their ecosystem and to separate them from the competition.
This is the reason why I disagree with AdamF - I do not think that GenTL is widely supported by camera manufacturers, particularly for GigE or USB3Vision camera. Supporting GenTL would effectively allow users to use any general purpose SDK while still leveraging the manufacturer's proprietary drivers
I think it would be easier for OpenCV to support GenTL instead of GigE/U3V at this point because the giant hurdle to develop GigE/U3V drivers across the available hardware platforms. GenTL support would at least only be a software-based interface at this point.
I'm not very familiar with libdc1394, but I know a little bit most of all other interfaces.
USB3 Vision, GigE Vision and all other standards may be connected using one common interface: GenICam :
The goal of GenICamTM is to provide a generic programming interface
for all kinds of cameras and devices. No matter what interface
technology (GigE Vision, USB3 Vision, CoaXPress, Camera Link HS,
Camera Link, 1394 DCAM, etc.) they are using or what features they are
implementing, the application programming interface (API) should be
always the same.
The GenICamTM standard consists of multiple modules according to the
main tasks to be solved:
GenApi: configuring the camera.
Standard
Feature Naming Convention (SFNC): standardized names and types for
common device features. Includes Pixel Format Naming Convention
(PFNC).
GenTL: transport layer interface, grabbing images.
CLProtocol: GenICam for Camera Link.
GenCP: generic control protocol.
GenTL SFNC: recommended names and types for transport layer interface.
Most of the biggest camera producers supplies GenTL providers to work with their cameras.
Unfortunately I don't know any open source High Level Api for GenICam. I know 2 image processing libraries with GenICam support: Adaptive Vision Library and Halcon but they are not cost free.
Another less popular in industry common image grabbing interface is: DirectShow.
DirectShow is supported for example by: Ximea, Net-Gmbh, Basler and almost all web cameras.
So in my opinion if you want to use one common interface for all cameras you should consider using GenICam interface.
Check out https://github.com/ni/usb3vision
It implements the core USB3 Vision specification as a kernel driver. To control a camera, you would still need to wrap some usermode logic around it that connects it up to GenApi (the reference implementation of GenICam) as well as handles buffers queued/de-queued to the driver.
Also, regarding your question about if it is possible to implement a vendor-independent driver, of course it is. That is indeed the point of the standards. Most camera vendors provide their own proprietary SDK with their cameras for various reasons, but there are independent SDKs that will work with any standards-compliant GigE Vision and USB3 Vision cameras. Whether any of these are open-source is a good question, and I am not aware of any that are. The above-mentioned USB3 Vision driver is used by National Instruments's IMAQdx driver, which is commercial and closed source.
An old thread, but in case someone else comes looking...
Plus 1 for Aravis for opensource and in Linux. At the time of me writing this response, the project is now supporting USB3 Vision cameras although some are better than others. There is a lot of activity on the repo at Github at present
On the paid side of things (In windows at least) there is an API called ActiveUSB (for USB3 cams) and ActiveGigE by A&B Software. I've no experience with the GigE software, but have used the USB3 vision library that they provide and it is quite good across different cameras as long as they adhere to the GeniCam standard. It also offers a trial period allowing you to decide if its right for you or not. It is useable in Python, C, C# & VB languages. If you are developing for a commercial product/ solution then its worth taking a look at. On the other hand, if you don't want or can't afford to spend any money, then Aravis is the way to go although.
Its also worth noting that some manufacturers are starting to provide demos written in Python that can be used to create your own API. As already mentioned, this is limited for use with the manufacturers cameras and not easily interchangeable unless you have good code writing skills.
I wanted to know what steps one would need to take to "hack" a camera's firmware to add/change features, specifically cameras of Canon or Olympus make.
I can understand this is an involved topic, but a general outline of the steps and what I issues I should keep an eye out for would be appreciated.
I presume the first step is to take the firmware, load it into a decompiler (any recommendations?) and examine the contents. I admit I've never decompiled code before, so this will be a good challenge to get me started, any advice? books? tutorials? what should I expect?
Thanks stack as always!
Note : I know about Magic Lantern and CHDK, I want to get technical advise on how they were started and came to be.
http://magiclantern.wikia.com/wiki/Decompiling
http://magiclantern.wikia.com/wiki/Struct_Guessing
http://magiclantern.wikia.com/wiki/Firmware_file
http://magiclantern.wikia.com/wiki/GUI_Events/550D
http://magiclantern.wikia.com/wiki/Register_Map/Brute_Force
I wanted to know what steps one would need to take to "hack" a
camera's firmware to add/change features, specifically cameras of
Canon or Olympus make.
General steps for this hacking/reverse engineering:
Gathering information about the camera system (main CPU, Image coprocessor, RAM/Flash chips..). Challenges: Camera system makers tend to hide such sensitive information. Also, datasheets/documentation for proprietary chips are not released to public at all.
Getting firmware: through dumping Flash memory inside the camera or extracting the firmware from update packages used for camera firmware update. Challenges: Accessing readout circuitry for flash is not a trivial job specially with the fact that camera systems have one of the most densely populated PCBs. Also, Proprietary firmware are highly protected with sophisticated encryption algorithms when embedded into update packages.
Dis-assembly: getting a "bit" more readable instructions out of the opcode firmware. Challenges: Although dis-assemblers are widely available, they will give you the "operational" equivalent assembly code out of the opcode with no guarantee for being human readable/meaningful.
Customization: Just after understanding most of the code functionalities, you can make modifications that need not to harm normal operation of the camera system. Challenges: Not an easy task.
Alternatively, I highly recommend you to look for an already open source camera software (also HW). You can learn a lot about camera systems.
Such projects are: Elphel and AXIOM