Is there any way to work with NetCDF files in Nim? - data-science

I've discovered Nim some days ago and am really interested in it.
However, I need to know if I'd able to work with NetCDF files (or at least GRB files) to see if it is suitable for my every-day life.
I've searched the web for some library or code, but couldn't find any, so if anybody knows a way to do it, please let me know.

Possible options (as of September 2020) to work with netcdf files using nim:
since netcdf4 is based on hdf5, maybe nimhdf5 could be useful
using nimpy bridge to access python libraries that access netcdf, such as netcdf4-python (your local python environment will be a dependency)
wrap netcdf C api using tools like c2nim or nimterop

Related

GraalVM: How do I import libraries from different languages in a single project? I am using IntelliJ

I have to make some functions that will use different lanaguages (python, R, js).
I got stuck at the part of generating random numbers in Python to initialize a list with random elements. I looked up on ways of initializing random lists, and then I decided to use result = polyglot.eval("python", "[random.randint(0,10) for i in range(20)];").
The problem that I face now is that I need to import the "random" library from python, or whatever libraries will I need from different languages. I heard that it might be a problem with the dependencies, but I am not sure...
What am I supposed to do? Is it even possible to import libraries from more languages in a single project? What other alternatives do I have?
Note that solution for different dynamic languages may differ.
Also js component is stable, while python (as of 2021) is still experimental.
Here is example for Python with modules
https://github.com/paulvi/java-python-graalvm-template
And if you really do polyglot (using Python object in Java code),
see https://github.com/hpi-swa-lab/graalpython-java-example
There is still issue how to actually deploy this in production
https://github.com/hpi-swa-lab/graalpython-java-example/issues/6
as just bundling venv subfolder into jar, will just work.
One solution is in ttps://github.com/paulvi/java-python-graalvm-template
Also randon, i.e. any library with graalvm is still big issue, as different packages have different issues, see https://github.com/oracle/graalpython/issues/228
I suggest, that before really mixing a lot of languages, just try one, e.g. js that is more stable, make it work, and then try next.
BTW PyCharm does not yet support graalpython.
If you do any open source, or later find somethin new, please let me know via GitHub issue

how-to-parse-a-ofx-version-1-0-2-file-in power BI?

I just read
How to parse a OFX (Version 1.0.2) file in PHP?
I am not a developer. What easy tool can I use to make this code run with no code skill or appetence ? web browser is pretty hard to use for non dev guys.
I need this to use the file into Power BI, which accept M code, json source or xml, but not sgml ofx or PHP.
Thanks in advance
Welcome Didier to StackOverflow!
I'm going to try and give you a clue how I'd approach the problem here. But keep in mind that your question really lacks details for us to help you, and I'm asking to update your question with example data that you want to integrate into PowerBI. Also, I'm not too familiar with PowerBI nor PHP, and won't go into making that PHP code you linked run for you.
Rather, I'd suggest to convert your OFX file into XML, and then use PowerBI's XML import on that converted file.
From your linked question, I get that your OFX file is in SGML format. There's a program specifically designed to convert SGML into XML (which is just a restricted form of SGML) called osx. I've detailed how to install it on Linux and Mac OS in another question related to SGML-to-XML down-converting; if you're on Windows, you may have luck by just downloading a really ancient (32bit) version of it from ftp://ftp.jclark.com/pub/sp/win32/sp1_3_4.zip. Alternatively, you can use my sgmljs.net software as explained in Converting HTML to XML though that tutorial is really about the much more complex task of converting HTML to XML/XHTML and will probably confuse you.
Anyway, if you manage to install osx, running it on your OFX file (which I assume to have the name yourfile.ofx just for illustration) is just a matter of invoking (on the Windows or Linux/Mac OS command line):
osx yourfile.ofx > yourfile.xml
to result in yourfile.xml which you can attempt to load with PowerBI.
Chances are your OFX file has additional text at the beginning (lines like XYZ:0001 that come before <ofx>). In that case, you can just remove those lines using a text editor before invoking osx on it. Maybe you also need a .dtd file or additional instructions at the top of the OFX file informing SGML about the grammar of your file; it's really difficult to say without seeing actual test data.
Before bothering with SGML and all that, however, I suggest to remove those first few lines in your OFX file (everything until the first < character) and check if PowerBI can already recognize your changed input file as XML (which, from other OFX example files, has a good chance of succeeding). Be sure to work on a copy of your original file rather than overwriting it. Then come back and update your question with your results and example data.

How to decode .gz files using Microsoft Access VBA?

I have some files which need reading using Access / VBA. They are compressed using ".gz" compression. How can I read those files in?
I figure this must be a solved problem but alas can't find anything. Command-line would be one option but it would involve the users of the VBA application having to have particular tools installed. Perhaps there is a library I can just include, which I can then ship with my VBA application (.accdr)?
There are quite a few libraries around, however, probably the most popular is zlib. A nice example using zlib, albeit written in VB6 (which shouldn't be too difficult to convert to VBA) is located here. One nice point about zlib is that it doesn't need registering (i.e. REGSVR*), so you should be able to drop it into the same folder as your DB (or even embed it into the DB then extract it automatically).

Structure of QuickTime's 'dref' atom 'alis' element

I need to rewrite a QuickTime reference movie, making it point to another set of files.
I'm working in Windows environment, so I don't have acces to the QuickTime API, and being the referenced files unaccesible, I can't also use the COM interface to load the movie because it can't resolve the referenced paths.
The documentation in the "QuickTime File Format Specification" says that the 'dref' atom can have a list of 'alis', 'url ' and 'rsrc' data references. In this case I need to parse the 'alis' elements. According to the reference, "Data reference is a Macintosh alias".
So long, I have not been able to see a declaration of the structure or any related information. Do you know the structure of an alias record? Where can I find detailed information about it's structure?
Thank you a lot for your help!
The format is very similar to the sort of alias that you could generate in the Finder by right-clicking an item, and creating an alias to it.
Aside: When the QuickTime format was originally specified, Apple intelligently chose to incorporate a number of other standards and paradigms that were extensively already being used elsewhere in the OS. This is one of the reasons why QT is (or was) able to do really clever things like reference movies. Unfortunately, there's also now a lot of cruft leftover from OS features that are no longer relevant (ie. AppleShare). Back in its heyday, QuickTime was slick, especially compared to its competitors; today, it's vastly underappreciated due to the buggy Windows port, and the relatively low processing power of the desktop systems of its time.
Back ontopic, unfortunately, the format for alias files is not an open/published standard, and there is precious little documentation on the topic on the 'net. There's one really old doc that deconstructs the alias format used in Mac OS Classic. Although the structure used in OS X is very similar, the alias files themselves tend to be much larger, as they contain numerous extra data strings at the end of the file that are not documented in the above-linked documentation.
Also, aliases created in the finder do look a bit different from the ones contained within the dref atom, although I've never run through them bit-by-bit to deduce the actual differences. If you want to take a peek at what those files, and have the OS X Developer Tools installed, you can run
setfile -a a [filename]
on a Finder-generated alias to strip the file of its alias-ness so that you can look at its contents in a hex editor (otherwise, the OS will just redirect you to the linked file - doh!). You can re-set the file's alias attribute, or arbitrarily designate any file as an alias by running
setfile -a A [filename]
Unfortunately, during my experiments, dumping the alis portion of a QT movie's dref atom has never seemed to generate an alias that Mac OS was able to interpret.
Fortunately (or not, as it was in my case), the functions that Mac OS allegedly uses to create/handle aliases are part of a public API called the Alias Manager, which is part of the very-low-level CoreServices framework. If you've got time to delve into this further, you can write some code to experiment with Mac OS's built-in alias-generating and interpreting capabilities.
Unfortunately, if you're dealing with an old/buggy file, you have no way of knowing if the file was actually generated by CoreServices' Alias Manager, or if that framework has changed/evolved/regressed since then. Because it's a closed format, 3rd-party developers who opt to not use the Alias Manager can only take guesses as to the format's "legal" structure.
You can use this Java program to see what is in the header, and extract data (it's a bit old, but may still work). What is more useful, though, is the thorough discussion by the author about the Quicktime header.
But I think you may just be looking for the Apple documentation, currently found here.

How to unrar/unzip with Objective-C, including single file retrieval

I have a need to handle various rar/zip files, in Objective-C. Ideally I'd like to be as flexible as possible in terms of rar/zip versions. I'd also like to be able to only extract certain files from the rar/zip files, after pulling out a list of the file contents.
If that wasn't enough, I'd like to be able to access and modify the zip comment.
Is this easily possible in objective-c? I've searched around a lot and found a lot of half-finished libraries that don't do everything I want, or only support rar up to version 2, or don't support extracting single files.
I know I could just use the command line unzip tool that ships with MacOS Panther and up, but this seems inelegant and doesn't help me with rar files, as no unrar application ships with MacOS by default.
Can anyone point me at a decent library that does one or the other of these two types of files, or a recommended best approach for dealing with this problem? I know that one option is to wrap the unrar source, and also wrap the zlib source, but this to me is a daunting task. If there's no other option I'll do it - any advice or guidance on this would be gratefully received.
Thanks!
Yes, doing that it's easy in objective C. For zip files just use ZLIB (it's already included in Mac OS X.
RAR is not that simple though. Look for a C library (not an Objective-C library). There will be way more C libraries for RAR handling than Objective-C ones. And you can use all C libraries you want within an Objective-C program.