Cannot import certain Math modules in Jython - jython

I'm trying to import some modules from the math library but only some of the modules will import. isnan and asinh for example will not import but pi and acos will. Based on the jython documentation these modules should be able to be imported.
Does anyone have a solution as to why only some of the modules are importing?

I'm afraid I have some bad news for you; but I may have a possible work-around.
The bad news
I recently did a pretty sizable project using Jython, and from that experience concluded that there are errors in the Jython documentation. Much of it seems to have been more or less copied directly from the Python documentation, and unfortunately there are some inconsistencies between the implementations.
If you first launch the Python interpreter, import math, and then do a dir on the module, you will find isnan:
Python 2.7.5 (default, Jun 17 2014, 18:11:42)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import math
>>> dir(math)
['__doc__', '__file__', '__name__', '__package__', 'acos', 'acosh', 'asin',
'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees',
'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp',
'fsum', 'gamma', 'hypot', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10',
'log1p', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']
>>> print math.isnan.__doc__
isnan(x) -> bool
Check if float x is not a number (NaN).
On the other hand, if you do the same in Jython, you will find that the math module has no isnan function.
When I installed Jython, I installed the source as well. So, just to be sure, I grepped through the source and found the following:
[mdiana#tc-xdevmd modules]$ pwd
/opt/jython/jython-2.5.3/src/org/python/modules
[mdiana#tc-xdevmd modules]$ grep -Ri "isnan" *
math.java: if (Double.isNaN(v))
math.java: if (Double.isNaN(x) || Double.isInfinite(x) || x == 0.0) {
struct.java: Float.isInfinite(v) || Float.isNaN(v))) {
struct.java: (Double.isInfinite(v) || Double.isNaN(v))) {
struct.java: Float.isInfinite(v) || Float.isNaN(v))) {
struct.java: (Double.isInfinite(v) || Double.isNaN(v))) {
That's further evidence that there is no isnan anywhere in the code (no matter what the documentation might say). Jython itself is using the Java libraries for that functionality.
Suggested work-around
So, you're not going crazy, and there is nothing wrong with your installation. Working with Jython, you will run into problems like this from time to time. Sorry. What I suggest is using the Java libraries, either java.lang.Float or java.lang.Double. Both of those have a static method isNaN:
[mdiana#tc-xdevmd ~]$ jython
Jython 2.5.3 (2.5:c56500f08d34+, Aug 13 2012, 14:48:36)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_31
Type "help", "copyright", "credits" or "license" for more information.
>>> import java.lang.Float as JFloat
>>> JFloat.isNaN(4.001)
False
All in all, I like Jython; but sometimes things like the above are the best we can do. Hope that helps.

Related

DB2 ODBC converting NULL floats and ints to ~0

Querying DB2 from python using ODBC, I am seeing NULL values converted to 0 (on Linux, seemingly corrupt but close to 0 on Mac M1 -- even more worryingly).
This is using the db2 docker image started like this:
docker run -itd --name db2 --privileged=true -p 50000:50000 -e LICENSE=accept -e DB2INST1_PASSWORD=xxxxx -e DBNAME=testdb -v <db storage dir>:/database ibmcom/db2
Code as follows recreates the issue:
import pyodbc
cs = "Driver={ODBC Driver v11.5.7 for DB2};Database=xxxxx;Hostname=xxxx;Port=50000;Protocol=TCPIP;Uid=xxxx;Pwd=xxxx;"
cnxn = pyodbc.connect(cs)
crsr = cnxn.cursor()
crsr.execute("SELECT CAST(NULL AS INT), CAST(NULL AS REAL) FROM SYSIBM.SYSDUMMY1");
print(crsr.messages)
print(crsr.fetchall())
Outputs:
❯ python float-test.py
[]
[(2, 4.2439915814e-314)]
Is it expected that I can't retrieve NULL values as plain data types? It seems to be allowed in PostgreSQL. I know I can cast around this but would rather not, obviously.
Extra Info
It does seem that the ODBC driver version 11.5.7 from Fix Central suffers this issue whilst the 11.5.6 version from https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli does not.
As mentioned in comments, it appears pyodbc is impacted and both plain ibm_db and DBI are not impacted (both return None, None). So at least there is a workaround.
The reason for the behaviour deifference is that pyodbc is using SQLGetData() while the other two use the SQLBindCol() methods of extracting the result set data.
IBM's clidriver on Linux x64, SQLGetData() sets the (SQLLEN *) StrLen_Or_IndPtr parameter to SQL_NULL_DATA when the value of the column in result-set is NULL. But the problem is that IBMs clidrver sets StrLen_or_IndPtr to SQL_NULL_DATA (as int, 4 bytes), when pyodbc code expects it to SQL_NULL_DATA (as SQLLEN, 8bytes on Linux x64) as SQLLEN is the documented datatype for the StrLen_or_IndPtr argument.
Therefore the comparison in pyodbc getdata.cpp GetDataDouble() :
if ( cbFetched == SQL_NULL_DATA )
Py_RETURN_NONE;
will be false, causing the code to return an unitialised variable instead of Py_None.
I do not know if the maintainers of pyodbc run their tests against a Db2-LUW product, but it looks like other parts of the code could suffer the same problem and other issues may lurk. Consider asking on github what is the support policy for Db2-LUW in pyodbc.
If you have a support contract, IBM should also be asked to comment on their reason for not respecting the datatype of StrLen_Or_IndPtr when writing SQL_NULL_DATA to this parameter on Linux x64.

Pipeline keeps failing at test and lint stage

I am pretty new to coding so I am sorry if I'm not giving enough context. I am trying to package and deploy a model in Python to a repo but keep getting the follow error when I push to GitLab
ERROR: No matching distribution found for numpy==1.23.1
$ if [[ "${COVERAGE_ARGS}" == "" ]]; then
$ dir_loc=`pwd`
$ COVERAGE_ARGS="--cov=${dir_loc}"
$ fi
$ pytest --junitxml=reports/report.xml --cov-config=${COV_CONFIG} ${COVERAGE_ARGS} ${EXTRA_ARGS}
============================= test session starts ==============================
platform linux -- Python 3.7.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /builds/Ahs1yUQY/1/dse/dscp/dse-dscp-python-hoffinator
plugins: cov-3.0.0, requests-mock-1.9.3
collected 0 items / 1 error
==================================== ERRORS ====================================
______________________ ERROR collecting test/test_main.py ______________________
ImportError while importing test module '/builds/Ahs1yUQY/1/dse/dscp/dse-dscp-python-hoffinator/test/test_main.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.7/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
test/test_main.py:5: in <module>
from src.hoffinator import hoffinator_functions
src/hoffinator/hoffinator_functions.py:3: in <module>
import numpy as np
E ModuleNotFoundError: No module named 'numpy'
- generated xml file: /builds/Ahs1yUQY/1/dse/dscp/dse-dscp-python-hoffinator/reports/report.xml -
---------- coverage: platform linux, python 3.7.10-final-0 -----------
My requirements.txt for dependencies is as follows
ccplatlogging
boto3~=1.20.33
email-validator~=1.1.3
appnope==0.1.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.5
attrs==21.4.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
cffi==1.15.1
cycler==0.11.0
debugpy==1.6.2
decorator==5.1.1
defusedxml==0.7.1
entrypoints==0.4
executing==0.8.3
fastjsonschema==2.16.1
fonttools==4.34.4
iniconfig==1.1.1
ipykernel==6.15.1
ipython==7.34.0
ipython-genutils==0.2.0
ipywidgets==7.7.1
jedi==0.18.1
Jinja2==3.1.2
joblib==1.1.0
jsonschema==4.7.2
jupyter==1.0.0
jupyter-client==7.3.4
jupyter-console==6.4.4
jupyter-core==4.11.1
jupyterlab-pygments==0.2.2
jupyterlab-widgets==1.1.1
kiwisolver==1.4.4
MarkupSafe==2.1.1
matplotlib==3.5.2
matplotlib-inline==0.1.3
mistune==0.8.4
nbclient==0.6.6
nbconvert==6.5.0
nbformat==5.4.0
nest-asyncio==1.5.5
notebook==6.4.12
numpy==1.23.1
packaging==21.3
pandas==1.4.3
pandocfilters==1.5.0
parso==0.8.3
patsy==0.5.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.2.0
pluggy==1.0.0
ppscore==1.2.0
prometheus-client==0.14.1
prompt-toolkit==3.0.30
psutil==5.9.1
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
pycparser==2.21
pycryptodome==3.15.0
Pygments==2.12.0
pyodbc==4.0.34
pyparsing==3.0.9
pyrsistent==0.18.1
pytest==7.1.2
python-dateutil==2.8.2
pytz==2022.1
pyzmq==23.2.0
qtconsole==5.3.1
QtPy==2.1.0
scikit-learn==0.24.2
scipy==1.8.1
seaborn==0.11.2
Send2Trash==1.8.0
six==1.16.0
soupsieve==2.3.2.post1
stack-data==0.3.0
statsmodels==0.13.2
teradatasql==17.20.0.0
terminado==0.15.0
threadpoolctl==3.1.0
tinycss2==1.1.1
tomli==2.0.1
tornado==6.2
traitlets==5.3.0
wcwidth==0.2.5
webencodings==0.5.1
widgetsnbextension==3.6.1
I do not understand why numpy is not found when it is cleary in the .txt. Is it a version issue?
Your console output shows that you are using Python 3.7.10. If you take a look at numpy release notes for 1.23.1 found here you’ll notice that it states:
The Python version supported for this release are 3.8-3.10.
You will need to either use a newer version of Python or use an older version of numpy.

How to interact with a subprocess through its stdin, stdout, stderr in Smalltalk?

This Python code shows how to call some process in Windows 10 and to send to it string commands, to read its string responses through stdin, stdout pipes of the process:
Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import *
>>> p = Popen("c:/python38/python.exe", stdin=PIPE, stdout=PIPE)
>>> p.stdin.write(b"print(1+9)\n")
11
>>> p.communicate()
(b'10\r\n', None)
>>>
As you can see the python.exe process returned 10 as an answer to print(1+9). Now I want to do the same in Pharo (or Squeak): in Windows 10 OS - I suppose something similar, i.e. short, simple, understandable, really working.
I installed OSProcess, ProcessWrapper (they were missing in Pharo, also its strange that I got warning that they are not marked for Pharo 8.0 and were not checked to work in Pharo 8.0, but OK), and I tried ProcessWrapper, PipeableOSProcess (copy-pasted different snippets from the Web), etc - with zero success! The results were:
nothing happens, python.exe was not started
VM errors console was opened (white console in the bottom of the Pharo, which is controlled with F2 menu)
different exceptions
etc
Would somebody show me simple working example how to start a process and to to send it commands, read answers, then send again, and so on in some loop - I plan to have such communication in a detached thread and to use it as some service, because Pharo, Smalltalk in general is missing most bindings, so then I will use subprocess communication like in "good" old days...
I know how to call a command and to get its output:
out := LibC resultOfCommand: 'dir ', aDir.
but I am talking about another scenario: a communication with a running process interactively (for example, with SSH or similar like in the example above - python.exe).
PS. Maybe it's possible to do it with LibC #pipe:mode even?
Let me start with that the PipeableOsProcess is probably broken on Windows. I have tried it and it just opened a command line and nothing else (it does not freeze my Pharo 8). The whole OSProcess does not work correctly in my eyes.
So I took a shot at LibC which is supposed to not work with Windows.
I’m a module defining access to standard LibC. I’m available under Linux and OSX, but not under Windows for obvious reasons :)
Next is to say that Python's Windows support is probably much better than Pharo's.
The solution, which is more like a workaround using files, is to use LibC and #runCommand: (I tried to come up with a similar example as you had shown above):
| count command result outputFile errorFile |
count := 9+1. "The counting"
command := 'echo ', count asString. "command run at the command line"
outputFile := 'output'. "a file into which the output is redirected"
errorFile := 'error'. "a file where the error output is redirected "
result := LibC runCommand: command, "run the command "
' >', outputFile, "redirect the output to output file"
' 2>', errorFile.
"reading back the value from output file"
outputFile asFileReference contents lines.
"reading back the value from the error file - which is empty in this case"
errorFile asFileReference contents lines.

how to run an execution from a launchplan with flytesdk

I'm trying to use the flytesdk to run an execution from a launch plan.
I was given an example of
lp = SdkLaunchPlan.fetch('project', 'domain', 'name', 'version')
ex = lp.execute('project', 'domain' inputs={'a': 1, 'b': 'hello'}, <name='optional idempotency string'>)
but it looks like SdkLaunchPlan.execute() is not implemented but SdkLaunchPlan.execute_with_literals() is.
I was able to execute it with this code:
#I omitted the version parameter because the launch plan is active
lp = flytekit.common.launch_plan.SdkLaunchPlan.fetch(project="prj", domain="development", name="train.single.test_launch_plan")
literals = flytekit.clis.helpers.construct_literal_map_from_parameter_map(lp.default_inputs, {"depth": "False"})
lp.execute_with_literals("prj", "development", literal_inputs=literals)
is this the correct way of doing this or is there a better one?
Which version of flytekit are you on? Both should work. I think execute is a bit easier to use when you have an ipython terminal running. I was able to launch an execution on my cluster with the following commands.
(examples3) alice:~ [docker-desktop] $ ipython
Python 3.7.5 (default, Nov 1 2019, 02:16:32)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.7.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from flytekit.clis.flyte_cli.main import _detect_default_config_file
Using default config file at /Users/alice/.flyte/config
In [2]: from flytekit.common.launch_plan import SdkLaunchPlan
In [3]: lp = SdkLaunchPlan.fetch("flyteexamples", "development", "app.workflows.work.WorkflowWithIO", "8f49b8d8c04251865a7a8aba1b423293efc51374")
In [4]: lp.execute('flyteexamples', 'development', inputs={'a': 42, 'b': 'hello world'})
The code for execute_with_literals is here:
https://github.com/lyft/flytekit/blob/5a0a8da9251bd13bd67b71e0b05b6e59ecb970f9/flytekit/common/launch_plan.py#L186
And the code for execute is here:
https://github.com/lyft/flytekit/blob/5a0a8da9251bd13bd67b71e0b05b6e59ecb970f9/flytekit/common/mixins/executable.py#L8
The difference between the two is that one is meant to work with raw Python literals and the other is meant to work with Flyte literal types.
My bad, it looks like my editor's (VSCode) autocomplete did not recognize the .execute() method...I tried it anyways and it's working as advertised

Jython stalls on 1 = 2

I have been investigating jython a bit, and at the jython terminal accidentally typed 1 = 2 instead of q = 2, and found this:
>>> 1 = 2
...
...
I had to Ctrl+C to get out of it. No other input seems to make it happy.
I put the same code in a script and ran it with the same behavior (it just hangs).
In CPython, I get a SyntaxError: can't assign to literal (as expected).
Any idea what's going on? Is this just a jython bug?
$ jython
Jython 2.5.2 (Debian:hg/91332231a448, Jun 3 2012, 09:02:34)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.7.0_51