I'm trying to prototype using the SmartRedis Python client to interact with the SmartSim Orchestrator. Is it possible to launch the orchestrator without any other models in the experiment? If so, what would be the best way to do so?
It is entirely possible to do that. A SmartSim Experiment can contain different types of 'entities' including Models, Ensembles (i.e. groups of Models), and Orchestrator (i.e. the Redis-backed database). None of these entities, however, are 'required' to be in the Experiment.
Here's a short script that creates an experiment which includes only a database.
from SmartSim import Experiment
NUM_DB_NODES = 3
exp = Experiment("Database Only")
db = exp.create_database(db_nodes=NUM_DB_NODES)
exp.generate(db)
exp.start(db)
After this, the Orchestrator (with the number of shards specified by NUM_DB_NODES) will have been spunup. You can then connect the Python client using the following line:
client = smartredis.Client(db.get_address()[0],NUM_DB_NODES>1)
I am trying to learn ML models for predicting stock prices and initially, I tried using DataReader
import pandas_datareader as web
df = web.DataReader('AAPL', data_source='yahoo', start='2016-01-01', end='2021-08-01')
But I get a RemoteDataError and kept hitting a dead end trying to figure it out so I tried using tiingo
https://tiingo-python.readthedocs.io/en/latest/readme.html
I read through the documentation and tried passing a dictionary with 'api_key' as a key
into my tiingo client, ie.
from tiingo import TiingoClient
client = TiingoClient()
config = {}
config['session'] = True
config['api_key'] = 'my_api_key'
client = TiingoClient(config)
The documentation says I can now use TiingoClient to make API calls, however,
RuntimeError: Tiingo API Key not provided. Please provide via environment variable or config argument.
It is quite challenging learning the ML models and its syntax but what compounds the difficulty for me is what some data-scientests consider to be trivial as they don't typically deal with gathering or scraping data. Maybe my question is trivial but I've spent about an hour trying to figure out how to import data properly for stock prices and the only method that worked for me so far is
df = web.get_data_yahoo('stock symbol')
but I would like to grasp the other ways of importing stock prices via Tiingo and DataReader so if anyone can provide explanations/tips/suggestions I'd greatly appreciate it.
EDIT: for my tiingo account I did not buy any subscription plan for using their data as I was under the impression I can access data for free with my api-key
This is what I use, but its identical to what you are using it seems.
config = {}
config['session'] = True
config['api_key'] = "key here"
client = TiingoClient(config)
Remove this line: TiingoClient()
I am trying to figure out how to import a symmetric key into NSS for use with encryption at the core crypto boundary. These functions are described
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Reference/NSS_cryptographic_module/FIPS_mode_of_operation
I have been able to do every other type of crypto operation by following the documentation because it mirrors PKCS 11 described here:
http://docs.oasis-open.org/pkcs11/pkcs11-base/v2.40/cos01/pkcs11-base-v2.40-cos01.html
However attempting to import any template where the CK_OBJECT_CLASS" is "CKO_SECRET_KEY" always returns "CKR_ATTRIBUTE_VALUE_INVALID 0x00000013". But I have no problem with assymetric (public/private)
CK_RV crv;
CK_FUNCTION_LIST_PTR pFunctionList;
CK_OBJECT_CLASS keyClass = CKO_SECRET_KEY;
CK_ATTRIBUTE keyTemplate[] = {
{CKA_CLASS, &keyClass, sizeof(keyClass)}
};
crv = pFunctionList->C_CreateObject(hRwSession, keyTemplate, 1, &hKey);
printf("failed with 0x%08X\n", crv);
But according to the documentation this should be returning "CKR_TEMPLATE_INCOMPLETE" as "CKO_SECRET_KEY" is a valid object class.
Again I have had no trouble with assymetric. I should Also point out that my function pointers is for FIPS mode only. Any insight is greatly appreciated!
It looks like the code you pasted is either incomplete or simply wrong. In particular, there's no concrete value for the key you're creating in the template (CKA_VALUE), which can easily cause the error you're getting from C_CreateObject.
I'm working on integrating an experiment in psychopy with the eyelink eyetracking system. The way to do this seems to be through pylink. Unfortunately I'm really unfamiliar with pylink and I was hoping there was a sample of an experiment that combines the two. I haven't been able to find one. If anyone would be able to share an example or point me towards a more accessible manual than the pylink api that sr-research provides I'd be really grateful.
Thanks!
I am glad you found your solution. I have not used iohub, but we do use psychopy and an eyelink and therefore some of the following code may be of use to others who wish to invoke more direct communication. Note that our computers use Archlinux. If none of the following makes any sense to you, don't worry about it, but maybe it will help others who are stumbling along the same path we are.
Communication between experimental machine and eye tracker machine
First, you have to establish communication with the eyelink. If your experimental machine is turned on and plugged into a live Eyelink computer then on linux you have to first set your ethernet card up, and then set the default address that Eyelink uses (this also works for the Eyelink 1000 - they kept the same address). Note your ethernet will probably have a different name than enp4s0. Try simply with ip link and look for something similar. NB: these commands are being typed into a terminal.
#To set up connection with Eyelink II computer:
#ip link set enp4s0 up
#ip addr add 100.1.1.2/24 dev enp4s0
Eyetracker functions
We have found it convenient to write some functions for talking to the Eyelink computer. For example:
Initialize Eyetracker
sp refers to the tuple of screenx, screeny sizes.
def eyeTrkInit (sp):
el = pl.EyeLink()
el.sendCommand("screen_pixel_coords = 0 0 %d %d" %sp)
el.sendMessage("DISPLAY_COORDS 0 0 %d %d" %sp)
el.sendCommand("select_parser_configuration 0")
el.sendCommand("scene_camera_gazemap = NO")
el.sendCommand("pupil_size_diameter = %s"%("YES"))
return(el)
NB: the pl function comes from import pylink as pl. Also, note that there is another python library called pylink that you can find on line. It is probably not the one you want. Go through the Eyelink forum and get pylink from there. It is old, but it still works.
Calibrate Eyetracker
el is the name of the eyetracker object initialized above. sp screen size, and cd is color depth, e.g. 32.
def eyeTrkCalib (el,sp,cd):
pl.openGraphics(sp,cd)
pl.setCalibrationColors((255,255,255),(0,0,0))
pl.setTargetSize(int(sp[0]/70), int(sp[1]/300))
pl.setCalibrationSounds("","","")
pl.setDriftCorrectSounds("","off","off")
el.doTrackerSetup()
pl.closeGraphics()
#el.setOfflineMode()
Open datafile
You can talk to the eye tracker and do things like opening a file
def eyeTrkOpenEDF (dfn,el):
el.openDataFile(dfn + '.EDF')
Drift correction
Or drift correct
def driftCor(el,sp,cd):
blockLabel=psychopy.visual.TextStim(expWin,text="Press the space bar to begin drift correction",pos=[0,0], color="white", bold=True,alignHoriz="center",height=0.5)
notdone=True
while notdone:
blockLabel.draw()
expWin.flip()
if keyState[key.SPACE] == True:
eyeTrkCalib(el,sp,cd)
expWin.winHandle.activate()
keyState[key.SPACE] = False
notdone=False
Sending and getting messages.
There are a number of built-in variables you can set, or you can add your own. Here is an example of sending a message from your python program to the eyelink
eyelink.sendMessage("TRIALID "+str(trialnum))
eyelink.startRecording(1,1,1,1)
eyelink.sendMessage("FIX1")
tFix1On=expClock.getTime()
Gaze contingent programming
Here is a portion of some code that uses the eyelink's most recent sample in the logic of the experimental program.
while notdone:
if recalib==True:
dict['recalib']=True
eyelink.sendMessage("RECALIB END")
eyelink.startRecording(1,1,1,1)
recalib=False
eventType=eyelink.getNextData()
if eventType==pl.STARTFIX or eventType==pl.FIXUPDATE or eventType==pl.ENDFIX:
sample=eyelink.getNewestSample()
if sample != None:
if sample.isRightSample():
gazePos = sample.getRightEye().getGaze()
if sample.isLeftSample():
gazePos = sample.getLeftEye().getGaze()
gazePosCorFix = [gazePos[0]-scrx/2,-(gazePos[1]-scry/2)]
posPix = posToPix(fixation)
eucDistFix = sqrt((gazePosCorFix[0]-posPix[0])**2+(gazePosCorFix[1]-posPix[1])**2)
if eucDistFix < tolFix:
core.wait(timeFix1)
notdone=False
eyelink.resetData()
break
Happy Hacking.
rather than PyLink, you might want to look into using the ioHub system within PsychoPy. This is a more general-purpose eye tracking system that also allows for saving data in a common format (integrated with PsychoPy events), and provides tools for data analysis and visualisation.
ioHUb is built to be agnostic to the particular eye tracker you are using. You just need to create a configuration file specific to your EyeLink system, and thereafter use the generic functions ioHiv provides for calibration, accessing gaze data in real-time, and so on.
There are some teaching resources accessible here: http://www.psychopy.org/resources/ECEM_Python_materials.zip
For future readers, I wanted to share my library for combining pylink and psychopy. I've recently updated it to work with python 3. It provides simple to use, high level functions.
https://github.com/colinquirk/templateexperiments/tree/master/eyelinker
You could also work at a lower level with the PsychoPyCustomDisplay class (see the pylink docs for more info about EyeLinkCustomDisplay).
For an example of it in use, see:
https://github.com/colinquirk/ChangeDetectionEyeTracking
(At the time of writing, this experiment code is not yet python 3 ready, but it should still be a useful example.)
The repo also includes other modules for creating experiments and recording EEG data, but they are not necessary if you are just interested in the eyelinker code.
I have a standalone java application which will interact with my web application running on node. I am trying to add new rules dynamically through web UI. So far I am unable to figure it out, how to create and add rules. Any suggestions for the right direction would be helpful
This is basically a duplicate of https://stackoverflow.com/questions/25036973 so the following is basically a duplicate of my answer to that question...
It's probably best to just look at the Drools examples source code. For instance the KieFileSystem example shows how to create a rule from a String and launch a session which includes it.
The essentials are that you create a KieServices, which contains a virtual file system. You then add rules to that file system. A little bit like the following:
KieServices ks = KieServices.Factory.get();
KieRepository kr = ks.getRepository();
KieFileSystem kfs = ks.newKieFileSystem();
kfs.write("src/main/resources/my/rules/therule.drl", "The source code of the rule");
KieBuilder kb = ks.newKieBuilder(kfs);
kb.buildAll();
you can add multiple Compiled rule DRL files like
knowledgebuilder.add(new ByteArrayResource(compiledDRL.getBytes()),ResourceType.DRL);
Get all the knowledgePackages and fire the all rules
knowledgeBase kbase = knowledgeBaseFactory.newKnowledgeBase();
kbase.addknowledgePackages(knowledgeBuilder.getKnowledgePackages());
knowledgeSession ksession = kbase.newStatefullKnowledgeSession();
ksession.insert(inputObject);
ksession.fireAllRules();
ksession.dispose();