I'm trying to setup the basic suite of spidermon monitors as described here I did a quick Google search and also found this. So I made a quick monitors.py, then copy and pasted the code in there.
I then proceeded to do this:
SPIDERMON_ENABLED = True
SPIDERMON_SPIDER_CLOSE_MONITORS = (
'spidermon.contrib.scrapy.monitors.SpiderCloseMonitorSuite',
)
in my settings.py in the scrapy project.
It keeps raising this error:
spidermon.exceptions.NotConfigured: You should specify a minimum number of items to check against
Which I believe I've done (SPIDERMON_MIN_ITEMS = 10 # "SPIDERMON_MIN_ITEMS" - at the top of the file).
What am I doing wrong? I just want to setup the pre-defined monitors and then optimize them later.
Spidermon couldn't find a valid value for SPIDERMON_MIN_ITEMS in the settings. This must be an integer value bigger than zero otherwise it'll throw the error described. SPIDERMON_ADD_FIELD_COVERAGE set is also mandatory in order to use all the monitors available in this MonitorSuite.
In order to run the built-in close MonitorSuite SpiderCloseMonitorSuite from Spidermon project, please confirm if the settings.py file - located in the root directory of your scrapy project - have the variables below:
EXTENSIONS = {
'spidermon.contrib.scrapy.extensions.Spidermon': 500,
}
SPIDERMON_ENABLED = True
SPIDERMON_MIN_ITEMS = 10
SPIDERMON_ADD_FIELD_COVERAGE = True
SPIDERMON_SPIDER_CLOSE_MONITORS = (
'spidermon.contrib.scrapy.monitors.SpiderCloseMonitorSuite',
)
Related
I'm new in Cytoscape. I want to know how can I run an app (for example MCL clustering algorithm) multiple times with different parameters in Cytoscape. Is there any way to write an script to do that instead of running manually multiple times for different parameters?
Thanks!
Thanks Scooter.
I saw his answer.
Still I have problem with MCODE.
I figured it out by reading this paper "Cytoscape Automation: empowering workflow-based network analysis".
I want to put the script here in the case that maybe somebody has question.
From python you need to import
import requests, json
import numpy
REST_ENDPOINT = 'http://localhost:1234'
and then let's say we want to use affinity propagation clustering algorithm, then you can go to help->Automation->CyRest command API. Here you can find the app and all its parameters. You load the input network from cytoscape in the beginning.
counter = 0
ap_clusters = dict()
for i in numpy.arange(-1.0, 1.1, 0.1):
message_body = {
"preference": str(round(i,1))
}
response = requests.post(REST_ENDPOINT + '/v1/commands/cluster/ap', data =
json.dumps(message_body), headers = {'Content-Type': 'application/json'})
response_data = response.json()['data']
ap_clusters[counter] = response_data['clusters']
counter += 1
above is a code to call AP clustering multiple times from python.
For AP and MCL the code works for multiple parameters. However when I tried to call MCODE with different set of parameters, it stopped the connection and closed the cytoscape app. It can only run for on set of parameters.
This is the error:
" raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))"
here is the code for mcode algorithm:
counter = 0
mcode_clusters = dict()
for i in numpy.arange(3,6,1):
for j in numpy.arange(0.1,0.56,0.05): #----vertex weight percentage
for h in ["on","off"]:
for f in ["on","off"]:
if f=="on":
for p in [0,0.1,0.2]: #---fluffing percentage
message_body = {
"fluff" : f,
"fluffNodeDensityCutoff" : str(round(p,1)),
"haircut" : h,
"maxDepthFromStart" : str(i),
"nodeScoreCutoff": str(round(j,1))
}
response = requests.post(REST_ENDPOINT + '/v1/commands/cluster/mcode', data = json.dumps(message_body), headers = {'Content-Type': 'application/json'})
response_data = response.json()['data']
mcode_clusters[counter] = response_data['clusters']
counter += 1
If you have any solutions I really appreciate it if you could share it with me.
Thanks.
SaRa
I think this was answered by Ruth pretty clearly in cytoscape-helpdesk:
You can do all of the above. Whatever is easiest for you.
There is a library py2cytoscape that you can use to issue commands to cytoscape from > python. Info can be found here: https://py2cytoscape.readthedocs.io/en/latest/
for more info on automation in cytoscape check out: http://manual.cytoscape.org/en/stable/Programmatic_Access_to_Cytoscape_Features_Scripting.html
But you can also run it through automation. You can create a text file with each of > your commands (for example a list of commands like: cluster mcl attribute="correlation" network=1234") and then go to Tools --> execute batch file to > execute the whole file. I am not sure if support loops. If you want to loop through anything I would recommend using python.
Thanks,
Ruth
I'll just add that currently, looping isn't supported in batch files.
-- scooter
In regard to my problem, I have to say that:
There are two mcode in cytoscape. One is in clusterMaker and the other belongs to cytoscape. When I tried to call mcode, I used the command "'/v1/commands/cluster/mcode'" I called the one in clusterMaker however the parameters' name was based on the one in cytoscape. I changed the command to "'/v1/commands/mcode/cluster'" and the the problem is solved now.
Many thanks.
SaRa
I'm writing a ZeroBrane Studio plugin for our Solarus Game Engine and It works like a charm. Autocompletion included.
I'm wondering now if it's do-able to register lua APIs for one file only.
I need this to offer autocompletion/documentation on global symbols that may vary per-script but are deducible from annex files from the engine.
To summary : Is it possible to register an api for a single file? For example in the onEditorLoad() event.
Thanks.
Greg
EDIT:
I tried the following without sucess:
local function switch_editor(editor)
if current_editor == editor then
ide:Print("same editor")
return
end
current_editor = editor
if not editor then
ide:Print("null ed")
return
end
lua_file_path = ide:GetDocument(editor).filePath
if lua_file_path:match('/data/maps/') then
ide:Print("map file!",type(editor))
local map_api = make_map_api(lua_file_path)
current_api = map_api
ide:AddAPI('lua','solarus_map',map_api)
else
ide:Print('other file')
if current_api then
ide:RemoveAPI('lua','solarus_map')
current_api = nil
end
end
end
api = {"baselib", "solarus", "solarus_map"}, --in interpreter table
... -- in the plugin table :
onEditorFocusSet = function(self,editor)
switch_editor(editor)
end,
Completion with the solarus api works fine but the on-fly registration of the solarus_map api seem not to be taken in account.
EDIT2:
Silly my, I must have done a typo, because after checking and rewriting some things pretty much as in the code pasted above... it works! Awesome!
The only small gotcha is that when switching to a file where I don't want the solarus_map API... ide:RemoveAPI isn't sufficient. Instead I must do ide:AddAPI('lua','solarus_map',{}) to replace the API with an empty one. Which I can live with.
To summary, to achieve a custom api which change from file to file:
Add the api name to the interpreter
In the onEditorFocusSet event, update the API with ide:AddAPI(...), eventually setting it to {} if it needs to be empty/disabled.
Code sample in the editions of my Question.
I started experimenting with DHTMLX Scheduler .Net but encountered some 'issues'.
I like to have everything ordered at my own way (css files in folder 'styling', js files in folder 'scripts', ...)
But now it seems like using this dll forces me to place everything in a folder that they chose. So when I init the Scheduler on my page, it automatically tries to call this files:
/Scripts/dhtmlxScheduler/dhtmlxscheduler.js
/Scripts/dhtmlxScheduler/dhtmlxscheduler.css
/Scripts/dhtmlxScheduler/locale/locale_nl.js
I would like to change these paths to for example '/scripts/dhtmlxscheduler.js' but I can't find a property in the Scheduler object where I'm able to do this.
Is this possible or do I have to put everything in the folder they would like to have it?
My working code thisfar:
Public Scheduler As DHXScheduler
Scheduler = New DHXScheduler()
Scheduler.InitialDate = New DateTime(Now.Year, Now.Month, Now.Day)
Scheduler.InitialView = "day"
Scheduler.Config.first_hour = 6
Scheduler.Config.last_hour = 20
Scheduler.Config.time_step = 30
Scheduler.Config.limit_time_select = True
Scheduler.Localization.Set(SchedulerLocalization.Localizations.Dutch)
Scheduler.DataAction = Me.ResolveUrl("/Handlers/Scheduler_Data.ashx")
Scheduler.SaveAction = Me.ResolveUrl("/Handlers/Scheduler_Save.ashx")
Scheduler.LoadData = True
Scheduler.EnableDataprocessor = True
I would like to find the possibility to set such a property to my own filepath, if possible ..
thx in advance
Further research showed me that I could change the default location with this line of code:
Scheduler.Codebase = "/js/dhtmlxscheduler"
However, this only sets the rootfolder, so all the files have to be located within this folder. I helped myself out by putting empty files over there and call my own styling and scripting from the location I wanted (just to get rid of the console errors).
So this solved my problem (as far as it could be solved).
I'm trying to organize a large number of CloudWatch alarms for maintainability, and the web console grays out the name field on an edit. Is there another method (preferably something scriptable) for updating the name of CloudWatch alarms? I would prefer a solution that does not require any programming beyond simple executable scripts.
Here's a script we use to do this for the time being:
import sys
import boto
def rename_alarm(alarm_name, new_alarm_name):
conn = boto.connect_cloudwatch()
def get_alarm():
alarms = conn.describe_alarms(alarm_names=[alarm_name])
if not alarms:
raise Exception("Alarm '%s' not found" % alarm_name)
return alarms[0]
alarm = get_alarm()
# work around boto comparison serialization issue
# https://github.com/boto/boto/issues/1311
alarm.comparison = alarm._cmp_map.get(alarm.comparison)
alarm.name = new_alarm_name
conn.update_alarm(alarm)
# update actually creates a new alarm because the name has changed, so
# we have to manually delete the old one
get_alarm().delete()
if __name__ == '__main__':
alarm_name, new_alarm_name = sys.argv[1:3]
rename_alarm(alarm_name, new_alarm_name)
It assumes you're either on an ec2 instance with a role that allows this, or you've got a ~/.boto file with your credentials. It's easy enough to manually add yours.
Unfortunately it looks like this is not currently possible.
I looked around for the same solution but it seems neither console nor cloudwatch API provides that feature.
Note:
But we can copy the existing alram with the same parameter and can save on new name
.
I have a trouble with Scons.Variables. I want to use config files with custom keys and values. My idea to load config files with keys and values and use it with SubstFile method.
For example (rough code) :
vars = Variables('templateValues.conf')
vars_dict = vars.UnknownVariables().keys() # bad code, need something to convert vars to Python dictionary
env.Substfile('myconfig.cfg.in', SUBST_DICT = vars_dict)
But vars.UnknownVariables() return empty list.
My test template file :
version = 105
mode = 'release'
source = 'database'
emulate = 'no'
And vars.UknownVariables() called :
vars = Variables('templateValues.conf')
print vars.UnknownVariables().keys()
# []
May be somebody try to implement something like this and can give some advances ?
I not found neeeded tools in Scons, but Python is great (i newbie in python now, few days studing only).
Google give me some useful links, such as SimpleConfigParser (i use method from CustomParser)
Implement it is very easy and i got what i need :
Import('env')
templVars = parse_config('template.conf')
varEnv = env.Clone(tools = ['textfile', 'default'])
varEnv.Substfile('config.cfg.in', SUBST_DICT = templVars)
Content of config.cfg.in file :
this is simple text with template values
Version is %version%
Build mode is %mode%
Emulator mode %emulate%
Thanks for using Avina !
Content of template.conf file :
%version% = 105
%mode% = 'test1'
%source% = 'database'
%emulate% = 'no'
And result file :
this is simple text with template values
Version is 105
Build mode is 'test1'
Emulator mode 'no'
Thanks for using Avina !