I want to change some parametres for one of my spiders in settings.
I read the doc official of Scrapy
i tried this:
class MySpider(CrawlSpider):
name = 'mySpider'
custom_settings = {
DOWNLOADER_MIDDLEWARES = {
'Crawling.middlewares.ProxyMiddleware' : 100,
'Crawling.middlewares.RandomUserAgentMiddleware' : 400,
'scrapy.dowloadermiddlewares.useragent.UserAgentMiddleware' : None,
'scrapyjs.SplashMiddleware' : 725,
},
COOKIES_ENABLED= True,
DOWNLOAD_DELAY= 0,
}
but i got this error
DOWNLOADER_MIDDLEWARES = {
^
SyntaxError: invalid syntax
Setting name must be a dict key, take a look at the doc
class MySpider(CrawlSpider):
name = 'mySpider'
custom_settings = {
'DOWNLOADER_MIDDLEWARES': {
'Crawling.middlewares.ProxyMiddleware' : 100,
'Crawling.middlewares.RandomUserAgentMiddleware' : 400,
'scrapy.dowloadermiddlewares.useragent.UserAgentMiddleware' : None,
'scrapyjs.SplashMiddleware' : 725,
},
'COOKIES_ENABLED': True,
'DOWNLOAD_DELAY': 0,
}
Related
I am currently trying to use mixed chart types with vue-chartjs for a bar chart and a line chart(vue-chartjs#4.1.2, vuejs#2.6.14, chartjs#3.0.0).
As you can see in the following image the grid for the line chart (colored red) is not aligned with the grid for the bar chart.
mixed chart types
How I'm implementing the mixed chart types
const
chartContainer = this.chartContainer = this.chartContainer || document.createElement('canvas'),
data = {
datasets : [
{
data : rcd.getLineData(tAx.records),
borderColor : '#ff9e9e',
hoverBackgroundColor : '#ff0000',
fill : false,
stepped : 'end',
type : 'line',
xAxisID: 'xLine',
radius: 0,
},
{
data : rcd.getBarData(tAx.records, record, tAxs),
borderColor : '#ff9e9e',
backgroundColor : 'rgba(187,242,193,1)',
hoverBackgroundColor : '#a5eeac',
fill : true,
stepped : true,
type : 'bar',
barPercentage : 1.0,
categoryPercentage: 1.0,
xAxisID: 'xBar'
}
};
chartContainer.setAttribute('height', 100);
chartContainer.setAttribute('width', tAxModel.totalSize);
this.element.appendChild(chartContainer);
this.chart = new Chart(
chartContainer,
{
drawBorder : false,
data,
options : {
responsive : false,
animation : {
duration : 0
},
elements : {
point : {
radius : 0,
hoverRadius : 0
}
},
layout : {
padding : {
top : 5,
left : -10,
bottom : -19
}
},
plugins : {
legend : {
display : false
}
},
scales : {
xLine : {
offset: false,
grid : {
display : true,
offset : true,
color: 'red'
},
ticks : {
display : false,
}
},
xBar : {
grid : {
display : true,
offset : true
},
ticks : {
display : false,
}
},
y : {
borderWidth : 0,
min : 0,
ticks : {
display : false,
borderWidth : 0,
},
grid : {
borderWidth : 0,
display : true
}
},
},
tooltips: {
mode: 'index'
},
interaction : {
mode: 'index',
intersect : false,
axis : 'x'
}
},
}
);
Both charts get arrays with the same length and the same x-values for each item.
For example bar and line chart receive an array with values like {x: 'February 13, 2023', y: (value for bar or line chart)}
Logging this.chart and this.chart._sortedMetaSets I kind of found out that the difference for one pointElement to the next pointElement for line charts data is not equal to bar charts data.
For instance line charts pointElements: {x: 0, ...}, {x: 71.25, ...} -> it's 71.25 - but for bar charts pointElements: {x: 35, ...}, {x: 105, ...} -> it's 70.
Furthermore I found that the categoryScale for bar chart has difference to the line charts categoryScale.
Examples are _gridLineItems, _valueRange and _startValue.
Is there anything wrong in my implementation or my data?
Thank you in advance!
This is my response.
`
{
"Response": {
"_": {
"QueryResponse": {
"_": null,
"#": {
"totalCount": "55"
}
}
},
"#": {
"time": "2020-12-22T18:21:58.862-08:00"
}
}
}
`
I am doing this in feature file
* print "total count="+ result.session.runtimeData.read_count.Response._.QueryResponse.#.totalCount
and getting
<eval>:1:78 Expected ident but found error result.session.runtimeData.read_taxcode_count.Response._.QueryResponse.#.totalCount ^ in <eval> at line number 1 at column number 78
^ points to #
tried enclosing it in [''] many places for the response but I am not able to get the value.
Sometimes you don't need to complicate things, try this:
* def count = get[0] response..totalCount
Also this would have worked:
* def count = response.Response._.QueryResponse['#'].totalCount
I am trying to pass following parameters to request body but its not working :
{
"Start": 0,
"Limit": 10,
"SysID": 632;
"ResultScope": "Enrolled",
"SearchParams":
[
{"Field":"ID", "Weight": 1, "Value": ["1234567"], "Operation": "Active", "Enabled": true}
]
}
What i tried:
var test = request.AddBody(new Search { Start = 0, Limit = 10, SysId = 632, ResultScope = "Enrolled", SearchParams = new List<Object> { "ID", 1, 1234567, "Active”, "true" } });
but its not working. I am getting target parametercount exception when i am trying to add it to request as Addobject after seralization
You might want to read the docs.
request.AddJsonBody(
new Search {
Start = 0,
Limit = 10,
SysId = 632,
ResultScope = "Enrolled",
SearchParams = new List<Object> { "ID", 1, 1234567, "Active”, "true" }
});
var response = await client.GetAsync<WhateverYouWantBack>(request);
I am trying to update an nested array using Array filters, for hands on first i am trying with basic array filter update query, i copied pasted the update query from the mongodb tutorial, But i am getting error like: Error:"No array filter found for identifier 'elem' in path 'grades.$[elem].mean'
"and i am using
'db version v4.0.2' and
'MongoDB shell version v4.0.2
Here is my collection Details,
{
"_id" : 1,
"grades" : [
{
"grade" : 80,
"mean" : 75,
"std" : 6
},
{
"grade" : 85,
"mean" : 90,
"std" : 4
},
{
"grade" : 85,
"mean" : 85,
"std" : 6
}
]
}
//End of First Record
{
"_id" : 2,
"grades" : [
{
"grade" : 90,
"mean" : 75,
"std" : 6
},
{
"grade" : 87,
"mean" : 90,
"std" : 3
},
{
"grade" : 85,
"mean" : 85,
"std" : 4
}
]
}
//End of Second record
update Query:
db.getCollection('students2').update(
{ },
{ $set: { "grades.$[elem].mean" : 100 } },
{
multi: true,
arrayFilters: [ { "elem.grade": { $gte: 85 } } ]
}
)
Throw's the Error:
No array filter found for identifier 'elem' in path 'grades.$[elem].mean'
Read the comments of this StackOverflow issue:
arrayFilters not working
It doesn't work in "older shells". I'm using the Robo 3T client and running into the same issue. The shell is apparently removing the arrayFilters object.
For loading the Cy3-exported network, this works great:
cy.add(network.elements); // where 'network' is the json object exported by Cy3
What command loads and applies the exported vizmap? This command ran to completion, but the visual style of the network bears no resemblance to what we see in Cy3.
cy.style().fromJson(vizmap).update() // where 'vizmap' is the json vizmap object from Cy3
Advice, anyone? Perhaps even a small example?
Thanks!
Paul
Rather than loading this data after init, have you tried using the init options instead. That's where style is specified.
cytoscape({
style: networkJson.path.to.style,
elements: networkJson.path.to.elements
// ...
});
Edit: It looks like Cytoscape-desktop generates JSON like this
[ {
"title" : "hallmarksOfCancer",
"style" : [ {
"selector" : "node",
"css" : {
"text-opacity" : 1.0,
"width" : 70.0,
"border-opacity" : 1.0,
"background-color" : "rgb(255,255,255)",
"background-opacity" : 1.0,
"font-size" : 20,
"shape" : "ellipse",
"border-width" : 1.0,
"border-color" : "rgb(0,0,0)",
"color" : "rgb(0,0,0)",
"height" : 30.0,
"text-valign" : "center",
"text-halign" : "center",
"font-family" : "SansSerif",
"font-weight" : "bold",
"content" : "data(label)"
}
}, {
"selector" : "node[score > 4.6]",
"css" : {
"width" : 140.0
}
}, {
"selector" : "node[score = 4.6]",
"css" : {
"width" : 140.0
}
}, {
"selector" : "node[score > 0.0][score < 4.6]",
"css" : {
"width" : "mapData(score,0.0,4.6,70.0,140.0)"
}
}, {
In that case, you can use something like style: visjson[i].style in your init options, where i ranges on 0 <= i <= visjson.length. Cytoscape desktop exports several stylesheets at once, so you 'll need to specify which one (i.e. i) you want to use.