This is a simple example of a scrapy.cfg file:
[settings]
default = crawly.settings
[deploy:s1]
url = http://localhost:6800
project = my_project
I wanna know if I could pass any parameters to my scrapyd instance using this file. What I wanna do is to pass a parameter to one of my middlewares. You can pass parameters to your spider or override your settings but not anything with middlewares. I was thinking if a parameter like project could be passed to myserver, I could use that in my middleware.
check out this corresponding on scrapy github, check the code if they ended up integrating the suggested change or take pablo advice for post egg generation
Related
Appendix A of The Definitive Guide to Jython describes downloading SetupTools for use with Jython.
https://jython.readthedocs.io/en/latest/appendixA/
This indicates to me that it should be possible to download and use SetupTools from within a Jython automation script in Maximo (v7.6 in my case). The book points us to the following url to copy a Jython script that will do this:
http://peak.telecommunity.com/dist/ez_setup.py
I add one line to the above script to call the function "use_setuptools":
use_setuptools()
Then I create a push button on a Maximo application and associate the aforementioned script with the button press I get the following error:
System Message BMXAA7837E - An error occured that prevented the
EZ_SETUP script for the EZ_SETUP launch point from running.
urllib2.HTTPError: HTTP Error 403: SSL is required in at line
number 280
The stack trace points to the following line in the function "download_setuptools" which is called by "use_setuptools":
src = urllib2.urlopen(url)
This appears to be because the url requested, in my case:
http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c11-py2.5.egg
...is redirected a few times before arriving at the following url:
https://files.pythonhosted.org/packages/98/d3/ed3bc7e3f4b143092862dab99d984261ac4be6a40d4fb1e66d2a10e9ea99/setuptools-0.6c11-py2.5.egg
Note the url uses HTTPS not HTTP. The following indicates why this may be so:
https://sourceforge.net/p/pypi/support-requests/300/
The jython.jar included with Maximo does not include the ssl module so we could either:
Download the ssl module manually and copy it to the correct location on the server.
Download the appropriate egg file manually over HTTPS and copy it to the correct location on the server.
Bypass the problem by creating a mirror for the file we're looking for that is accessible over HTTP and use that url in the code.
Whilst these are feasible I'd prefer to modify the code to ignore the SSL certificate if possible, however all the workarounds on StackOverflow and elsewhere seem to require that you're able to "import ssl" in order to bypass it which rather seems to defeat the purpose.
Ideally I'm looking for a solution that modifies the code from the url provided above to get it to work with Maximo/Jython 2.5.2 and doesn't require downloading and adding new modules or packages and all that this entails with Maximo. Bypassing or temporarily disabling ssl is fine as the code checks the hash of the downloaded .egg file. This would be my preferred solution if possible.
In my experience, automation scripting works best if you can stay "as Java as possible" and "as Maximo as possible". So, I would use the LIB_HTTPCLIENT script from the Scripting 76 Features document (the first example code, whose name is given by inference in the second bit of code) to try to download the SetupTools.
In case that document moves again, here is the LIB_HTTPCLIENT script. Note that the url variable is expected to be passed to this library script by the calling script.
from psdi.iface.router import HTTPHandler
from java.util import HashMap
from java.util import String
handler = HTTPHandler()
map = HashMap()
map.put("URL",url)
map.put("HTTPMETHOD","GET")
responseBytes = handler.invoke(map,None)
response = String(responseBytes,"utf-8")
I have a page with a text field and button. After I fill out text field and press a button my controller is connecting to an API and getting some data based on the text.
I prepared a FAKE_API for testing. Both REAL_API and FAKE_API are in the service container. The FAKE_API is being prioritized when the environment is set to test (.env.test file). The controller gets the API object via dependency injection (constructor argument).
When I am testing using PhpBrowser from Codeception, the environment of the test itself is set to test - this can be checked by var_dump($_ENV['APP_ENV']) from the test.
However, (and this is the issue), if I add var_dump($_ENV['APP_ENV']) to the controller code and run the same test, I can see that the controller actually uses the regular 'dev' environment (set in .env file). This means that the REAL API is being used instead of my FAKE_API.
How can I force PhpBrowser tests to use my .env.test? Is it even possible?
You can't do that.
PhpBrowser communicates to system under test via HTTP, so it can't set environment variables of the system.
Your options are:
Deploy API in test configuration
Pass environment using GET or POST parameters or headers and make your app code accept it. (this is a bad idea)
I am looking at the class ConfigClientWatch in the package package org.springframework.cloud.config.client;
I was expecting that I could use this to poll the server periodically to see if the config had changed and then execute an refresh.
I am not able to get this to work? How does the value
String newState = this.environment.getProperty("config.client.state");
Get updated.
I have not been able to find any documentation on this.
Thanks in Advance
Raghu
Unfortunately, this property is only used by Vault backend. Anyway, there is a thread in the Spring Config's GitHub proposing changes to support other backends such as Git.
If you are using Git-backed configurations, this solution may work for you:
https://github.com/spring-cloud/spring-cloud-config/issues/1378#issuecomment-492073851
Please, upvote the GitHub thread so this feature gets accepted.
I would like to use a custom module for which I require "hostname" so that I can initiate SSH connection from the custom module and run commands. So I pass transport = "local" to the Runner object. However, I find no way to obtain "hostname" information in the custom module.
I am using Ansible 1.9.2 using Python API.
A module only has the information available that was explicitly passed to it. What you might be interested in instead is an action plugin, which by (non-exisiting) definition runs local on the control machine and has access to more (all?) data.
You can see some action plugin code here: https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/action
PS: Don't you want to upgrade to Ansible 2 before getting started writing custom modules/plugins? The API changed completely and once you upgrade you have to rewrite you module/plugin.
Okay, silly me. It's exactly the same way in the API too. You can extract hostname using {{ inventory_hostname }}.
I don't know how to make unit test of my rest controller. Here is my code :
public function testpostAction(){
$this->dispatch('/chain');
$this->request->setHeader('Content-Type', 'text/json')
->setMethod('POST')
->setPost(array(
'chain_name' => 'mychaintest'
));
$this->assertAction('post'); ???
}
How I make a post?
Not sure if this is what you need but, if you want to make a POST call (http) to test your REST service, you can use Zend_Http_Client:
http://framework.zend.com/manual/en/zend.http.client.html
Anyway, if this is for unit testing it will be more complicated, since you'll need your application (the current build being testet) to be live and accesible in the server. That depends on how you have configured your build environment.
There should be a staging (virtual) machine where the build is (automatically) deployed before tests are run. That machine should be visible to the machine runing the tests.
Hope this helped. Cheers!
So, basically your question is how to emulate calling PUT and DELETE in your controller tests?
Since this apparently doesn't work:
$this->request->setMethod('PUT');
You can access both these actions with plain HTTP POST by providing _method parameter.
So to call PUT:
$this->request->setMethod('POST');
$this->dispatch('articles/123?_method=put');
To call DELETE:
$this->request->setMethod('POST');
$this->dispatch('articles/123?_method=delete');
More reading on how to deal with RESTful routing here - http://framework.zend.com/manual/en/zend.controller.router.html#zend.controller.router.routes.rest