I know that requests module in python help to follow redirections, But if Target use Refresh Meta Tag
for Redirect user to another page requests module can't diagnose this attempt ...
For Example https://demo.tractmanager.com/ Redirect us to another page and requests module just can diagnose port of this page https://demo.tractmanager.com:443/
I want fix this problem with Regular Expressions Module (re) But I think that isn't good choice for solve it!
what's your suggestion and prefer?
For example I coded like that
fopen2 = open("clean url.txt", "r")
splurl = fopen2.read().split('\n')
for i in splurl:
getthis = requests.get(i)
sop = BeautifulSoup(getthis.text, 'html.parser')
if "http-equiv=\"refresh\"" in sop:
print('yes')
else:
print(getthis.url)
Thanks :)
Related
I need to add some custom headers to every boto3 request that is sent out. Is there a way to manage the connection itself to add these headers?
For boto2, connection.AWSAuthConnection has a method build_base_http_request which has been helpful. I've yet to find an analogous function within the boto3 documentation though.
This is pretty dated but we encountered the same issue, so I'm posting our solution.
I wanted to add custom headers to boto3 for specific requests.
I found this: https://github.com/boto/boto3/issues/2251, and used the event system for adding the header
def _add_header(request, **kwargs):
request.headers.add_header('x-trace-id', 'trace-trace')
print(request.headers) # for debug
some_client = boto3.client(service_name=SERVICE_NAME)
event_system = some_client.meta.events
event_system.register_first('before-sign.EVENT_NAME.*', _add_header)
You can try using a wildcard for all requests:
event_system.register_first('before-sign.*.*', _add_header)
*SERVICE_NAME- you can find all available services here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/index.html
For more information about register a function to a specific event: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/events.html
Answer from #May Yaari is pretty awesome. To the concern raised by #arainchi:
This works, there is no way to pass custom data to event handlers, currently we have to do it in a non-pythonic way using global variables/queues :( I have opened issue ticket with Boto3 developers for this exact case
Actually, we could leverage the python functional programming property: returning a function inside a function to get around:
In the case we want to add a custom value custom_variable to the header, we could do
some_client = boto3.client(service_name=SERVICE_NAME)
event_system = some_client.meta.events
event_system.register_first('before-sign.EVENT_NAME.*', _register_callback(custom_variable))
def _register_callback(custom_variable):
def _add_header(request, **kwargs):
request.headers.add_header('header_name_you_want', custom_variable)
return _add_header
Or a more pythonic way using lambda
some_client = boto3.client(service_name=SERVICE_NAME)
event_system = some_client.meta.events
event_system.register_first('before-sign.EVENT_NAME.*', lambda request, **kwargs: _add_header(request, custom_variable))
def _add_header(request, custom_variable):
request.headers.add_header('header_name_you_want', custom_variable)
I am trying to query and pull changelog details using python.
The below code returns the list of issues in the project.
issued = jira.search_issues('project= proj_a', maxResults=5)
for issue in issued:
print(issue)
I am trying to pass values obtained in the issue above
issues = jira.issue(issue,expand='changelog')
changelog = issues.changelog
projects = jira.project(project)
I get the below error on trying the above:
JIRAError: JiraError HTTP 404 url: https://abc.atlassian.net/rest/api/2/issue/issue?expand=changelog
text: Issue does not exist or you do not have permission to see it.
Could anyone advise as to where am I going wrong or what permissions do I need.
Please note, if I pass a specific issue_id in the above code it works just fine but I am trying to pass a list of issue_id
You can already receive all the changelog data in the search_issues() method so you don't have to get the changelog by iterating over each issue and making another API call for each issue. Check out the code below for examples on how to work with the changelog.
issues = jira.search_issues('project= proj_a', maxResults=5, expand='changelog')
for issue in issues:
print(f"Changes from issue: {issue.key} {issue.fields.summary}")
print(f"Number of Changelog entries found: {issue.changelog.total}") # number of changelog entries (careful, each entry can have multiple field changes)
for history in issue.changelog.histories:
print(f"Author: {history.author}") # person who did the change
print(f"Timestamp: {history.created}") # when did the change happen?
print("\nListing all items that changed:")
for item in history.items:
print(f"Field name: {item.field}") # field to which the change happened
print(f"Changed to: {item.toString}") # new value, item.to might be better in some cases depending on your needs.
print(f"Changed from: {item.fromString}") # old value, item.from might be better in some cases depending on your needs.
print()
print()
Just to explain what you did wrong before when iterating over each issue: you have to use the issue.key, not the issue-resource itself. When you simply pass the issue, it won't be handled correctly as a parameter in jira.issue(). Instead, pass issue.key:
for issue in issues:
print(issue.key)
myIssue = jira.issue(issue.key, expand='changelog')
We have just changed our domain after protracted name change (the name actually happened two years ago!) and our DokuWiki installation has stopped being able to see any groups and memberships.
The config has been updated to reflect the new server and DCs and login is working correctly, it is only the groups that aren't working.
$conf['auth']['ldap']['server'] = 'ldap://MYDC.mydomain.co.uk:389';
$conf['auth']['ldap']['binddn'] = '%{user}#mydomain.co.uk';
$conf['auth']['ldap']['usertree'] = 'dc=mydomain,dc=co,dc=uk';
$conf['auth']['ldap']['userfilter'] = '(userPrincipalName=%{user}#mydomain.co.uk)';
$conf['auth']['ldap']['mapping']['name'] = 'displayname';
$conf['auth']['ldap']['mapping']['grps'] = 'array(\'memberof\' => \'/CN=(.+?),/i\')';
$conf['auth']['ldap']['grouptree'] = 'dc=mydomain,dc=co,dc=uk';
$conf['auth']['ldap']['groupfilter'] = '(&(cn=*)(Member=%{dn})(objectClass=group))';
$conf['auth']['ldap']['referrals'] = '0';
$conf['auth']['ldap']['version'] = '3';
$conf['auth']['ldap']['debug'] = 1;
Obviously I have edited the doain name there, but for the life of me I can't see what's wrong here, It all worked fine yesterday on the old domain.
I should also state that this is an old version of DokuWiki that for various reasons I can't actually update.
The debug line gives me a "ldap search: success" line, but if I add "?do=check" onto any url within the system I get "You are part of the groups"...... and nothing, it can't see any groups.
It's a massive pain as we have a pretty intricate ACL setup for the site, so it's not like I can just throw it open to all.
If anyone has any suggestions, no matter how obvious, please pass them on.
Solved it by changing the dokuwiki authentication plugin that was used, the 'authad' is more simple to use and just works with what I'm doing.
As a side bonus it also means that I have finally been able to get the install upgraded to the current version.
I have a standalone java application which will interact with my web application running on node. I am trying to add new rules dynamically through web UI. So far I am unable to figure it out, how to create and add rules. Any suggestions for the right direction would be helpful
This is basically a duplicate of https://stackoverflow.com/questions/25036973 so the following is basically a duplicate of my answer to that question...
It's probably best to just look at the Drools examples source code. For instance the KieFileSystem example shows how to create a rule from a String and launch a session which includes it.
The essentials are that you create a KieServices, which contains a virtual file system. You then add rules to that file system. A little bit like the following:
KieServices ks = KieServices.Factory.get();
KieRepository kr = ks.getRepository();
KieFileSystem kfs = ks.newKieFileSystem();
kfs.write("src/main/resources/my/rules/therule.drl", "The source code of the rule");
KieBuilder kb = ks.newKieBuilder(kfs);
kb.buildAll();
you can add multiple Compiled rule DRL files like
knowledgebuilder.add(new ByteArrayResource(compiledDRL.getBytes()),ResourceType.DRL);
Get all the knowledgePackages and fire the all rules
knowledgeBase kbase = knowledgeBaseFactory.newKnowledgeBase();
kbase.addknowledgePackages(knowledgeBuilder.getKnowledgePackages());
knowledgeSession ksession = kbase.newStatefullKnowledgeSession();
ksession.insert(inputObject);
ksession.fireAllRules();
ksession.dispose();
We are not able to call balanced.card.create from a Phonegap application. This is reproduced in a stock Phonegap application here: https://github.com/kevg/phonegap-balanced. Full details are in the README.md on github, but the basic summary is:
For those not familiar with phonegap, the main page that loads is
index.html. This initializes phonegap in index.js. When the device is
ready, we will show a hidden DIV with a button named "Execute
Balanced." When you click this button, app.executeBalanced in index.js
will be called which prompts for the balanced marketplace URI, loads
balanced.js with $.getScript, and then calls balanced.card.create with
a test credit card.
The expected result is that callbackHandler is called or an exception
is caught. Instead, it seems the execution of the Javascript thread
disappears into balanced.card.create, never to return and without any
error.
Alrighty, I found the bug in balanced.js. So, in Phonegap, window.location.href returns something like file:///.../index.html. Balanced.js creates an iframe to something like https://js.balancedpayments.com/proxy#file
var src = proxy + "#" + encodeURIComponent(window.location.href);
https://github.com/balanced/balanced-js/blob/master/src/utils.js#L48
In the script returned in proxy.html (which I can't find on github), it does:
c.parentURL=decodeURIComponent(
window.location.hash.replace(/^#/,"")
).replace(/#.*$/,"")
c.parentDomain=c.parentURL.replace(/([^:]+:\/\/[^\/]+).*/,"$1")
The regex doesn't match because file: has three slashes. Now, at first, I thought I could just convert the regex to:
/([^:]+:\/+[^\/]+).*/
However, then there's another problem, because balanced does a security origin check on the match:
if (d.origin.toLowerCase() !== c.toLowerCase()) return !1;
However, the regex returns file:///firstcomponent, whereas event.origin does not include a host name for the file scheme, so these won't match even with a fixed regex.
I can't change anything in the script returned in the proxy response because if I load that from a domain other than balancedpayments.com, then the AJAX POST fails (return code 0 with a blank body). Therefore, the only thing I can control is the hash passed to the iframe.
However, since this regex is a replace, we can simply pass exactly what we know we need (we don't care that the regex is a no-op).
Therefore, the solution is to change L48 above to:
var src = proxy + "#" + encodeURIComponent("file://");
This works.