We have is very simple Web service.
HTTP/HTTPS request on out service from Internet clients.
In HEAD return UID.
We want to testing our service by Yandex Tank.
load.yml
phantom:
address: 211.81.41.11:443 #IP тестового стенда, порт 443
# address: her.your.ru:443 #IP тестового стенда, порт 443
ssl: true #use https
uris:
- "/"
load_profile:
load_type: rps
schedule: const(5000, 320s) удержание 5000 rps в течение 320 сек
instances: 5000
header_http: "1.1"
headers:
- "[Host: her.your.ru]"
- "[Connection: keep-alive]"
- "[User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36]"
uploader: #модуль для постройки графика
enabled: true
operator: my-username
package: yandextank.plugins.DataUploader
token_file: token.txt
console:
enabled: true
telegraf:
enabled: false
start
docker run --rm -v /opt/docker/yandex.tank/her.your.ru:/var/loadtest -it direvius/yandex-tank
testing is done
check and saw this problem: threads not const and not same RPS.
Why exist this problem? what check on host where I run testing
(4CPU, 16Gb: during testing utilization CPUs less 20%)
and how solve this problem?
Related
My Playwright UI tests (which use page, not request) work fine. Only the API tests are failing. For example const response = await request.get('https://reqres.in/api/users/3')
I found that it must be the corporate proxy by leaving the corporate wifi, joining my phone's hotspot, and seeing the tests pass. I've played with VSCode proxy settings but have not found one that fixes the timeout. It also fails when I run the test script directly from the root directory with npx playwright test --config=api.config.ts so I don't think it's VSCode.
The error message (containing my code) is below, the problem comes from the get method. The script times out there, and the code after it is never attempted. All API methods, such as get, post delete time out similarly.
1) [chromium] › api.spec.ts:6:10 › API Testing › Simple API Test - Assert Response Status ========
apiRequestContext.get: read ETIMEDOUT
=========================== logs ===========================
→ GET https://reqres.in/api/users/3
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.5195.19 Safari/537.36
accept: */*
accept-encoding: gzip,deflate,br
============================================================
5 |
6 | test.only('Simple API Test - Assert Response Status', async ({ request }) => {
> 7 | const response = await request.get('https://reqres.in/api/users/3')
| ^
8 | console.log(response)
9 | //expect(response.status()).toBe(200)
10 |
at /Users/USERNAME/playwright-framework/tests/api/api.spec.ts:7:40
Solved. The proxy is set within the Playwright config file, along with the rest of the more common configuration items. If required, username and password can be set here too.
For example:
use: {
headless: false,
viewport: { width: 1280, height: 720 },
actionTimeout: 10000,
screenshot: 'only-on-failure',
ignoreHTTPSErrors: true,
proxy: { server: 'http://yourproxy.com:8080/' },
trace: 'off',
video: 'off'
},
When I use python to carry a cookie to send a simulated request to
https://www.amazon.com/gp/delivery/ajax/address-change.html
a response of \n\n will be returned. But when I use Charles as the middleman agent, the same http message responds normally.
Similarly, when you use nodejs to send a simulation request, you can also get a normal response. I tried three different network request libraries in python, requests, httpx, and aiohttp got the same result.
For the response of \n\n, I locked the problem on the tls handshake package requested by python. After modifying urllib3.util.ssl_.DEFAULT_CIPHERS, it still returns a response of \n\n.
After comparison with Wireshark captures, it is found that in addition to the CIPHERS part, the Signature Algorithm part is also fixed, and the Signature Algorithm parts of the three request libraries are the same, with the way of curl, the way of nodejs, the way of Charles, and the way of Chrome to obtain the TSL Client Hello package. It's not the same.
I want to simulate the Signature Algorithm part of the Python TSL Client Hello package into Chrome. After I DEBUG the source code of the python request library, I found that the SSL Signature Algorithm part control seems to exist in the openssl so file.
This problem is It has troubled me for a long time, hope it can be resolved, thank you very much
import requests
from aiohttp import ClientSession
import httpx
cookies = {
'csm-hit': 'tb:s-B8ZK0QTPQCGWKHY3QDT5|1620287052879&t:1620287054928&adb:adblk_no',
'i18n-prefs': 'USD',
'lc-main': 'en_US',
'session-id': '143-0501748-3847056',
'session-id-time': '2082787201l',
'session-token': 'NxLWWkB7RnpUvmQEl7OcUzk44D9PnlSt/swrqvnSwBvry9WAPSeQt5U2hVCa7IeEEDwj+qzLHwrNhCnA+7pN8H7HELP5WYZuPjtTJ1d8jrTxLueLIQB+wh+3e+1c1vRrfYDa4FTsdm6jN2QR55zq0ybhNJt0jrXCTdlaktZ+e0tHPIjQnCsu1lidMvyOksR+',
'skin': 'noskin',
'sp-cdn': 'L5Z9:CN',
'ubid-main': '134-5202210-0613519',
}
headers = {
'Host': 'www.amazon.com',
'user-agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4094.1',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'accept-language': 'zh-CN,zh;q=0.9',
'anti-csrftoken-a2z': 'gBtJDelwICZ60r+pGBgwbzjAf4Wr+LTRIoyWRyMAAAAMAAAAAGC1xeJyYXcAAAAA',
'content-type': 'application/x-www-form-urlencoded;charset=utf-8',
}
data = 'locationType=LOCATION_INPUT&zipCode=90001&storeContext=generic&deviceType=web&pageType=Gateway&actionSource=glow&almBrandId=undefined'
url = 'https://www.amazon.com/gp/delivery/ajax/address-change.html'
# url = 'https://www.python-spider.com/nginx'
your_proxy_url = 'http://127.0.0.1:8888'
# your_proxy_url = ''
#
with httpx.Client(
# http2=True,
# proxies=your_proxy_url,
verify=False) as client:
# This HTTP request will be tunneled instead of forwarded.
response = client.post(url=url, headers=headers, cookies=cookies, data=data)
print(response.status_code)
print(response.text)
# cert='/Users/yangyanhui/lbs/spider/amazon/amazon_cookie_pool/charles-ssl-proxying-certificate.pem'
response = requests.post(url, headers=headers, cookies=cookies, data=data)
print(response.status_code)
print(response.text)
import aiohttp, asyncio
# asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) # 加上这一行
async def main(): # aiohttp必须放在异步函数中使用
async with ClientSession(cookies=cookies, headers=headers) as session:
async with session.post(url, data=data,
# proxy=your_proxy_url,
verify_ssl=False) as resp:
print(await resp.text())
print(resp.status)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
curl -H 'Host: www.amazon.com' -H 'Cookie: csm-hit=tb:s-B8ZK0QTPQCGWKHY3QDT5|1620287052879&t:1620287054928&adb:adblk_no; i18n-prefs=USD; lc-main=en_US; session-id=143-0501748-3847056; session-id-time=2082787201l; session-token=NxLWWkB7RnpUvmQEl7OcUzk44D9PnlSt/swrqvnSwBvry9WAPSeQt5U2hVCa7IeEEDwj+qzLHwrNhCnA+7pN8H7HELP5WYZuPjtTJ1d8jrTxLueLIQB+wh+3e+1c1vRrfYDa4FTsdm6jN2QR55zq0ybhNJt0jrXCTdlaktZ+e0tHPIjQnCsu1lidMvyOksR+; skin=noskin; sp-cdn=L5Z9:CN; ubid-main=134-5202210-0613519' -H 'user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 UBrowser/6.2.4094.1' -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' -H 'accept-language: zh-CN,zh;q=0.9' -H 'anti-csrftoken-a2z: gBtJDelwICZ60r+pGBgwbzjAf4Wr+LTRIoyWRyMAAAAMAAAAAGC1xeJyYXcAAAAA' -H 'content-type: application/x-www-form-urlencoded;charset=utf-8' --data-binary "locationType=LOCATION_INPUT&zipCode=90001&storeContext=generic&deviceType=web&pageType=Gateway&actionSource=glow&almBrandId=undefined" --compressed 'https://www.amazon.com/gp/delivery/ajax/address-change.html'
maybe consider using golang which has a package can modify your tls fingerprint
I tried making a get call with axios from my Vue js codebase/ environment to Jenkins API and I'm unable to do so.
I've read every resource that I could but wasn't able to fix this particular problem. I even created a .htaccess file to see if it help but wasn't useful.I ran out of options so I came here for help.
Below are the axios codes that I used within my App.vue file.
axios.get(
*URL to access Jenkins that is currently running on a tomcat server*,
{
headers: {
"jenkins-crumb": "* Some numbers and letters*",
},
auth: {
username: "*obvious username*",
password: "*obvious password*"
},
withCredentials: true,
crossdomain: true
}
)
.then(response => (this.info= response)).catch(error => (console.log(error)));
Console log output:
Access to XMLHttpRequest at 'url' from origin 'http://localhost:8080' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Network output:
General
Request URL: URL
Request Method: OPTIONS
Status Code: 403
Remote Address: localhost:8080
Referrer Policy: no-referrer-when-downgrade
Request Headers
Provisional headers are shown
Access-Control-Request-Headers: authorization,jenkins-crumb
Access-Control-Request-Method: GET
Origin: http://localhost:8080
Referer: http://localhost:8080/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
Please help!
I am using phantomjs with mink:
default:
extensions:
Behat\MinkExtension\Extension:
goutte: ~
selenium2:
browser: phantomjs
wd_host: http://localhost:8643/wd/hub
capabilities:
webStorageEnabled: true
But I need to masquerade as the latest chrome. I have tried this:
/**
* #BeforeStep
*/
public function masqueradeAsLatestChrome(StepEvent $event)
{
$this->getSession()->setRequestHeader('User-Agent', 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36');
}
But I get the exception:
[Behat\Mink\Exception\UnsupportedDriverActionException]
Exception has been thrown in "beforeStep" hook, defined in FeatureContext::masqueradeAsLatestChrome()
Request header is not supported by Behat\Mink\Driver\Selenium2Driver
The version of chrome isn't critical but the web application must think its talking to a very recent version of chrome.
Selenium does not provide this capability, as it is not something a
user can do. It's recommended you use a proxy to inject additional
headers to the requests generated by the browser.
https://code.google.com/p/selenium/issues/detail?id=2047#c1
Sadly… However, the PhantomJS does provide an interface for setting the headers. Your best shot would be to send a direct command to it using it's REST API. There's also a cool PHP wrap library that would make it 200 times easier.
You should use the new Behat/Mink driver made by Juan Francisco Calderón Zumba
https://github.com/jcalderonzumba
Here is a direct link to the driver https://github.com/jcalderonzumba/MinkPhantomJSDriver
This driver allows you to specify the request headers that you need
(It works with Behat 3.0 but I think it requires at least PHP 5.4)
you can pass additional settings in selenium2driver via
extra_capabilities:
so in your case:
default:
extensions:
Behat\MinkExtension\Extension:
goutte: ~
selenium2:
browser: phantomjs
wd_host: http://localhost:8643/wd/hub
capabilities:
webStorageEnabled: true
extra_capabilities:
phantomjs.page.settings.userAgent: "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36"
For https://github.com/facebook/php-webdriver you can use this case:
$capabilities = [
WebDriverCapabilityType::BROWSER_NAME => 'phantomjs',
WebDriverCapabilityType::PLATFORM => 'ANY',
WebDriverCapabilityType::ACCEPT_SSL_CERTS => false,
WebDriverCapabilityType::JAVASCRIPT_ENABLED => true,
'phantomjs.page.settings.userAgent' => "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246",
];
I'm using winston to stream log messages from Express based on various comments elsewhere, my setup is essentially:
var express = require("express"),
winston = require("winston");
// enable web server logging; pipe those log messages through winston
var requestLogger = new (winston.Logger)(
{
transports: [
new (winston.transports.File)(
{
filename: "logs/request.log",
json: false,
timestamp: false
}
)
]
}
),
winstonStream = {
write: function(message, encoding) {
requestLogger.info(message.replace(/(\r?\n)$/, ''));
}
};
this.use(express.logger({stream: winstonStream}));
But I'd like a way to suppress the output of the log level because I know for this particular logger it will always be "info". So rather than:
info: 127.0.0.1 - - [Fri, 20 Sep 2013 13:48:02 GMT] "POST /v1/submission HTTP/1.1" 200 261 "http://localhost:8887/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.65 Safari/537.36"
I would get:
127.0.0.1 - - [Fri, 20 Sep 2013 13:48:02 GMT] "POST /v1/submission HTTP/1.1" 200 261 "http://localhost:8887/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.65 Safari/537.36"