Phantom js won't open any pages - phantomjs

I'm trying to make Phantom JS load a page, but no matter what, it responds with an "operation canceled :( I've tried various versions settings that ignore ssl and various websites. But neither works.
I can reach all sites with a normal browser and run PhantomJS locally from a fully updated mac osx
./phantomjs --ignore-ssl-errors=true --ignore-ssl-errors=yes --ssl-protocol=tlsv1 ../../index.js http://askbar.dk
yelds
= onNavigationRequested
destination_url: http://askbar.dk/
type (cause): Other
will navigate: true
from page's main frame: true
= onResourceRequested()
request: {
"headers": [
{
"name": "Accept",
"value": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
},
{
"name": "User-Agent",
"value": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"
}
],
"id": 1,
"method": "GET",
"time": "2016-04-11T20:05:39.998Z",
"url": "http://askbar.dk/"
}
= onNavigationRequested
destination_url: about:blank
type (cause): Other
will navigate: true
from page's main frame: true
http://askbar.dk/: Operation canceled
phantomjs://code/index.js:60 in onResourceError
= onResourceReceived()
id: 1, stage: "end", response: {"contentType":null,"headers":[],"id":1,"redirectURL":null,"stage":"end","status":null,"statusText":null,"time":"2016-04-11T20:05:39.999Z","url":"http://askbar.dk/"}
Any ideas ?

Related

Cypress API Call Forbidden 403 Error ( "x-frame-options": "SAMEORIGIN, DENY" )

Issue :
I successfully Login and make a Post Call and receive for following
error after making a cy.request( )POST API call :
I was able to make cy.request( ) get API calls which retured 200 , so API call error seems specific to the above cy.request( ) POST API call
Error:
The response we got was:
Status: 403 - Forbidden
Headers: {
"date": "Thu, 19 Dec 2019 16:10:19 GMT",
"server": "Apache",
"x-frame-options": "SAMEORIGIN, DENY",
Full Error
CypressError: cy.request() failed on:
https://uat1.dns2.merchantportal.eu/MerchantApiWeb/rest/secure/user/financial/preauthorisation
The response we received from your web server was:
403: Forbidden
This was considered a failure because the status code was not '2xx' or '3xx'.
If you do not want status codes to cause failures pass the option: 'failOnStatusCode: false'
The request we sent was:
Method: POST
URL: https://uat1.dns2.merchantportal.eu/MerchantApiWeb/rest/secure/user/financial/preauthorisation
Headers: {
"Connection": "keep-alive",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36",
"accept": "/",
"Cookie": "NSC_JOfmamujd2zndlkdwlzogneusels0bQ=ffffffff09405a8245525d5f4f58455e445a4a42378b; ROUTEID=.1; JSESSIONID=mu_x-VvVK3_rycxkJ9eef9w7lUPzw-SFLqiyXZnd.r5cvap1013.1dc.com.MP.WEB",
"accept-encoding": "gzip, deflate",
"content-type": "application/json",
"content-length": 101
}
Body: {"fromDate":"2019-12-19","toDate":"2019-12-19","pageNumber":1,"sortBy":"preAuthDate","sortOrder":"D"}
The response we got was:
Status: 403 - Forbidden
Headers: {
"date": "Thu, 19 Dec 2019 16:10:19 GMT",
"server": "Apache",
"x-frame-options": "SAMEORIGIN, DENY",
"strict-transport-security": "max-age=15552000; includeSubdomains; preload, max-age=31536000 ; includeSubDomains",
"expires": "0",
"cache-control": "no-cache, no-store, max-age=3600, must-revalidate",
"x-xss-protection": "1; mode=block",
"pragma": "no-cache",
"x-content-type-options": "nosniff",
"content-type": "application/json;charset=ISO-8859-1",
"content-security-policy": "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://cdnjs.cloudflare.com https://*.tiqcdn.com https://www.googletagmanager.com https://www.google-analytics.com; connect-src 'self' https://www.google-analytics.com/; img-src 'self' data: https://www.google-analytics.com https:; style-src 'self' 'unsafe-inline' blob:; font-src 'self' blob:; frame-src 'self' data: *; frame-ancestors 'self'; media-src *;",
"x-powered-by": "",
"keep-alive": "timeout=10",
"connection": "Keep-Alive",
"transfer-encoding": "chunked"
}
Body: {
"code": "MPERR08",
"message": "You are not privileged to request this resource.",
"access-denied": true,
"cause": "AUTHORIZATION_FAILURE"
}
Error I Recieved - Click to see

How to find who deletes a table in the logs

I have a dataset ID, tables were created. Which were deleted. I need to check who deleted them and when.
The easiest way would be in the GCP console click in the "ACTIVITY" tab, filter the "Resource Type" by "Big Query" and look for the "Delete Table" entry. If you click the entry it'll expand and show info about the deletion, along with the account that performed the deletion.
You need to enable auditlogs export.
There are many types of deletes:
delete from UI / API
delete with QUERY (overwrite as destination table)
delete via CREATE OR REPLACE syntax
autodelete done via expiration
Common type of deletes are explained here:
Type 1: you issued a table deleted Query/API call etc..
Then you can run a query like:
SELECT * FROM `dataset.cloudaudit_googleapis_com_activity_20190919`
where resource.type='bigquery_resource'
and protopayload_auditlog.methodName='tableservice.delete'
you get back a large table which cannot be posted here as a nice image, but as simplified JSON it is here:
[
{
"logName": "projects/editedname/logs/cloudaudit.googleapis.com%2Factivity",
"resource": {
"type": "bigquery_resource",
"labels": {
"project_id": "editedname",
}
},
"protopayload_auditlog": {
"serviceName": "bigquery.googleapis.com",
"methodName": "tableservice.delete",
"resourceName": "projects/editedname/datasets/dataset/tables/industry2",
"authenticationInfo": {
"principalEmail": "something#domain.com",
"authoritySelector": null,
"serviceAccountKeyName": null,
"serviceAccountDelegationInfo": []
},
"authorizationInfo": [
{
"resource": "projects/editedname/datasets/dataset/tables/industry2",
"permission": "bigquery.tables.delete",
"granted": "true",
"resourceAttributes": null
}
],
"requestMetadata": {
"callerIp": "1.2.3.4",
"callerSuppliedUserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.75 Safari/537.36,gzip(gfe)",
"callerNetwork": null,
"requestAttributes": null,
"destinationAttributes": null
},
},
"timestamp": "2019-09-19 08:47:00.381 UTC",
"receiveTimestamp": "2019-09-19 08:47:00.590316 UTC",
"severity": "NOTICE",
}
]
As you can see you have the table that was deleted eg: industry2 in my example, also the user's or service account email address, and the date, even metadata of the caller by IP, and userAgent.
More about the available information here
Type 2: You issued a table override query or a CREATE OR REPLACE syntax
For these kind of "deletes" you won't find a separate deleted entry. But you can look for the "truncated" flag in the metadataJson

Traefik doesn't route to proper backend for some containers

Some of my frontends aren't routed properly to backend.
I do have 2 container:
- portainer;
- homeassistant;
Portainer works properly but homeassistant is not routed properly to backend.
my docker-compose:
version: "3.6"
services:
portainer:
image: portainer/portainer
container_name: portainer
domainname: ${DOMAINNAME}
hostname: portainer
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ${USERDIR}/dockerha/uavportainer/data:/data
- ${USERDIR}/dockerha/shared:/shared
environment:
- TZ=${TZ}
networks:
- ${NETWORK_NAME}
labels:
- "traefik.enable=true"
- "traefik.frontend.rule=Host:${DOMAINNAME}; PathPrefixStrip: /portainer"
- "traefik.backend=portainer"
- "traefik.port=9000"
- "traefik.frontend.entryPoints=http"
homeassistant:
image: homeassistant/home-assistant
container_name: homeassistant
domainname: ${DOMAINNAME}
hostname: homeassistant
volumes:
- ${USERDIR}/dockerha/uavha:/config
- /etc/localtime:/etc/localtime:ro
- ${USERDIR}/dockerha/shared:/shared
ports:
- "8123:8123"
environment:
- PUID=${PUID}
- PGID=${PGID}
- TZ=${TZ}
networks:
- ${NETWORK_NAME}
labels:
- "traefik.enable=true"
- "traefik.frontend.rule=Host:${DOMAINNAME};PathPrefixStrip:/hass"
- "traefik.backend=hass"
- "traefik.port=8123"
- "traefik.frontend.entryPoints=http"
toml file:
[accessLog]
filepath = "/etc/traefik/access.log"
format = "json"
debug = true
logLevel = "DEBUG" #"ERROR" #DEBUG, INFO, WARN, ERROR, FATAL, PANIC
#InsecureSkipVerify = true
defaultEntryPoints = ["http", "https"]
# WEB interface of Traefik - it will show web page with overview of frontend an$
[api]
entryPoint = "traefik"
dashboard = true
address = ":8080"
# usersFile = "/shared/.htpasswd"
# Force HTTPS
[entryPoints]
[entryPoints.http]
address = ":80"
[file]
watch = true
[docker]
Endpoint = "unix:///var/run/docker.sock"
Domain = "192.168.56.102"
watch = true
SwarmMode = true
network = "uavhanet"
ExposedRyDefault = false
The below are 2 different access logs:
working one:
{
"BackendAddr": "10.0.14.11:9000",
"BackendName": "backend-portainer",
"BackendURL": {
"Scheme": "http",
"Opaque": "",
"User": null,
"Host": "10.0.14.11:9000",
"Path": "",
"RawPath": "",
"ForceQuery": false,
"RawQuery": "",
"Fragment": ""
},
"ClientAddr": "192.168.56.1:58089",
"ClientHost": "192.168.56.1",
"ClientPort": "58089",
"ClientUsername": "-",
"DownstreamContentSize": 3,
"DownstreamStatus": 200,
"DownstreamStatusLine": "200 OK",
"Duration": 3927995,
"FrontendName": "Host-192-168-56-102-PathPrefixStrip-portainer-4",
"OriginContentSize": 3,
"OriginDuration": 3468375,
"OriginStatus": 200,
"OriginStatusLine": "200 OK",
"Overhead": 459620,
"RequestAddr": "192.168.56.102",
"RequestContentSize": 0,
"RequestCount": 63,
"RequestHost": "192.168.56.102",
"RequestLine": "GET /portainer/api/teams HTTP/1.1",
"RequestMethod": "GET",
"RequestPath": "/portainer/api/teams",
"RequestPort": "-",
"RequestProtocol": "HTTP/1.1",
"RetryAttempts": 0,
"StartLocal": "2019-02-19T12:45:44.966258565Z",
"StartUTC": "2019-02-19T12:45:44.966258565Z",
"downstream_Content-Length": "3",
"downstream_Content-Type": "application/json",
"downstream_Date": "Tue, 19 Feb 2019 12:45:44 GMT",
"downstream_X-Content-Type-Options": "nosniff",
"downstream_X-Xss-Protection": "1; mode=block",
"level": "info",
"msg": "",
"origin_Content-Length": "3",
"origin_Content-Type": "application/json",
"origin_Date": "Tue, 19 Feb 2019 12:45:44 GMT",
"origin_X-Content-Type-Options": "nosniff",
"origin_X-Xss-Protection": "1; mode=block",
"request_Accept": "application/json, text/plain, */*",
"request_Accept-Encoding": "gzip, deflate",
"request_Accept-Language": "en-US,en;q=0.9",
"request_Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwidXNlcm5hbWUiOiJhZG1pbiIsInJvbGUiOjEsImV4cCI6MTU1MDU5ODIzMn0.nOzaZYo7gu3An_nHZMpCqPnQ0XNFEvEfrivZ40RumUI",
"request_Connection": "keep-alive",
"request_Cookie": "_ga=GA1.1.364506726.1550080124; _gid=GA1.1.376969405.1550080124; portainer.UI_STATE=%7B%22dismissedInfoPanels%22%3A%7B%7D%2C%22dismissedInfoHash%22%3A%22y%2BAiiJOM%2Feqr4gyAl6VKBw%3D%3D%22%7D; pma_lang=en; portainer.pagination_containers=25; portainer.pagination_images=25; _gat=1",
"request_Referer": "http://192.168.56.102/portainer/",
"request_User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36",
"request_X-Forwarded-Prefix": "/portainer",
"time": "2019-02-19T12:45:44Z"
}
not working properly:
{
"BackendAddr": "",
"BackendName": "Traefik",
"BackendURL": {
"Scheme": "",
"Opaque": "",
"User": null,
"Host": "",
"Path": "/frontend_latest/hass-icons-8b13a1ef.js",
"RawPath": "",
"ForceQuery": false,
"RawQuery": "",
"Fragment": ""
},
"ClientAddr": "192.168.56.1:58071",
"ClientHost": "192.168.56.1",
"ClientPort": "58071",
"ClientUsername": "-",
"DownstreamContentSize": 19,
"DownstreamStatus": 404,
"DownstreamStatusLine": "404 Not Found",
"Duration": 100373,
"FrontendName": "backend not found",
"OriginContentSize": 19,
"OriginDuration": 13969,
"OriginStatus": 404,
"OriginStatusLine": "404 Not Found",
"Overhead": 86404,
"RequestAddr": "192.168.56.102",
"RequestContentSize": 0,
"RequestCount": 78,
"RequestHost": "192.168.56.102",
"RequestLine": "GET /frontend_latest/hass-icons-8b13a1ef.js HTTP/1.1",
"RequestMethod": "GET",
"RequestPath": "/frontend_latest/hass-icons-8b13a1ef.js",
"RequestPort": "-",
"RequestProtocol": "HTTP/1.1",
"RetryAttempts": 0,
"StartLocal": "2019-02-19T12:46:54.875219779Z",
"StartUTC": "2019-02-19T12:46:54.875219779Z",
"downstream_Content-Type": "text/plain; charset=utf-8",
"downstream_X-Content-Type-Options": "nosniff",
"level": "info",
"msg": "",
"origin_Content-Type": "text/plain; charset=utf-8",
"origin_X-Content-Type-Options": "nosniff",
"request_Accept": "*/*",
"request_Accept-Encoding": "gzip, deflate",
"request_Accept-Language": "en-US,en;q=0.9",
"request_Connection": "keep-alive",
"request_Cookie": "_ga=GA1.1.364506726.1550080124; _gid=GA1.1.376969405.1550080124; portainer.UI_STATE=%7B%22dismissedInfoPanels%22%3A%7B%7D%2C%22dismissedInfoHash%22%3A%22y%2BAiiJOM%2Feqr4gyAl6VKBw%3D%3D%22%7D; pma_lang=en; portainer.pagination_containers=25; portainer.pagination_images=25",
"request_Referer": "http://192.168.56.102/hass/",
"request_User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36",
"time": "2019-02-19T12:46:54Z"
}
What's wrong ?
Why I get a 404 and not the a 200 ?
Thank you in advance.
In the second case, you have this "RequestLine": "GET /frontend_latest/hass-icons-8b13a1ef.js HTTP/1.1", this doesn't match any PathPrefixStrip, if when you go to /IP/hass you redirect to /frontend_latest/ using the traefik ip, from my point of view gets a 404, because traefik doesn't have any rule for /frontend_latest/

Set user agent in PhantomJS and Protractor

I am using Protractor with PhantomJS. How can I set the user agent that PhantomJS sends? Can it be different for different tests?
Set the phantomjs.page.settings.userAgent under capabilities:
capabilities: {
"browserName": "phantomjs",
"phantomjs.page.settings.userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/53 (KHTML, like Gecko) Chrome/15.0.87"
}

How to set a user agent and use a proxy?

The documents read as follows:
user-agent
since v0.3.7
(string) Override the User-Agent header in HTTP requests made from the application.
The following placeholders are available:
%name: replaced by the name field in the manifest.
%ver: replaced by the version field in the manifest, if available.
%nwver: replaced by the version of node-webkit.
%webkit_ver: replaced by the version of WebKit engine.
%osinfo: replaced by the OS and CPU information you would see in browser's user agent string.
Can someone give me an example of how this implemented? eg. Current user agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 ...
Say I want to make it:
Chrome/5.0 (Windows NT) Chrome/35.0.1916 ...
Also, I'd like to use a proxy... is that possible within node-webkit?
User-Agent: In the package.json:
"name":"TheNameOfTheApp",
"version":"0.0.1",
"private":true,
"main":"index.html",
"dependencies":{
"express":"latest",
...
}
"window": {
"title": "MyApp",
"toolbar": true,
"frame": true,
"position": "center",
"fullscreen": false,
"resizable": true,
"width": 1000,
"height": 600,
"min_width": 600,
"min_height": 400
},
"user-agent":"%name NodeWebKit", //-> "TheNameOfTheApp NodeWebKit"
"node-remote": "<local>"