S3 backend: can't overwrite AWS_S3_ENDPOINT - amazon-s3

I want to use an S3 backend to remotely store my tfstate.
Since I want to use a bucket on Wasabi, and not AWS, I set the endpoint to s3.wasabisys.com.
However, terraform still tries to use AWS.
I tried to use the TF_AWS_S3_ENDPOINT environment variable, but even hardcoded values don't work:
terraform {
backend "s3" {
bucket = "my-bucket"
key = "my-key"
region = "us-east-1"
endpoint = "s3.wasabisys.com"
access_key = "xxxx"
secret_key = "xxxx"
}
}
Output:
stanislas#mbp ~> terraform init
2018/10/25 08:53:35 [INFO] Terraform version: 0.11.10
2018/10/25 08:53:35 [INFO] Go runtime version: go1.11.1
2018/10/25 08:53:35 [INFO] CLI args: []string{"/usr/local/Cellar/terraform/0.11.10/bin/terraform", "init"}
2018/10/25 08:53:35 [DEBUG] Attempting to open CLI config file: /Users/stanislas/.terraformrc
2018/10/25 08:53:35 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2018/10/25 08:53:35 [INFO] CLI command args: []string{"init"}
2018/10/25 08:53:35 [DEBUG] command: loading backend config file: /Users/stanislas/git/xxx
2018/10/25 08:53:35 [DEBUG] command: no data state file found for backend config
Initializing the backend...
2018/10/25 08:53:35 [DEBUG] New state was assigned lineage "be941477-7111-2a13-ceed-55e4fba0bcbd"
2018/10/25 08:53:35 [INFO] Building AWS region structure
2018/10/25 08:53:35 [INFO] Building AWS auth structure
2018/10/25 08:53:35 [INFO] Setting AWS metadata API timeout to 100ms
2018/10/25 08:53:36 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id
2018/10/25 08:53:36 [INFO] AWS Auth provider used: "StaticProvider"
2018/10/25 08:53:36 [INFO] Initializing DeviceFarm SDK connection
2018/10/25 08:53:36 [DEBUG] [aws-sdk-go] DEBUG: Request sts/GetCallerIdentity Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: sts.amazonaws.com
User-Agent: aws-sdk-go/1.14.31 (go1.11.1; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.11.10
Content-Length: 43
Authorization: AWS4-HMAC-SHA256 Credential=xxx, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=xxx
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20181025T065336Z
Accept-Encoding: gzip
Action=GetCallerIdentity&Version=2011-06-15
-----------------------------------------------------
2018/10/25 08:53:36 [DEBUG] [aws-sdk-go] DEBUG: Response sts/GetCallerIdentity Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
Connection: close
Content-Length: 306
Content-Type: text/xml
Date: Thu, 25 Oct 2018 06:53:36 GMT
X-Amzn-Requestid: xxx
-----------------------------------------------------
2018/10/25 08:53:36 [DEBUG] [aws-sdk-go] <ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<Error>
<Type>Sender</Type>
<Code>InvalidClientTokenId</Code>
<Message>The security token included in the request is invalid.</Message>
</Error>
<RequestId>xxx</RequestId>
</ErrorResponse>
2018/10/25 08:53:36 [DEBUG] [aws-sdk-go] DEBUG: Validate Response sts/GetCallerIdentity failed, not retrying, error InvalidClientTokenId: The security token included in the request is invalid.
status code: 403, request id: xxx
2018/10/25 08:53:36 [DEBUG] plugin: waiting for all plugin processes to complete...
Error configuring the backend "s3": InvalidClientTokenId: The security token included in the request is invalid.
status code: 403, request id: xxx
Please update the configuration in your Terraform files to fix this error
then run this command again.
As you can see, Terraform sends a request to sts.amazonaws.com.
Am I missing something?

Have you tried to use the set the following options to true ?
skip_requesting_account_id
skip_credentials_validation
skip_get_ec2_platforms
skip_metadata_api_check
As those seems to be needed when not using AWS (https://github.com/hashicorp/terraform/pull/15553#issuecomment-383294678).

Related

Setting Terraform env variables correctly for Azure resources

I have a GitHub Actions workflow that includes this part for Terraform;
terraform:
name: 'Terraform'
needs: build
runs-on: ubuntu-latest
# Add env variables for service principal
env:
TF_LOG: TRACE
ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
steps:
- name: 'Checkout'
uses: actions/checkout#v2
- name: 'Azure CLI'
uses: azure/login#v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- run: |
az account show
- name: 'Terraform Setup'
uses: hashicorp/setup-terraform#v1
with:
cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
- name: 'Terraform Init'
run: terraform init
- name: 'Terraform Format'
run: terraform fmt -check
- name: 'Terraform Plan'
run: terraform plan
- name: Terraform Apply
if: github.ref == 'refs/heads/develop' && github.event_name == 'push'
run: terraform apply -auto-approve
And I have the following .tf files;
main.tf
provider "azurerm" {
version = "=2.5.0"
features {}
}
variables.tf
variable "subscription_id" {
description = "The Azure subscription ID."
}
variable "client_id" {
description = "The Azure Service Principal app ID."
}
variable "client_secret" {
description = "The Azure Service Principal password."
}
variable "tenant_id" {
description = "The Azure Tenant ID."
}
terraform.tfvars
subscription_id = "2d0bd.."
client_id = "hl4kj..."
client_secret = "kj2ee..."
tenant_id = "f9cc2..."
And I also have the following secrets set in secrets section of the repository;
ARM_CLIENT_ID
ARM_CLIENT_SECRET
ARM_SUBSCRIPTION_ID
ARM_TENANT_ID
When I run the workflow I get the following log and error, terraform plan gets stuck;
/home/runner/work/_temp/cd8bfc2c-354b-41a4-9d10-f1ed7183c439/terraform-bin plan ./terraform
2020/11/05 13:49:02 [INFO] Terraform version: 0.13.5
2020/11/05 13:49:02 [INFO] Go runtime version: go1.14.7
2020/11/05 13:49:02 [INFO] CLI args: []string***"/home/runner/work/_temp/cd8bfc2c-354b-41a4-9d10-f1ed7183c439/terraform-bin", "plan", "./terraform"***
2020/11/05 13:49:02 [DEBUG] Attempting to open CLI config file: /home/runner/.terraformrc
2020/11/05 13:49:02 Loading CLI configuration from /home/runner/.terraformrc
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /home/runner/.terraform.d/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /home/runner/.local/share/terraform/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2020/11/05 13:49:02 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2020/11/05 13:49:02 [INFO] CLI command args: []string***"plan", "./terraform"***
2020/11/05 13:49:02 [TRACE] Meta.Backend: built configuration for "azurerm" backend with hash value 4172574508
2020/11/05 13:49:02 [TRACE] Preserving existing state lineage "2214372f-9818-d87e-197a-ef8533e8fa6e"
2020/11/05 13:49:02 [TRACE] Preserving existing state lineage "2214372f-9818-d87e-197a-ef8533e8fa6e"
2020/11/05 13:49:02 [TRACE] Meta.Backend: working directory was previously initialized for "azurerm" backend
2020/11/05 13:49:02 [TRACE] Meta.Backend: using already-initialized, unchanged "azurerm" backend configuration
2020/11/05 13:49:02 [TRACE] Meta.Backend: instantiated backend of type *azure.Backend
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azuread v0.7.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/azuread/0.7.0/linux_amd64
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azurerm v2.5.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/azurerm/2.5.0/linux_amd64
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/helm v1.3.2 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/helm/1.3.2/linux_amd64
2020/11/05 13:49:02 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/random v3.0.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/random/3.0.0/linux_amd64
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/random/3.0.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/random 3.0.0
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/azuread/0.7.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/azuread 0.7.0
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/azurerm/2.5.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/azurerm 2.5.0
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/helm/1.3.2/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/helm 1.3.2
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/11/05 13:49:02 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/11/05 13:49:02 [DEBUG] checking for provisioner in "."
2020/11/05 13:49:02 [DEBUG] checking for provisioner in "/home/runner/work/_temp/cd8bfc2c-354b-41a4-9d10-f1ed7183c439"
2020/11/05 13:49:02 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2020/11/05 13:49:02 [TRACE] Meta.Backend: backend *azure.Backend does not support operations, so wrapping it in a local backend
2020/11/05 13:49:02 [INFO] backend/local: starting Plan operation
2020/11/05 13:49:02 [TRACE] backend/local: requesting state manager for workspace "default"
2020/11/05 13:49:02 [TRACE] backend/local: requesting state lock for workspace "default"
2020/11/05 13:49:02 [DEBUG] Azure Backend Request:
HEAD /tstate/terraform.tfstate HTTP/1.1
Host: tstateidentity15466.blob.core.windows.net
User-Agent: Terraform/0.13.5
X-Ms-Date: Thu, 05 Nov 2020 13:49:02 GMT
X-Ms-Version: 2018-11-09
Acquiring state lock. This may take a few moments...
2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate:
HTTP/1.1 200 OK
Content-Length: 978
Accept-Ranges: bytes
Content-Md5: qi87ZYbc9/fceVy/LIgnjQ==
Content-Type: application/json
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881909C5B3223"
Last-Modified: Thu, 05 Nov 2020 13:42:17 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Access-Tier: Hot
X-Ms-Access-Tier-Inferred: true
X-Ms-Blob-Type: BlockBlob
X-Ms-Creation-Time: Tue, 20 Oct 2020 11:48:51 GMT
X-Ms-Lease-State: broken
X-Ms-Lease-Status: unlocked
X-Ms-Meta-Terraformlockid: eyJJRCI6ImY3YWI2MTVlLTQ2MjItMDY5Yy00YjFiLWM4ZGNkM2ZiODg4ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI61bm5lckBmdi1hejEyMC0yMjQiLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVUMTM6NDI6MTYuNDkyMjUzOTiJ0c3RhdGUvdGVycmFmb3JtLnRmc3RhdGUi***
X-Ms-Request-Id: b9a10809-d01e-002f-5a-b3a500000
X-Ms-Server-Encrypted: true
X-Ms-Version: 2018-11-09
2020/11/05 13:49:03 [DEBUG] Azure Backend Request:
PUT /tstate/terraform.tfstate?comp=lease HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
Content-Length: 0
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Lease-Action: acquire
X-Ms-Lease-Duration: -1
X-Ms-Proposed-Lease-Id: 99a99396-3a95-215-693d-023e7f07f
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip
2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate?comp=lease:
HTTP/1.1 201 Created
Content-Length: 0
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881909C5B3223"
Last-Modified: Thu, 05 Nov 2020 13:42:17 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Lease-Id: 99a99396-3a95-2175-693d-01e823f07f
X-Ms-Request-Id: b9a10885-d01e-002f-3c7a-b3a5000000
X-Ms-Version: 2018-11-09
2020/11/05 13:49:03 [DEBUG] Azure Backend Request:
HEAD /tstate/terraform.tfstate HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Lease-Id: 99a99396-3a95-2175-693d-0823e7f07f
X-Ms-Version: 2018-11-09
2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate:
HTTP/1.1 200 OK
Content-Length: 978
Accept-Ranges: bytes
Content-Md5: qi87ZYbc9/fceVy/LIgnjQ==
Content-Type: application/json
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881909C5B3223"
Last-Modified: Thu, 05 Nov 2020 13:42:17 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Access-Tier: Hot
X-Ms-Access-Tier-Inferred: true
X-Ms-Blob-Type: BlockBlob
X-Ms-Creation-Time: Tue, 20 Oct 2020 11:48:51 GMT
X-Ms-Lease-Duration: infinite
X-Ms-Lease-State: leased
X-Ms-Lease-Status: locked
X-Ms-Meta-Terraformlockid: eyJJRCI6ImY3YWI2MTVlLTQ2MjItMY5Yy00YjFiLWM4ZGNkM2ZiODg4ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI6IinJ1bm5lckBmdi1hejEyMC0yMjQiLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVUMTM6NDI6MTYuNDkyMjUzOTk5WiIsIlBhdGgiOiJ0c3RhdGUvdGycmFmb3JtLnRmc3RhdGUi***
X-Ms-Request-Id: b9a108f2-d01e-002f-187a-b3a24000000
X-Ms-Server-Encrypted: true
X-Ms-Version: 2018-11-09
2020/11/05 13:49:03 [DEBUG] Azure Backend Request:
PUT /tstate/terraform.tfstate?comp=metadata HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
Content-Length: 0
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Lease-Id: 99a99396-3a95-2175-693d-01e823e7f07f
X-Ms-Meta-Terraformlockid: eyJJRCI6Ijk5YTk5Mzk2LTNhOTUtMjE3NS02OTNkLTAxZTgyM2U3ZjA3ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI6IiIsIlJ1bm5lckBmdi1hejE3NC0yMTciLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVUMTM6NDk6MDIuNzgzNDQI5WiIsIlBhdGgiOiJ0c3RhdGUvdGVycmFmb3JtLnRmc3RhdGUi***
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip
2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate?comp=metadata:
HTTP/1.1 200 OK
Content-Length: 0
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881918E9DEEFF"
Last-Modified: Thu, 05 Nov 2020 13:49:03 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Request-Id: b9a10962-d01e-002f-707a-b524000000
X-Ms-Request-Server-Encrypted: true
X-Ms-Version: 2018-11-09
2020/11/05 13:49:03 [TRACE] backend/local: reading remote state for workspace "default"
2020/11/05 13:49:03 [DEBUG] Azure Backend Request:
GET /tstate/terraform.tfstate HTTP/1.1
Host: tstateproject.blob.core.windows.net
User-Agent: Terraform/0.13.5
X-Ms-Date: Thu, 05 Nov 2020 13:49:03 GMT
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip
2020/11/05 13:49:03 [DEBUG] Azure Backend Response for https://tstateproject.blob.core.windows.net/tstate/terraform.tfstate:
HTTP/1.1 200 OK
Content-Length: 978
Accept-Ranges: bytes
Content-Md5: qi87ZYbc9/fceVy/LIgnjQ==
Content-Type: application/json
Date: Thu, 05 Nov 2020 13:49:02 GMT
Etag: "0x8D881918E9DEEFF"
Last-Modified: Thu, 05 Nov 2020 13:49:03 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
X-Ms-Blob-Type: BlockBlob
X-Ms-Creation-Time: Tue, 20 Oct 2020 11:48:51 GMT
X-Ms-Lease-Duration: infinite
X-Ms-Lease-State: leased
X-Ms-Lease-Status: locked
X-Ms-Meta-Terraformlockid: eyJJRCI6Ijk5YTk5Mzk2LTNhtMjE3NS02OTNkLTAxZTgyM2U3ZjA3ZiIsIk9wZXJhdGlvbiI6Ik9wZXJhdGlvblR5cGVQbGFuIiwiSW5mbyI6IiIsIldobyI6InJ1bm5lckBmdi1hejE3NC0yMTciLCJWZXJzaW9uIjoiMC4xMy41IiwiQ3JlYXRlZCI6IjIwMjAtMTEtMDVM6NDk6MDIuNzgzNDQwNjI5WiIsIlBhdGgiOiJ0c3RhdGUvdGVycmFmb3JtLnRmc3RhdGUi***
X-Ms-Request-Id: b9a109dd-d01e-002f-577a-b3a524000000
X-Ms-Server-Encrypted: true
X-Ms-Version: 2018-11-09
***
"version": 4,
"terraform_version": "0.13.5",
"serial": 12,
"lineage": "7f667e4-4407-c040-32ba-dce44bfda167",
"outputs": ***,
"resources": [
***
"mode": "managed",
"type": "azurerm_resource_group",
"name": "aks",
"provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
"instances": [
***
"schema_version": 0,
"attributes": ***
"id": "/subscriptions/***/resourceGroups/sociallme-k8s-rg",
"location": "westeurope",
"name": "project-k8s-rg",
"tags": ***
"env": "Dev project rg",
"source": "project"
***,
"timeouts": null
***,
"private": "...AwfX0="
***
]
***
]
***
2020/11/05 13:49:03 [TRACE] backend/local: retrieving local state snapshot for workspace "default"
2020/11/05 13:49:03 [TRACE] backend/local: building context for current working directory
2020/11/05 13:49:03 [DEBUG] backend/local: will prompt for input of unset required variables [subscription_id client_id client_secret tenant_id]
2020/11/05 13:49:03 [DEBUG] command: asking for input: "var.client_id"
var.client_id
The Azure Service Principal app ID.
Error: The operation was canceled.
What am I missing? Thanks!
The run is canceled because it is expecting input for var.client_id.
2020/11/05 13:49:03 [DEBUG] backend/local: will prompt for input of unset required variables [subscription_id client_id client_secret tenant_id]
2020/11/05 13:49:03 [DEBUG] command: asking for input: "var.client_id"
var.client_id
The Azure Service Principal app ID.
Error: The operation was canceled.
You suggest it is included with terraform.tfvars, but there is not indication it is read in. A lot of default .gitignore files for terraform ignore terraform.tfvars.
Further you are mixing your authentication method with both variable input and environment variables. The practice I follow is to store the secrets in github and use the environment. It is a security risk and considered bad practice to commit your credentials.
To fix your issues you can probably delete these [subscription_id client_id client_secret tenant_id] variables.

Gatling: WebSocketHandshakeException: Invalid handshake response getStatus: 400 Bad Request

I'm trying to open ws connection, but get the error:
val openConnection = exec( ws("Connect -> WS").wsName("user").connect("wss://socket.develop.test.com?access_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9"))
Then, I get:
09:49:53.343 [DEBUG] i.g.h.c.i.DefaultHttpClient - Installing SslHandler for wss://socket.develop.test.com?access_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9
09:49:53.381 [DEBUG] i.n.h.s.u.InsecureTrustManagerFactory - Accepting a server certificate: CN=Kubernetes Ingress Controller Fake Certificate, O=Acme Co
09:49:53.418 [DEBUG] i.n.h.c.h.w.WebSocketClientHandshaker13 - WebSocket version 13 client handshake key: YndmXwIGgZseWyRlmXBoyw==, expected response: H2YGNn6p+DyLyudnc1JCluHjj7E=
09:49:53.419 [DEBUG] i.g.h.c.i.WebSocketHandler - ctx.write msg=DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: EmptyByteBufBE)
GET ?access_token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9
HTTP/1.1
accept: */*
upgrade: websocket
connection: upgrade
sec-websocket-key: YndmXwIGgZseWyRlmXBoyw==
host: socket.develop.test.com
origin: https://socket.develop.test.com
sec-websocket-version: 13
09:49:53.423 [DEBUG] i.n.h.s.SslHandler - [id: 0xfa53644a, L:/192.168.150.134:54082 - R:socket.develop.test.com/78.47.16.48:443] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
09:49:53.450 [DEBUG] i.g.h.c.i.WebSocketHandler - Read msg=HttpObjectAggregator$AggregatedFullHttpResponse(decodeResult: success, version: HTTP/1.1, content: CompositeByteBuf(ridx: 0, widx: 163, cap: 163, components=1))
HTTP/1.1 400 Bad Request
Server: openresty/1.15.8.1
Date: Tue, 10 Mar 2020 06:49:53 GMT
Content-Type: text/html
Content-Length: 163
Connection: close
09:49:53.452 [DEBUG] i.g.h.c.i.WebSocketHandler - Crash
io.netty.handler.codec.http.websocketx.WebSocketHandshakeException: Invalid handshake response getStatus: 400 Bad Request
at io.netty.handler.codec.http.websocketx.WebSocketClientHandshaker13.verify(WebSocketClientHandshaker13.java:267)
In gatling.conf, I set also:
enableSni = false
useOpenSsl = false
enableHostnameVerification = false
If I try to open connection with any WS clients like SmartWebsocketClient - everything is ok, but by means of gatling I can't to open connection. Any ideas to fix it?
wss://domain?access_token... is not a valid URL. The path component must start with a / and not a ?. This means that the request is invalid which can explain the response of 400 Bad Request.
The URL must be at least wss://domain/?access_token... (i.e. a / before the ?) but maybe the rest of the URL is unexpected by the server too. Please check the actual requirements of the server.
This is a bug in Netty that doesn't properly compute WebSocket handshake request url when path is empty.
I've just contributed a patch: https://github.com/netty/netty/pull/10095.
Sibling issue in Gatling: https://github.com/gatling/gatling/issues/3876

Kong responds 404 when accessed vi Postman

I have set up an API and Kong. I have successfully installed Kong. In fact, i can access my APIs via curl on cli. However, when I use Postman, which I use all of the time, I get the response
{
"request_path": "/api/v1/",
"message": "API not found with these values",
"request_host": [
"192.168.33.13"
]
}
Ps. Im new to kong so please bear with me.
$ kong start
[INFO] Kong 0.7.0
[INFO] Using configuration: /etc/kong/kong.yml
[INFO] Setting working directory to /usr/local/kong
[INFO] database...........cassandra keyspace=kong ssl=verify=false enabled=false replication_factor=1 contact_points=127.0.0.1:9042 replication_strategy=SimpleStrategy timeout=5000 data_centers=
[INFO] dnsmasq............address=127.0.0.1:8053 dnsmasq=true port=8053
[INFO] serf ..............-profile=wan -rpc-addr=127.0.0.1:7373 -event-handler=member-join,member-leave,member-failed,member-update,member-reap,user:kong=/usr/local/kong/serf_event.sh -bind=0.0.0.0:7946 -node=precise64_0.0.0.0:7946 -log-level=err
[INFO] Trying to auto-join Kong nodes, please wait..
[WARN] Cannot auto-join the cluster because no nodes were found
[WARN] ulimit is currently set to "1024". For better performance set it to at least "4096" using "ulimit -n"
[INFO] nginx .............admin_api_listen=0.0.0.0:8001 proxy_listen=0.0.0.0:8000 proxy_listen_ssl=0.0.0.0:8443
[OK] Started
$ curl -i -X POST --url http://localhost:8001/apis/ --data 'name=geospatial' --data 'upstream_url=http://192.168.33.10/' --data 'request_host=192.168.33.10'
HTTP/1.1 201 Created
Date: Fri, 08 Apr 2016 14:38:22 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.7.0
{"upstream_url":"http:\/\/192.168.33.10\/","id":"240e6cc6-626f-4629-9551-0d341a57adba","name":"geospatial","created_at":1460126302000,"request_host":"192.168.33.10"}
When I curl the api...
$ curl -i -X GET -H "Host: 192.168.33.10" "http://192.168.33.13:8000/api/v1/"
HTTP/1.1 200 OK
Date: Fri, 08 Apr 2016 14:56:04 GMT
Content-Type: application/json
Content-Length: 70
Connection: keep-alive
Server: Werkzeug/0.11.4 Python/2.7.6
X-Kong-Upstream-Latency: 4
X-Kong-Proxy-Latency: 0
Via: kong/0.7.0
{"status": 200, "message": 200, "data": "Hello World!", "error": null}
How ever when I try to use Postman, I get a 404 status response.
with this body
{
"request_path": "/api/v1/",
"message": "API not found with these values",
"request_host": [
"192.168.33.13"
]
}
Am I missing something?
Disclaimer:
I am using Postman because I am lazy and because my clients will mostly be using Postman as well. So dont tell me to to use curl instead :)
Just encountered the same problem.
In order to send to Kong you need to add Host header, but Postman has some restricted headers that are blocked and Host is one of them.
You need to download Chrome Inspector to send the Host header.
For more details -
https://www.getpostman.com/docs/requests
You need to setup the header parameter named "Host" in order to Kong find your API.
You can see that your curl command have this parameter
-H "Host: 192.168.33.10"

Ejabberd OAuth / REST 401 Unauthorized

I'm trying to request the Ejabberd REST Web Services with the /api/connected_users endpoint but the request always returns me 401 Unauthorized HTTP errors.
Here is my OAuth configuration.
-
port: 5280
module: ejabberd_http
request_handlers:
"/websocket": ejabberd_http_ws
# OAuth Support
"/oauth": ejabberd_oauth
# ReST API:
"/api": mod_http_api
web_admin: true
http_bind: true
register: true
captcha: false
...
commands_admin_access: configure
commands:
- add_commands: user
oauth_expire: 3600
oauth_access: all
As explained in the documentation I use the following URL to generate an OAuth 2 Access Token for the admin user.
http://localhost:5280/oauth/authorization_token?response_type=token&client_id=myclient&redirect_uri=http://localhost:5280&scope=sasl_auth
It returns me my OAuth Token.
http://localhost:5280/?access_token=oLn8Hebh051l2PdCM15tSvHrEI25CpBs&token_type=bearer&expires_in=3600&scope=sasl_auth&state=
Finally to request the api/connected_users endpoint I do the following.
curl -v -X GET -H "X-Admin: true" -H "Authorization: Bearer oLn8Hebh051l2PdCM15tSvHrEI25CpBs" http://localhost:5280/api/connected_users
But it always returns me 401 Unauthorized errors.
In my ejabberd.log file I have this.
2016-02-09 09:47:12.177 [info] <0.497.0>#ejabberd_listener:accept:333 (#Port<0.16419>) Accepted connection 127.0.0.1:62395 -> 127.0.0.1:5280
2016-02-09 09:47:12.177 [debug] <0.546.0>#ejabberd_http:init:154 S: [{[<<"websocket">>],ejabberd_http_ws},{[<<"oauth">>],ejabberd_oauth},{[<<"api">>],mod_http_api},{[<<"register">>],mod_register_web},{[<<"admin">>],ejabberd_web_admin},{[<<"http-bind">>],mod_http_bind}]
2016-02-09 09:47:12.177 [info] <0.546.0>#ejabberd_http:init:158 started: {gen_tcp,#Port<0.16419>}
2016-02-09 09:47:12.177 [debug] <0.546.0>#ejabberd_http:process_header:281 (#Port<0.16419>) http query: 'GET' <<"/api/connected_users">>
2016-02-09 09:47:12.177 [debug] <0.546.0>#ejabberd_http:process:353 [<<"api">>,<<"connected_users">>] matches [<<"api">>]
2016-02-09 09:47:12.178 [info] <0.546.0>#mod_http_api:log:388 Admin call connected_users [] from 127.0.0.1:62395
So how to configure Ejabberd to allow the admin user to request all the Ejabberd REST Web Services ?
My users are stored in Mysql, as OAuth Tokens are stored in Mnesia could it be the problem ?
Thanks,
Baptiste
Be sure you are using the #host when entering username if it's specified in that way in ejabberd.yml. I wasn't receiving any response from REST requests because I was using just admin for user, when should be admin#somehost in user name.
Please let me know if this doesn't help.
make sure you enter correct User(jid): ( User (jid): user#hostname)
it worked for me.

Debug "about:blank" in CasperJs

I have the following simple casperjs script (I cannot name the actual URL - sorry) on a Windows 7 machine:
var casper = require('casper').create({verbose:true,logLevel: "debug"});
casper.start('https://[XXX].de', function() {
console.log(this.getCurrentUrl());
});
casper.run();
The Output states that it failed - and the current url is : "about:blank"
[info] [phantom] Starting...
[info] [phantom] Running suite: 2 steps
[debug] [phantom] opening url: https://[XXX].de, HTTP GET
[debug] [phantom] Navigation requested: url=https://[XXX].de, type=Other, lock=true, isMainFrame=true
[warning] [phantom] Loading resource failed with status=fail: https://[XXX].de
[debug] [phantom] Successfully injected Casper client-side utilities
about:blank
[info] [phantom] Step 2/2: done in 39205ms.
[info] [phantom] Done 2 steps in 39309ms
When send a GET request with the Firefox RESTCLient Plugin - I get:
Status Code: 200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection: close
Content-Type: text/html
Date: Tue, 11 Dec 2012 11:09:37 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
Server: unknown
Transfer-Encoding: chunked
My question is:
How can I debug this? Is this a known issue ?
Seems like the command line arguments passed to casperjs are not being passed to PhantomJS. I also ran into this issue and to fix this I ran my script as follows:
PhantomJS.exe --ignore-ssl-errors=true myscript.js
When I tried passing the parameters the other way around
PhantomJS.exe myscript.js --ignore-ssl-errors=true
it does not work and gives the same error as you have been facing.
To pass phantomjs parameters to casperjs, you can directly update your casper binary(casperjs.py on linux or casperjs.bat on windows).
On Linux, Open the casperjs.py, Update the CASPER_COMMAND array which is forming the phantomjs command and execute.
To add "--ignore-ssl-errors=yes", extend the CASPER_COMMAND by the following,
CASPER_COMMAND.extend(['--ignore-ssl-errors=yes']);
For the same, on windows or For more info can be found here