I'm trying to integrate a Datadog monitor check on sshd process in my terraform codebase, but I'm getting datadog_monitor.host_is_up2: error updating monitor: API error 400 Bad Request: {"errors":["The value provided for parameter 'query' is invalid"]}
What I did was to copy the monitor's query I created on the Datadog panel and pasted it into the tf file:
resource "datadog_monitor" "host_is_up2" {
name = "host is up"
type = "metric alert"
message = "Monitor triggered"
escalation_message = "Escalation message"
query = "process.up.over('process:ssh').last(4).count_by_status()"
thresholds {
ok = 0
warning = 1
critical = 2
}
notify_no_data = false
renotify_interval = 60
notify_audit = false
timeout_h = 60
include_tags = true
silenced {
"*" = 0
}
}
ofc the query example "avg(last_1h):avg:aws.ec2.cpu{environment:foo,host:foo} by {host} > 2" works
What's the right way to check via Datadog API or terraform if a specific service, like sshd, is up or not?
There are two error in your code:
The type used is wrong. It should be service check instead of metric alert.
You need to enclose process.up in a pair of ''.
Once done, your code will run flawlessly.
Related
I have an API which gets the success or error message on console.I am new to python and trying to read the response. Google throws so many examples to use subprocess but I dont want to run,call any command or sub process. I just want to read the output after below API call.
This is the response in console when success
17:50:52 | Logged in!!
This is the github link for the sdk and documentation
https://github.com/5paisa/py5paisa
This is the code
from py5paisa import FivePaisaClient
email = "myemailid#gmail.com"
pw = "mypassword"
dob = "mydateofbirth"
cred={
"APP_NAME":"app-name",
"APP_SOURCE":"app-src",
"USER_ID":"user-id",
"PASSWORD":"pw",
"USER_KEY":"user-key",
"ENCRYPTION_KEY":"enc-key"
}
client = FivePaisaClient(email=email, passwd=pw, dob=dob,cred=cred)
client.login()
In general it is bad practice to get a value from STDOUT. There are some ways but it's pretty tricky (it's not made for it). And the problem doesn't come from you but from the API which is wrongly designed, it should return a value e.g. True or False (at least) to tell you if you logged in, and they don't do it.
So, according to their documentation it is not possible to know if you're logged in, but you may be able to see if you're logged in by checking the attribute client_code in the client object.
If client.client_code is equal to something then it should be logged in and if it is equal to something else then not. You can try comparing it's value when you successfully login or when it fails (wrong credential for instance). Then you can put a condition : if it is None or False or 0 (you will have to see this by yourself) then it is failed.
Can you try doing the following with a successful and failed login:
client.login()
print(client.client_code)
Source of the API:
# Login function :
# (...)
message = res["body"]["Message"]
if message == "":
log_response("Logged in!!")
else:
log_response(message)
self._set_client_code(res["body"]["ClientCode"])
# (...)
# _set_client_code function :
def _set_client_code(self, client_code):
try:
self.client_code = client_code # <<<< That's what we want
except Exception as e:
log_response(e)
Since this questions asks how to capture "stdout" one way you can accomplish this is to intercept the log message before it hits stdout.
The minimum code to capture a log message within a Python script looks this:
#!/usr/bin/env python3
import logging
logger = logging.getLogger(__name__)
class RequestHandler(logging.Handler):
def emit(self, record):
if record.getMessage().startswith("Hello"):
print("hello detected")
handler = RequestHandler()
logger.addHandler(handler)
logger.warning("Hello world")
Putting it all together you may be able to do something like this:
import logging
from py5paisa import FivePaisaClient
email = "myemailid#gmail.com"
pw = "mypassword"
dob = "mydateofbirth"
cred={
"APP_NAME":"app-name",
"APP_SOURCE":"app-src",
"USER_ID":"user-id",
"PASSWORD":"pw",
"USER_KEY":"user-key",
"ENCRYPTION_KEY":"enc-key"
}
client = FivePaisaClient(email=email, passwd=pw, dob=dob,cred=cred)
class PaisaClient(logging.Handler):
def __init__():
self.loggedin = False # this is the variable we can use to see if we are "logged in"
def emit(self, record):
if record.getMessage().startswith("Logged in!!")
self.loggedin = True
def login():
client.login()
logging.getLogger(py5paisa) # get the logger for the py5paisa library
# tutorial here: https://betterstack.com/community/questions/how-to-disable-logging-from-python-request-library/
logging.basicConfig(handlers=[PaisaClient()], level=0, force=True)
c = PaisaClient()
c.login()
I want to show the status of the build in an e-mail with just a simple FAIL or PASS text appear in the body. There does not seem to be any kind of predefined "buildStatus" variable that I can access or setup in TeamCity. I guess I need to access the "failureConditions" function at bottom but not sure how, tried lots of things but nothing worked, this is my script:
package _Self.buildTypes
import jetbrains.buildServer.configs.kotlin.v2019_2.failureConditions.BuildFailureOnText
import jetbrains.buildServer.configs.kotlin.v2019_2.failureConditions.failOnText
import jetbrains.buildServer.configs.kotlin.v2019_2.triggers.schedule
allowExternalStatus = true
params {
param("MinorVersion", "0")
param("RevisionVersion", "0")
}
powerShell {
name = "Email"
scriptMode = script {
content = """
function Send-ToEmail([string]${'$'}email, [string]${'$'}attachmentpath){
${'$'}message = new-object Net.Mail.MailMessage;
${'$'}message.From = "teamcity#blog.com";
${'$'}message.To.Add(${'$'}email);
${'$'}message.Subject = "%env.TEAMCITY_PROJECT_NAME% | %VersionNumber% ";
${'$'}message.Body = "The build: PASS or FAIL text here";
${'$'}smtp = new-object Net.Mail.SmtpClient("blog.local", "25");
${'$'}smtp.EnableSSL = ${'$'}true;
${'$'}smtp.send(${'$'}message);
write-host "Mail Sent" ;
}
Send-ToEmail -email "me#blog.com" -attachmentpath ${'$'}path;
""".trimIndent()
}
}
}
failureConditions {
failOnText {
conditionType = BuildFailureOnText.ConditionType.CONTAINS
pattern = "FAIL"
reverse = false
}
}
To solve this in T/C:
Create new custom env. var. e.g. MyBuildStatus
Create new build step e.g. Set build status (only executes if build successful)
Create a new Parameter name='env.MyBuildStatus.Status' value='SUCCESS'
Add variable to email subject: $message.Subject = "%env.MyBuildStatus.Status%
Why T/C has no build in "buildstatus" env. variable is interesting.
I'm running Terraform using VScode editor which uses PowerShell as the default shell and getting the same error when I try to validate it or to run terraform init/plan/apply through VScode, external PowerShell or CMD.
The code was running without any issues until I added Virtual Machine creation code. I have clubbed the variables.tf, terraform.tfvars and the main Terraform code below.
terraform.tfvars
web_server_location = "West US 2"
resource_prefix = "web-server"
web_server_address_space = "1.0.0.0/22"
web_server_address_prefix = "1.0.1.0/24"
Environment = "Test"
variables.tf
variable "web_server_location" {
type = string
}
variable "resource_prefix" {
type = string
}
variable "web_server_address_space" {
type = string
}
#variable for network range
variable "web_server_address_prefix" {
type = string
}
#variable for Environment
variable "Environment" {
type = string
}
terraform_example.tf
# Configure the Azure Provider
provider "azurerm" {
# whilst the `version` attribute is optional, we recommend pinning to a given version of the Provider
version = "=2.0.0"
features {}
}
# Create a resource group
resource "azurerm_resource_group" "example_rg" {
name = "${var.resource_prefix}-RG"
location = var.web_server_location
}
# Create a virtual network within the resource group
resource "azurerm_virtual_network" "example_vnet" {
name = "${var.resource_prefix}-vnet"
resource_group_name = azurerm_resource_group.example_rg.name
location = var.web_server_location
address_space = [var.web_server_address_space]
}
# Create a subnet within the virtual network
resource "azurerm_subnet" "example_subnet" {
name = "${var.resource_prefix}-subnet"
resource_group_name = azurerm_resource_group.example_rg.name
virtual_network_name = azurerm_virtual_network.example_vnet.name
address_prefix = var.web_server_address_prefix
}
# Create a Network Interface
resource "azurerm_network_interface" "example_nic" {
name = "${var.resource_prefix}-NIC"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
ip_configuration {
name = "internal"
subnet_id = azurerm_subnet.example_subnet.id
private_ip_address_allocation = "Dynamic"
public_ip_address_id = azurerm_public_ip.example_public_ip.id
}
}
# Create a Public IP
resource "azurerm_public_ip" "example_public_ip" {
name = "${var.resource_prefix}-PublicIP"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
allocation_method = var.Environment == "Test" ? "Static" : "Dynamic"
tags = {
environment = "Test"
}
}
# Creating resource NSG
resource "azurerm_network_security_group" "example_nsg" {
name = "${var.resource_prefix}-NSG"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
# Security rule can also be defined with resource azurerm_network_security_rule, here just defining it inline.
security_rule {
name = "RDPInbound"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "3389"
source_address_prefix = "*"
destination_address_prefix = "*"
}
tags = {
environment = "Test"
}
}
# NIC and NSG association
resource "azurerm_network_interface_security_group_association" "example_nsg_association" {
network_interface_id = azurerm_network_interface.example_nic.id
network_security_group_id = azurerm_network_security_group.example_nsg.id
}
# Creating Windows Virtual Machine
resource "azurerm_virtual_machine" "example_windows_vm" {
name = "${var.resource_prefix}-VM"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
network_interface_ids = [azurerm_network_interface.example_nic.id]
vm_size = "Standard_B1s"
delete_os_disk_on_termination = true
storage_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServerSemiAnnual"
sku = "Datacenter-Core-1709-smalldisk"
version = "latest"
}
storage_os_disk {
name = "myosdisk1"
caching = "ReadWrite"
create_option = "FromImage"
storage_account_type = "Standard_LRS"
}
os_profile {
computer_name = "hostname"
admin_username = "adminuser"
admin_password = "Password1234!"
}
os_profile_windows_config {
disable_password_authentication = false
}
tags = {
environment = "Test"
}
}
Error:
PS C:\Users\e5605266\Documents\MyFiles\Devops\Terraform> terraform init
There are some problems with the configuration, described below.
The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
Error: Invalid character
on terraform_example.tf line 89, in resource "azurerm_virtual_machine" "example_windows_vm":
89: location = azurerm_resource_group.example_rg.location
This character is not used within the language.
Error: Invalid expression
on terraform_example.tf line 89, in resource "azurerm_virtual_machine" "example_windows_vm":
89: location = azurerm_resource_group.example_rg.location
Expected the start of an expression, but found an invalid expression token.
Error: Argument or block definition required
on terraform_example.tf line 90, in resource "azurerm_virtual_machine" "example_windows_vm":
90: resource_group_name = azurerm_resource_group.example_rg.name
An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.
Error: Invalid character
on terraform_example.tf line 90, in resource "azurerm_virtual_machine" "example_windows_vm":
90: resource_group_name = azurerm_resource_group.example_rg.name
This character is not used within the language.
*
I've encountered this problem myself in several different contexts, and it does have a common solution which is no fun at all: manually typing the code back in...
This resource block seems to be where it runs into problems:
resource "azurerm_virtual_machine" "example_windows_vm" {
name = "${var.resource_prefix}-VM"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
network_interface_ids = [azurerm_network_interface.example_nic.id]
vm_size = "Standard_B1s"
delete_os_disk_on_termination = true
storage_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServerSemiAnnual"
sku = "Datacenter-Core-1709-smalldisk"
version = "latest"
}
storage_os_disk {
name = "myosdisk1"
caching = "ReadWrite"
create_option = "FromImage"
storage_account_type = "Standard_LRS"
}
os_profile {
computer_name = "hostname"
admin_username = "adminuser"
admin_password = "Password1234!"
}
os_profile_windows_config {
disable_password_authentication = false
}
tags = {
environment = "Test"
}
}
Try copying that back into your editor as is. I cannot see any problematic characters in it, and ironically StackOverflow may have done you a solid and filtered them out. Literally copy/pasting it over the existing block may remedy the situation.
I have seen Terraform examples online with stylish double quotes (which aren't ASCII double quotes and won't work) many times. That may be what you are seeing.
Beyond that, you'd need to push your code to GitHub or similar so I can see the raw bytes for myself.
In the off-chance this helps someone who runs into this error and comes across it on Google, I just thought I would post my situation and how I fixed it.
I have an old demo Terraform infrastructure that I revisited after months and, long story short, I issued this command two days ago and forgot about it:
terraform plan -out=plan.tf
This creates a zip archive of the plan. Upon coming back two days later and running a terraform init, my terminal scrolled garbage and "This character is not used within the language." for about 7 seconds. Due to the .tf extension, terraform was looking at the zip data and promptly pooping its pants.
Through moving individual tf files to a temp directory and checking their validity with terraform init, I found the culprit, deleted it, and functionality was restored.
Be careful when exporting your plan files, folks!
I ran into the same problem and found this page.
I solved the issue and decided to post here.
I opened my plan file in Notepad++ and selected View-Show all symbols.
I removed all the TAB characters and replaced them with spaces.
In my case, the problem was fully resolved by this.
In my case, when I ran into the same problem ("This character is not used within the language"), I found the encoding of the files was UTF-16 (it was a generated file from PS). Changing the file encoding to UTF-8 (as mentioned in this question) solved the issue.
I found I got this most often when I go from Windows to linux. The *.tf file does not like the windows TABs and Line Breaks.
I tried to some of the same tools I use when I have this problem with *.sh, but so far I've resorted to manually cleaning up the lines I've seen in there error.
In my case, the .tf file was generated by the following command terraform show -no-color > my_problematic.tf, and this file's encoding is in "UTF-16 LE BOM", converting it to UTF-8 fixed my issue.
A follow up to this:
one SCDF source, 2 processors but only 1 processes each item
The 2 processors (del-1 and del-2) in the picture are receiving the same data within milliseconds of each other. I'm trying to rig this so del-2 never receives the same thing as del-1 and vice versa. So obviously I've got something configured incorrectly but I'm not sure where.
My processor has the following application.properties
spring.application.name=${vcap.application.name:sample-processor}
info.app.name=#project.artifactId#
info.app.description=#project.description#
info.app.version=#project.version#
management.endpoints.web.exposure.include=health,info,bindings
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration
spring.cloud.stream.bindings.input.group=input
Is "spring.cloud.stream.bindings.input.group" specified correctly?
Here's the processor code:
#Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
public Object transform(String inputStr) throws InterruptedException{
ApplicationLog log = new ApplicationLog(this, "timerMessageSource");
String message = " I AM [" + inputStr + "] AND I HAVE BEEN PROCESSED!!!!!!!";
log.info("SampleProcessor.transform() incoming inputStr="+inputStr);
return message;
}
Is the #Transformer annotation the proper way to link this bit of code with "spring.cloud.stream.bindings.input.group" from application.properties? Are there any other annotations necessary?
Here's my source:
private String format = "EEEEE dd MMMMM yyyy HH:mm:ss.SSSZ";
#Bean
#InboundChannelAdapter(value = Source.OUTPUT, poller = #Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public MessageSource<String> timerMessageSource() {
ApplicationLog log = new ApplicationLog(this, "timerMessageSource");
String message = new SimpleDateFormat(format).format(new Date());
log.info("SampleSource.timeMessageSource() message=["+message+"]");
return () -> new GenericMessage<>(new SimpleDateFormat(format).format(new Date()));
}
I'm confused about the "value = Source.OUTPUT". Does this mean my processor needs to be named differently?
Is the inclusion of #Poller causing me a problem somehow?
This is how I define the 2 processor streams (del-1 and del-2) in SCDF shell:
stream create del-1 --definition ":split > processor-that-does-everything-sleeps5 --spring.cloud.stream.bindings.applicationMetrics.destination=metrics > :merge"
stream create del-2 --definition ":split > processor-that-does-everything-sleeps5 --spring.cloud.stream.bindings.applicationMetrics.destination=metrics > :merge"
Do I need to do anything differently there?
All of this is running in Docker/K8s.
RabbitMQ is given by bitnami/rabbitmq:3.7.2-r1 and is configured with the following props:
RABBITMQ_USERNAME: user
RABBITMQ_PASSWORD <redacted>:
RABBITMQ_ERL_COOKIE <redacted>:
RABBITMQ_NODE_PORT_NUMBER: 5672
RABBITMQ_NODE_TYPE: stats
RABBITMQ_NODE_NAME: rabbit#localhost
RABBITMQ_CLUSTER_NODE_NAME:
RABBITMQ_DEFAULT_VHOST: /
RABBITMQ_MANAGER_PORT_NUMBER: 15672
RABBITMQ_DISK_FREE_LIMIT: "6GiB"
Are any other environment variables necessary?
I need to program a client to Domino Server using Notes C API which registers a new Lotus Notes user. Using REGNewUser (see http://www-12.lotus.com/ldd/doc/domino_notes/8.5.3/api853ref.nsf/ef2467c10609eaa8852561cc0067a76f/0326bfa2438ebe9985256678006a6ff2?OpenDocument&Highlight=0,REGNew*) and it looks promising except for the fact that I need to make the user's mail file replicate from the specified mail server to the mail server's cluster partner. There is the flag
fREGExtMailReplicasUsingAdminp
and the documentation is very brief about it:
"Create mail replicas via the administration process"
If I google the flag I get 4 (!) hits.
How do I specify where the mail file replica is created? Does anyone have any more information about what this flag is actually doing?
Thanks
Kai
After 3 weeks of research in the Notes C API reference I found:
In REGNewPerson there are 2 structures REG_MAIL_INFO_EXT and REG_PERSON_INFO and if you set the above mentioned flag in REG_PERSON_INFO then you have to provide a list of replica servers in REG_MAIL_INFO_EXT like this:
REG_MAIL_INFO_EXT mail_info, *pmail_info;
REG_PERSON_INFO person_info, *pperson_info;
...
pmail_info = &mail_info;
pperson_info = &person_info;
...
// pmail_info->pMailForwardAddress = NULL; // brauchen wir nicht.
pmail_info->pMailServerName = mailserver;
pmail_info->pMailTemplateName = mailfiletemplate;
// do the list crap for replica servers
if (error = ListAllocate (0, 0, FALSE, &hReplicaServers, &pList, &list_size)) {
goto __error;
}
OSUnlock (hReplicaServers);
pList = NULL;
if (error = ListAddEntry (hReplicaServers, // handle to list
FALSE, // do not include data type
&list_size, // pass list size in memory
0, // index to add
replicationserver, // value to add
(WORD) strlen(replicationserver))) // size of value to add
{
goto __error;
}
// now we can add the handle to the structure
pmail_info->hReplicaServers = hReplicaServers;
...
pperson_info->MailInfo = pmail_info;
...
pperson_info->Flags = fREGCreateIDFileNow | fREGCreateAddrBookEntry | fREGCreateMailFileUsingAdminp;
pperson_info->FlagsExt = fREGExtEnforceUniqueShortName | fREGExtMailReplicasUsingAdminp;
In my case this did the trick.