Extract domains from deployed serverless endpoints - serverless-framework

After running a serveless deploy command, I would like the retrieve the domain name of the deployed endpoint(s) for further usage in other services. The best I've come up until now is to grep the sls info output.
e.g.
sls info --stage integration | grep GET - | head -1 | egrep -o 'https?://[^ ]+' | awk -F[/:] '{print $4}'
Is there a better, less hacky way, of achieving this?

You can use this plugin.
In your serverless.yml
plugins:
- serverless-scriptable-plugin
custom:
scriptHooks:
after:deploy:finalize:
- scripts/postDeploy.js
And in postDeploy.js
const getStackOutputs = async () => {
const provider = serverless.getProvider('aws');
const { stage, region } = provider.options;
const stackName = provider.naming.getStackName();
const result = await provider.request(
'CloudFormation',
'describeStacks',
{ StackName: stackName },
stage,
region,
);
const outputsArray = result.Stacks[0].Outputs;
const outputs = {};
for (let i = 0; i < outputsArray.length; i++) {
outputs[outputsArray[i].OutputKey] = outputsArray[i].OutputValue;
}
const hostname = new (require('url')).URL(outputs.ServiceEndpoint).hostname;
console.log(hostname);
};
getStackOutputs();
Another option is this plugin but you'll need to use a forked version as described here.

Related

How can I recursively read a directory in Deno?

I'm trying to recursively read a file in Deno using Deno.readDir, but the example they provide only does the folder given:
for await (const entry of Deno.readDir(Deno.cwd())) {
console.log(entry.name);
}
How can I make this recursive?
Deno's standard library includes a function called walk for this purpose. It's available in std/fs/walk.ts. Here's an example:
/Users/deno/so-74953935/main.ts:
import { walk } from "https://deno.land/std#0.170.0/fs/walk.ts";
for await (const walkEntry of walk(Deno.cwd())) {
const type = walkEntry.isSymlink
? "symlink"
: walkEntry.isFile
? "file"
: "directory";
console.log(type, walkEntry.path);
}
Running in the terminal:
% pwd
/Users/deno/so-74953935
% ls -AF
.vscode/ deno.jsonc deno.lock main.ts
% ls -AF .vscode
settings.json
% deno --version
deno 1.29.1 (release, x86_64-apple-darwin)
v8 10.9.194.5
typescript 4.9.4
% deno run --allow-read main.ts
directory /Users/deno/so-74953935
file /Users/deno/so-74953935/main.ts
file /Users/deno/so-74953935/deno.jsonc
file /Users/deno/so-74953935/deno.lock
directory /Users/deno/so-74953935/.vscode
file /Users/deno/so-74953935/.vscode/settings.json
Since that function returns an async generator, you can make your own generator function that wraps around Deno.readDir:
(Do note that the example provided will join the path and name, giving you strings such as /directory/name.txt)
import { join } from "https://deno.land/std/path/mod.ts";
export async function* recursiveReaddir(
path: string
): AsyncGenerator<string, void> {
for await (const dirEntry of Deno.readDir(path)) {
if (dirEntry.isDirectory) {
yield* recursiveReaddir(join(path, dirEntry.name));
} else if (dirEntry.isFile) {
yield join(path, dirEntry.name);
}
}
}
for await (const entry of recursiveReaddir(Deno.cwd())) {
console.log(entry)
}
OR, you can use recursive_readdir, which is a 3rd party library in Deno made for this purpose.

Workaround for `count.index` in Terraform Module

I need a workaround for using count.index inside a module block for some input variables. I have a habit of over-complicating problems, so maybe there's a much easier solution.
File/Folder Structure:
modules/
main.tf
ignition/
main.tf
modules/
files/
main.tf
template_files/
main.tf
End Goal: Create an Ignition file for each instance I'm deploying. Each Ignition file has instance-specific info like hostname, IP address, etc.
All of this code works if I use a static value or a variable without cound.index. I need help coming up with a workaround for the address, gateway, and hostname variables specifically. If I need to process the count.index inside one of the child modules, that's totally fine. I can't seem to wrap my brain around that though. I've tried null_data_source and null_resource blocks from the child modules to achieve that, but so far no luck.
Variables:
workers = {
Lab1 = {
"lab1k8sc8r001" = "192.168.17.100/24"
}
Lab2 = {
"lab2k8sc8r001" = "192.168.18.100/24"
}
}
gateway = {
Lab1 = [
"192.168.17.1",
]
Lab2 = [
"192.168.18.1",
]
}
From modules/main.tf, I'm calling the ignition module:
module "ignition_workers" {
source = "./modules/ignition"
virtual_machines = var.workers[terraform.workspace]
ssh_public_keys = var.ssh_public_keys
files = [
"files_90-disable-auto-updates.yaml",
"files_90-disable-console-logs.yaml",
]
template_files = {
"files_eth0.nmconnection.yaml" = {
interface-name = "eth0",
address = element(values(var.workers[terraform.workspace]), count.index),
gateway = element(var.gateway, count.index % length(var.gateway)),
dns = join(";", var.dns_servers),
dns-search = var.domain,
}
"files_etc_hostname.yaml" = {
hostname = element(keys(var.workers[terraform.workspace]), count.index),
}
"files_chronyd.yaml" = {
ntp_server = var.ntp_server,
}
}
}
From modules/ignition/main.tf I take the files and template_files variables to build the Ignition config:
module "ingition_file_snippets" {
source = "./modules/files"
files = var.files
}
module "ingition_template_file_snippets" {
source = "./modules/template_files"
template_files = var.template_files
}
data "ct_config" "fedora-coreos-config" {
count = length(var.virtual_machines)
content = templatefile("${path.module}/assets/files_ssh_authorized_keys.yaml", {
ssh_public_keys = var.ssh_public_keys
})
pretty_print = true
snippets = setunion(values(module.ingition_file_snippets.files), values(module.ingition_template_file_snippets.files))
}
I am not quite sure what you are trying to achieve so I can not give any detailed examples.
But modules in terraform do not support count or for_each yet. So you can also not use count.index.
You might want to change your module to take lists/maps of input and create those lists/maps via for-expressions by transforming them from some input variables.
You can combine for with if to create a filtered subset of your source list/map. Like in:
[for s in var.list : upper(s) if s != ""]
I hope this helps you work around the missing count support.

How can I override the jetbrains jdk dependency required by idea-ultimate or idea-community in config.nix?

I’ve been setting up my local nix config as per nixpkgs manual's declarative package management.
I’d like to include idea-ultimate as one of myPackages, but at this time the dependency idea has on the jetbrains jdk is broken, pointing to a non-existing package for macOS.
It’s trying to download jbrsdk-11_0_2-osx-x64-b485.1.tar.gz instead of jbrsdk-11_0_4-osx-x64-b485.1.tar.gz.
I was assuming I could fix that by overriding jetbrainsjdk as follows, but I’m getting: error: attribute 'jetbrainsjdk' missing, at /Users/ldeck/.config/nixpkgs/config.nix:4:20 when I do anything like nix-env -qa ‘jetbrains.*’.
What is the right way to override idea-ultimate so that it uses the fixed jdk?
Here’s my ~./config/nixpkgs/config.nix.
{
allowUnfree = true;
packageOverrides = pkgs: rec {
jetbrainsjdk = pkgs.jetbrainsjdk.override {
version = "520.11";
src = pkgs.fetchurl {
url = "https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b${jetbrainsjdk.version}.tar.gz";
sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9z";
};
};
myProfile = pkgs.writeText "my-profile" ''
export PATH=$HOME/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/sbin:/bin:/usr/sbin:/usr/bin
export MANPATH=$HOME/.nix-profile/share/man:/nix/var/nix/profiles/default/share/man:/usr/share/man
'';
myPackages = with pkgs; buildEnv {
name = "my-packages";
paths = [
(runCommand "profile" {} ''
mkdir -p $out/etc/profile.d
cp ${myProfile} $out/etc/profile.d/my-profile.sh
'')
aspell
bc
coreutils
direnv
emacs
emscripten
ffmpeg
gdb
git
hello
jq
nixops
nox
scala
silver-searcher
];
pathsToLink = [ "/share/man" "/share/doc" "/bin" "/etc" "/Applications" ];
extraOutputsToInstall = [ "man" "doc" ];
};
};
}
UPDATE 1
Thanks to #ChrisStryczynski who suggested I needed with pkgs, I’ve gotten a little further.
But now the problem is when attempting to install idea-ultimate with the custom jdk, it’s still requiring the broken, non-existing, jbrsdk-11_02-osx-x64-b485.1.tar.gz.drv from somewhere.
Updated config and logs below.
{
allowUnfree = true;
packageOverrides = pkgs: **with pkgs;** rec {
myJetbrainsJdk = **pkgs.jetbrains.jdk.overrideAttrs** (oldAttrs: rec {
version = "520.11";
src = pkgs.fetchurl {
url = "https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9z";
};
});
myIdeaUltimate = pkgs.jetbrains.idea-ultimate.override {
jdk = myJetbrainsJdk;
};
...
myPackages = with pkgs; buildEnv {
...
myIdeaUltimate
];
...
};
};
}
Logs
nix-channel --update; nix-env -iA nixpkgs.myPackages
unpacking channels...
replacing old 'my-packages'
installing 'my-packages'
these derivations will be built:
/nix/store/9kfi3k9q6hi7z3lwann318hndbah535v-idea-ultimate.desktop.drv
/nix/store/ica1m5yq3f3y05xnw7ln1lnfvp0yjvyf-download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz.drv
/nix/store/bf2hwhrvfl8g77gdiw053rayh06x0120-jetbrainsjdk-520.11.drv
/nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv
/nix/store/fwwk976sd278zb68zy9wm5pkxss0rnhg-jetbrainsjdk-485.1.drv
/nix/store/s3m2bpcyrnx9dcq4drh95882n0mk1d6m-ideaIU-2019.2.4-no-jbr.tar.gz.drv
/nix/store/9kiajpmmsp3i6ysj4vdqq8dzi84mnr73-idea-ultimate-2019.2.4.drv
/nix/store/jh1ixm54qinv8pk6kypvv6n6cfr4sws8-my-packages.drv
these paths will be fetched (0.02 MiB download, 0.12 MiB unpacked):
/nix/store/hp90sbwznq1msv327f0lb27imvcvi80h-libnotify-0.7.8
building '/nix/store/9kfi3k9q6hi7z3lwann318hndbah535v-idea-ultimate.desktop.drv'...
copying path '/nix/store/hp90sbwznq1msv327f0lb27imvcvi80h-libnotify-0.7.8' from 'https://cache.nixos.org'...
building '/nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv'...
trying https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz from any mirror
builder for '/nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv' failed with exit code 1
building '/nix/store/ica1m5yq3f3y05xnw7ln1lnfvp0yjvyf-download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz.drv'...
cannot build derivation '/nix/store/fwwk976sd278zb68zy9wm5pkxss0rnhg-jetbrainsjdk-485.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9kiajpmmsp3i6ysj4vdqq8dzi84mnr73-idea-ultimate-2019.2.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/jh1ixm54qinv8pk6kypvv6n6cfr4sws8-my-packages.drv': 1 dependencies couldn't be built
error: build of '/nix/store/jh1ixm54qinv8pk6kypvv6n6cfr4sws8-my-packages.drv' failed
Thanks to How do you discover the override attributes for a derivation, a solution has been found using an overlay.
# ~/config/nixpkgs/overlays/02-jetbrains.nix
self: super:
{
jetbrains = super.jetbrains // {
jdk = super.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
version = "520.11";
src = super.fetchurl {
url = "https://bintray.com/jetbrains/intellij-jbr/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
sha256 = "3fe1297133440a9056602d78d7987f9215139165bd7747b3303022a6f5e23834";
};
passthru = oldAttrs.passthru // {
home = "${self.jetbrains.jdk}/Contents/Home";
};
});
idea-ultimate = super.jetbrains.idea-ultimate.overrideAttrs (_: {
name = "idea-ultimate.2019.2.4";
src = super.fetchurl {
url = "https://download.jetbrains.com/idea/ideaIU-2019.2.4-no-jbr.tar.gz";
sha256 = "09mz4dx3zbnqw0vh4iqr8sn2s8mvgr7zvn4k7kqivsiv8f79g90a";
};
});
};
}
Install: nix-env -iA 'nixpkgs.jetbrains.idea-ultimate'.
Execute: idea-ultimate.
The crucial part of the puzzle was overriding the passthru.home variable to point to the overlayed JDK rather than the one required by super. Otherwise, you’ll be downloading the old JDK for runtime purposes.
passthru = oldAttrs.passthru // {
home = "${self.jetbrains.jdk}/Contents/Home”;
};
Without /Contents/Home appended, idea won’t startup since self.jetbrains.jdk isn’t a valid home.
Instead of:
nix-env -iA nixpkgs.myPackages
Just do:
nix-env -iA nixpkgs.myIdeaUltimate
The problem being:
myPackages = with pkgs; buildEnv {
...
myIdeaUltimate
];
...
};
Here you are still referencing the old pkgs.myIdeaUltimate.
Nixos seems to do some processing that replaces the pkgs with the appropriate from packageOverrides.

Need to make a port detector

I am trying to make a service which can detect when a port gets occupied.
So netstat -lntu | grep tcp can list all occupied ports. So a difference between the output from first netstat executioin and one after the a port gets occupied can get me the port value.
And it needs to be fully automated so i made a infinite loop which will continuously check for differences between two consecutive netstat.
Here is my code in js and i am running it using node-daemonize2:
var shell = require("shelljs");
var InfiniteLoop = require("infinite-loop");
var il = new InfiniteLoop();
var diff = require('diff');
var oldStr = '';
var newStr = '';
shell.exec("netstat -lntu | grep tcp",{silent:true}, function (code, output, error) {
oldStr = output;
});
il.add(function () {
shell.exec("netstat -lntu | grep tcp",{silent:true}, function (code, output, error) {
newStr = output;
if(!newStr.replace(oldStr,'')==''){
var dD = diff.diffChars(oldStr, newStr)[1];
if(dD.added){
let value = dD.value.replace(/\n/g,'');
re = /[0-9][0-9][0-9][0-9]/
value = re.exec(value);
console.log(value[0]);
}
}
oldStr = newStr;
});
});
il.run();
Now this thing works perfectly fine, but is resource heavy. So is there a way to make it light or a completely new and more efficient way to approach this problem?

Export URL for code organisation

I'm fairly new to React native, and i would like to have all my endpoints URLs in the same place
So, i have in my environement.js:
module.exports = {
SERVER: 'http://url.com',
API_VERSION: '/api/v3',
FULL_URL: this.SERVER + this.API_VERSION,
PLAYERS: this.FULL_URL + '/players',
TEAMS: this.FULL_URL + '/teams',
};
And on the other side in getGame.js
GLOBAL = require('../environement');
function getGame() {
console.log(GLOBAL.PLAYERS)
}
When i run my app i got
undefined/players
What am i doing wrong?
Thanks
You can try something like this using ES6.
const SERVER = 'http://url.com';
const API_VERSION = '/api/v3';
export const FULL_URL = SERVER + API_VERSION;
export const PLAYERS = FULL_URL + '/players';
export const TEAMS = FULL_URL + '/teams';
And then
import { FULL_URL, PLAYERS, TEAMS } from 'path/to/the/file'
Please take a look to this solution: https://github.com/luggit/react-native-config .
Also it will be very helpful when you will be have a few different environments and build agent for app in teamcity for example.