azure devop selfhosted agent, newman command not recognized - npm

Trying to run my postman collection in azure devops inside a self-hosted agent. When I try to run the command inside the agent "newman run postman_collection.json -e postman_environment.json -r cli,htmlextra" it's running fine. But when I run the same through a a command line script task in release pipeline it's throwing the error "newman is not recognized..". I also tried to have a npm task for newman installation i.e. "npm install -g newman" it's also throwing the erro "##[error]Unable to locate executable file: 'newman'. Please verify either the file path exists or the file can be found within a d...."

azure devop selfhosted agent, newman command not recognized
According to the error message "##[error]Unable to locate executable file: 'newman" when you using the npm install -g newman, you could try to add C:\Users\[BUILDSERVER-USERNAME]\AppData\Roaming\npm to the PATH variable for the [BUILDSERVER-USERNAME] user.
You could refer to this document How to fix the Newman task for Team Foundation Server silently failing for some more details.
Besides, when we use command line to install the newman, it will take a few minutes to install it, so we need to wait for a few minutes before we using the command line:
"newman run postman_collection.json -e postman_environment.json -r cli,htmlextra"
You could add powershell task to sleep a few minutes:
echo "Sleeping for 10 mins..."
Start-Sleep -s 600

Related

Newman command not found

I am trying to run m collection through jenkins but it shows below error:
** Started by user trideep mukherjee Running as SYSTEM Building in workspace
C:\Program Files (x86)\Jenkins\workspace\Newman [Newman] $
cmd /c call C:\WINDOWS\TEMP\jenkins8980552465408575800.bat C:\Program
Files (x86)\Jenkins\workspace\Newman>cd
C:\Users\tride\AppData\Roaming\npm\node_modules\newman
C:\Users\tride\AppData\Roaming\npm\node_modules\newman>newman
--version ‘newman’ is not recognized as an internal or external command, operable program or batch file.
C:\Users\tride\AppData\Roaming\npm\node_modules\newman>exit 9009 Build
step 'Ex**
I have tried the all possible ways mention in the blog to add the local path of the newman but didn’t worked.
Please see the environment variable I have added
Set newman installation path to path in windows or in linux as environment variable.
Then it will work

WebDriverException: Message: invalid argument: can't kill an exited process in a Docker container returns odd errors

I'm running a small Python script that scrapes some data from a public website.
When I run the Dockerfile instructions line by line in an interactive terminal using the selenium:latest image the script runs fine.
docker run -it -v /Users/me/Desktop/code/scraper/:/scraper selenium/standalone-firefox bash
As soon as I run it using my Dockerfile and docker-compose file I get this error:
app_1 | selenium.common.exceptions.WebDriverException: Message: invalid argument: can't kill an exited process
I am using the MOZ_HEADLESS=1 env var. It's being passed properly.
I have tried running the script as someone other than root but then I get log errors.
Dockerfile
FROM selenium/standalone-firefox:latest
# https://github.com/SeleniumHQ/docker-selenium/issues/725
USER root
RUN apt-get update -y
RUN apt-get install -y firefox python-pip
WORKDIR /scraper
COPY . /scraper
RUN pip install -r /scraper/requirements.txt
ENV MOZ_HEADLESS=1
CMD ["python", "/scraper/browserscraper.py"]
If I run those instructions in the Dockerfile from an interactive terminal, I have no problems.
It either has to do with the user being root running the script via the Dockerfile or something about it missing a screen for output because I'm not actually SSH'ed in like I am running it from the command line with -it.
Any ideas?

vsts-npm-auth can't get authentication token on VSTS build

I'm attempting to use vsts-npm-auth to get the authentication token for our VSTS package repository. On my development machine I can run the commands
npm install -g vsts-npm-auth
vsts-npm-auth -config path-to-my\.npmrc
and it succeeds in providing me with an authentication token. I'm now trying to recreate this as a build step on VSTS, so I create the powershell script auth-vsts.ps1
$npmrcFile = "$PSScriptRoot\path-to-my\.npmrc";
npm install -g vsts-npm-auth;
vsts-npm-auth -config $npmrcFile;
and add it as a powershell task. However, the task fails as follows
2017-05-30T09:37:41.1082686Z ##[section]Starting: auth-vsts
2017-05-30T09:37:41.1092712Z ==============================================================================
2017-05-30T09:37:41.1092712Z Task : PowerShell
2017-05-30T09:37:41.1092712Z Description : Run a PowerShell script
2017-05-30T09:37:41.1092712Z Version : 1.2.3
2017-05-30T09:37:41.1092712Z Author : Microsoft Corporation
2017-05-30T09:37:41.1092712Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkID=613736)
2017-05-30T09:37:41.1092712Z ==============================================================================
2017-05-30T09:37:41.1112679Z ##[command]. 'd:\a\1\s\auth-vsts.ps1'
2017-05-30T09:37:47.3792461Z C:\NPM\Modules\vsts-npm-auth -> C:\NPM\Modules\node_modules\vsts-npm-auth\bin\vsts-npm-auth.exe
2017-05-30T09:37:47.3792461Z C:\NPM\Modules
2017-05-30T09:37:47.3802239Z `-- vsts-npm-auth#0.25.0
2017-05-30T09:37:47.3802239Z
2017-05-30T09:37:47.3802239Z
2017-05-30T09:37:47.3802239Z vsts-npm-auth v0.25.0.0
2017-05-30T09:37:47.3802239Z -----------------------
2017-05-30T09:37:47.3802239Z Creating npmrcFile. Path: D:\a\1\s\.npmrc
2017-05-30T09:37:47.3802239Z Getting new credentials for source:https://our-domain/_packaging/SharedLib/npm/registry/, scope:vso.packaging_write vso.drop_write
2017-05-30T09:37:49.8729702Z Caught exception: The prompt option is invalid because the process is not interactive.
2017-05-30T09:37:49.8729702Z Parameter name: PromptType
2017-05-30T09:37:49.8729702Z Caught exception: The prompt option is invalid because the process is not interactive.
2017-05-30T09:37:49.8729702Z Parameter name: PromptType
2017-05-30T09:37:49.8729702Z Couldn't get an authentication token for //our-domain/_packaging/SharedLib/npm/registry/:_authToken.
2017-05-30T09:37:50.1769711Z ##[error]Process completed with exit code 1.
2017-05-30T09:37:50.1809715Z ##[section]Finishing: auth-vsts
The error gives no indication as to why it can't obtain the credentials. Any ideas why this might be?
I faced this issue while trying to execute via Visual Studio Code`s powershell terminal
vsts-npm-auth -config .npmrc
But running the same command via simple console solved this issue and I was redirected to authentication window.
Can suggest that due to internal limitations powershell disabled to open another windows.
The error did indicate why it cannot obtain the credentials:
The prompt option is invalid because the process is not interactive.
This could be caused by the build agent does not run in interactive mode which make the credential dialog cannot be prompted. If you are using Hosted Build Agent, the build agent is run as service and there isn't any way to change to interactive mode.
However, the issue here is that if you want to use the feed in a build step, it does not make sense to prompt a credential dialog during the build process since the build step cannot enter the required credential automatically. Not sure if there is any specific requirement in your environment, but the general workflow should be uploading the .npmrc file generated in your local machine to the Source Control so that npm can use the auth token in the file to install/publish packages to VSTS Feed.
Inside your project, you can open a terminal and run
vsts-npm-auth -F -C .npmrc
This script refreshes the npm token. Here I set two parameters: -F forces the refresh (if not set, the token is refreshed only if it is already expired), while -C fileName defines the configuration file.
The vsts authentication system sometimes authenticates the use by popping up a browser window. If the terminal you're running the command from is not interactive (e.g., ssh terminal, vscode terminal) it won't be able to pop up that window, and the authentication will fail.
This worked for me
npx vsts-npm-auth -config .npmrc

"docker run -dti" with a dumb terminal

updated: added the missing docker attach.
Hi am trying to run a docker container, with -dti. but I cannot access with a terminal set to dumb. is there a way to change this (it is currently set to xterm, even though my ssh client is dumb)
example:
create the container
docker run -dti --name test -v /my-folder alpine /bin/ash
docker attach test
apk --update add nodejs
cd /my-folder
npm install -g gulp
the last command always contains ascii escape chars to move the cursor.
I have tried "export TERM=dumb" inside the running container, but it does not work.
is there a way to "run" this using the dumb terminal?
I am running this from a script on another computer, via (dumb) ssh.
using the -t which sets this https://docs.docker.com/engine/reference/run/#env-environment-variables, however removing effects the command prompt (the prompt is not shown)
possible solution 1 remove the -t and keep the -i. To see if the command has completed echo out a known token (ENDENDEND). ie
docker run -di --name test -v /my-folder alpine /bin/ash
docker attach test
apk --update add nodejs;echo ENDENDEND
cd /my-folder;echo ENDENDEND
npm install -g gulp;echo ENDENDEND
not pretty, but it works (there is no ascii in the results)
Possible solution 2 use the journal, docker can log out to the linux journal, this can be gathered as commands are executed in the container. (I have yet to fully test this one out. however the log seems to be a nicer output of what happened)
update:
Yep -t is the problem.
However if you want to see the entire process when running a command, maybe this way is better:
docker run -di --name test -v/my-folder alpine /bin/ash
docker exec -it test /bin/ash
finally you need to kill the container after all jobs finished.
docker run -d means "Run container in background and print container ID"
not start the container as a daemon
I was hitting this issue on OSx running docker, i had to do 2 things to stop the terminal/ascii/ansi escape sequences.
remove the "t" option on the docker run command (from docker run -it ... to docker run -i...)
ensure to force bash or sh shells used on osx when running the command from a script file, not the default zsh
Also
the escape sequences were not always visible on the terminal
even so, they still usually caused content corruption, even with SED brought to bear
they always were shown in my editor

How to fail gitlab CI build?

I am trying to fail a build in gitlab CI and get email notification about it.
My build script is this:
echo "Listing files!"
ls -la
echo "##########################Preparing build##########################"
mkdir build
cd build
echo "Generating make files"
cmake -G "Unix Makefiles" -D CMAKE_BUILD_TYPE=Release -D CMAKE_VERBOSE_MAKEFILE=on ..
echo "##########################Building##########################"
make
I have commited the code that breaks build. However, instead of finishing, build seems to be stuck in "running" state after exiting make. Last line is:
make: *** [all] Error 2
I also get no notifications.
How can i diagnose what is happening?
Upd.: in runner, following is repeated in log:
Submitting build <..> to coordinator...response error: 500
In production.log and sideq.log of gitlab_ci, following is written:
ERROR: Error connecting to Redis on localhost:6379 (ECONNREFUSED)
Full message with stacktrace is here: pastebin.
I have the same problem, i can help you with a workaround but im trying to fully fix it.
1- most of the times he hangs but the jobs keeps on going and actually finishes it, you can see the processes inside the machine, example: in my case it compiles and in the end it uses docker to publish the build, so the process docker doesn't exist until he reaches that phase.
2- to workaround this issue you have to make the data persistent and "retry" the download over and over again until he downloads everything he needs.
PS: stating what kind of OS you are using always helps.