I'd like to get some statistics data from TeamCity. I don't know if It's better to do this using TeamCity REST API or by queering TeamCity Database to get the data I need.
Can someone advice me which one is better ?
FYI: I tried to use TeamCity Restful web services but if I request a project containing a lot of build configs, I get a timeout exception :
iagnostic.web.DiagnosticFilter - Request processing took too long: 192577 ms, request: GET '/app/rest/builds/?locator= ...
TeamCity version: 9.1.1(build 37059)
I finally used REST API. I get failed tests of the last build for each buildConfig.
Related
I'm trying to get the latest build number from a specific branch. I'm able to do with the below api call. It get the build number whether the job is success or failed, but not if its ABORTED. can anyone let me know the api call to get the build number even if thats aborted?
api call - curl https://bamboo.test.com/rest/api/latest/result/AVC-SFADR15?max-results=1
includeAllStates is the paramter you need
Use below command:
curl https://bamboo.test.com/rest/api/latest/result/AVC-SFADR15?max-results=1&includeAllStates
I'm developing a application that can collect mpreduce job progress info to analyze.The first way is parse log file.but It's ugly。Is there any method like hook or plugin can do this
You can probably use the YARN application API to get most of the information. See this Yarn Application API
Here is an excerpt from the page:
... All query parameters for this api will filter on all applications. However the queue query parameter will only implicitly filter on unfinished applications that are currently in the given queue.
There are other YARN APIs too, that you can utilize to achieve your goal. It is certainly better than scanning log files.
I have a spark streaming program running on Yarn Cluster in "yarn-cluster" mode. (-master yarn-cluster).
I want to fetch spark job statistics using REST APIs in json format.
I am able to fetch basic statistics using REST url call:
http://yarn-cluster:8088/proxy/application_1446697245218_0091/metrics/json. But this is giving very basic statistics.
However I want to fetch per executor or per RDD based statistics.
How to do that using REST calls and where I can find the exact REST url to get these statistics.
Though $SPARK_HOME/conf/metrics.properties file sheds some light regarding urls i.e.
5. MetricsServlet is added by default as a sink in master, worker and client driver, you can send http request "/metrics/json" to get a snapshot of all the registered metrics in json format. For master, requests "/metrics/master/json" and "/metrics/applications/json" can be sent seperately to get metrics snapshot of instance master and applications. MetricsServlet may not be configured by self.
but that is fetching html pages not json. Only "/metrics/json" fetches stats in json format.
On top of that knowing application_id pro-grammatically is a challenge in itself when running in yarn-cluster mode.
I checked REST API section of Spark Monitoring page, but that didn't worked when we run spark job in yarn-cluster mode. Any pointers/answers are welcomed.
You should be able to access the Spark REST API using:
http://yarn-cluster:8088/proxy/application_1446697245218_0091/api/v1/applications/
From here you can select the app-id from the list and then use the following endpoint to get information about executors, for example:
http://yarn-cluster:8088/proxy/application_1446697245218_0091/api/v1/applications/{app-id}/executors
I verified this with my spark streaming application that is running in yarn cluster mode.
I'll explain how I arrived at the JSON response using a web browser. (This is for a Spark 1.5.2 streaming application in yarn-cluster mode).
First, use the hadoop url to view the RUNNING applications. http://{yarn-cluster}:8088/cluster/apps/RUNNING.
Next, select a running application, say http://{yarn-cluster}:8088/cluster/app/application_1450927949656_0021.
Next, click on the TrackingUrl link. This uses a proxy and the port is different in my case: http://{yarn-proxy}l:20888/proxy/application_1450927949656_0021/. This shows the spark UI. Now, append the api/v1/applications to this URL: http://{yarn-proxy}l:20888/proxy/application_1450927949656_0021/api/v1/applications.
You should see a JSON response with the application name supplied to SparkConf and the start time of the application.
I was able to reconstruct the metrics in the columns seen in the Spark Streaming web UI (batch start time, processing delay, scheduling delay) using the /jobs/ endpoint.
The script I used is available here. I wrote a short post describing and tying its functionality back to the Spark codebase. This does not need any web-scraping.
It works for Spark 2.0.0 and YARN 2.7.2, but may work for other version combinations too.
You'll need to scrape through the HTML page to get the relevant metrics. There isn't a Spark rest endpoint for capturing this info.
I want to start a simple process that can be in activiti-explorer or I deployed,using activiti-rest api. I use SoapUI or RESTClient.Actually I wrote URL and I can see all of the process,after I start the process with rest-services, I can see the process started in SoapUI,becasuse the reponse is true,but in activiti-explorer,process isn t starting.What should I do? I m waiting your helps
Thanks
My guess is that you deployed the activiti-webapp-explorer2 and activiti-webapp-rest as separate war files and each is using an in memory database.
This means, the applications are essentially seperate.
You have two choices.
Deploy as separate webapps but set them to use the same database instance/
Merge the rest API jar into the explorer webapp.
Both methods are acceptable.
I'm trying to build and deploy a web application using MSBuild and TeamCity. I've read multiple SO posts and finally decided on using these parameters
/p:Configuration=Debug
/p:OutputPath=bin
/p:DeployOnBuild=True
/p:DeployTarget=MSDeployPublish
/p:username=BuildUser
/p:password=Password1
/p:AllowUntrustedCertificate=True
/p:DeployIisAppPath=<siteName>
/p:MSDeployPublishMethod=WMSVC
/p:MsDeployServiceUrl=http://<servername>
However, I get the following error:
Could not complete the request to remote agent URL .(Could not
complete the request to remote agent URL
'https://10.10.10.42/MSDEPLOYAGENTSERVICE:8172/msdeploy.axd?site=CFS.Services.Hosting'.)
For some reason either teamcity or msbuild assumes that I want to deploy via https which I don't. If I set the p:MsDeployServiceUrl = http:// like I do in Visual Studio
I get the following error:
Could not complete the request to remote agent URL
'https://http//10.10.10.42:8172/msdeploy.axd?site=CFS.Services.Hosting'.
Any help would be greatly appreciated.
Have you tried using MSDeployPublishMethod=RemoteAgent? Based on the value for MSDeployServiceURL (i.e. a URL using http instead of https) I'm guessing that you are using the remote agent service instead of calling MSDeploy via the Web Management Service. If that's not right let me know and I can try and dig into this a bit more.