How to upload data from pc to firestore. Getting "Failed to detect whether we are running on Google Compute Engine." - kotlin

I'm trying to upload a couple of xml files to firestore from my computer (1000 or so). The users in my android app need to be able to search for these files, and because firebase storage doesn't provide that feature I thought I'd just dump them in firestore.
To test it I'm trying to upload 10 files for a start. The code I have looks something like this:
fun main(){
val db = getDatabase()
getXMLFiles().take(10).forEach { file ->
db.collection("xml").add(XMLFile(file.name, file))
}
}
private fun getDatabase() : Firestore {
val serviceAccount = FileInputStream("<project name>-firebase-adminsdk-<some other stuff>.json")
val firestoreOptions = FirestoreOptions.newBuilder()
.setTimestampsInSnapshotsEnabled(true).build()
val options = FirebaseOptions.Builder()
.setCredentials(GoogleCredentials.fromStream(serviceAccount))
.setDatabaseUrl("https://<project name>.firebaseio.com")
.setFirestoreOptions(firestoreOptions)
.build()
FirebaseApp.initializeApp(options)
return FirestoreClient.getFirestore()
}
fun getXMLFiles() = File("/XML").walk().asIterable()
data class XMLFile(val name: String, val file: File)
Nothing gets uploaded and half the time (weirdly inconsistently) I get this error message:
Apr 15, 2019 2:05:53 PM com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
WARNING: Failed to detect whether we are running on Google Compute Engine.
java.net.ConnectException: No route to host (connect failed)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.Socket.connect(Socket.java:591)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1242)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1181)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1075)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1009)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:104)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.auth.oauth2.ComputeEngineCredentials.runningOnComputeEngine(ComputeEngineCredentials.java:210)
at com.google.auth.oauth2.DefaultCredentialsProvider.tryGetComputeCredentials(DefaultCredentialsProvider.java:290)
at com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentialsUnsynchronized(DefaultCredentialsProvider.java:207)
at com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:124)
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:127)
at com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:100)
at com.google.cloud.ServiceOptions.defaultCredentials(ServiceOptions.java:304)
at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:278)
at com.google.cloud.firestore.FirestoreOptions.<init>(FirestoreOptions.java:225)
at com.google.cloud.firestore.FirestoreOptions$Builder.build(FirestoreOptions.java:219)
at UploaderKt.getDatabase(Uploader.kt:32)
at UploaderKt.main(Uploader.kt:13)
at UploaderKt.main(Uploader.kt)
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Process finished with exit code 0
I checked, and it does find the .json with the credentials, so that is not the problem. I can't find anything anywhere about the error message
Apr 15, 2019 2:05:53 PM com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
WARNING: Failed to detect whether we are running on Google Compute Engine.
So I hoped someone could help.
Btw, without showing to much, my .json looks like this:
{
"type": "service_account",
"project_id": "XXX",
"private_key_id": "XXX",
"private_key": "-----BEGIN PRIVATE KEY-----XXX-----END PRIVATE KEY-----\n",
"client_email": "firebase-adminsdk-XXX#XXX.iam.gserviceaccount.com",
"client_id": "XXX",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/firebase-adminsdk-XXX%40XXX.iam.gserviceaccount.com"
}
Edit: might be stating the obvious but I have no problems connecting to the internet otherwise

Related

Ktor-server-test-host did not cleaned up Exposed database instannce across tests

I'm working on a web service using Ktor 1.6.8 and Exposed 0.39.2.
My application module and database is setup as following:
fun Application.module(testing: Boolean = false) {
val hikariConfig =
HikariConfig().apply {
driverClassName = "org.postgresql.Driver"
jdbcUrl = environment.config.propertyOrNull("ktor.database.url")?.getString()
username = environment.config.propertyOrNull("ktor.database.username")?.getString()
password = environment.config.propertyOrNull("ktor.database.password")?.getString()
maximumPoolSize = 10
isAutoCommit = false
transactionIsolation = "TRANSACTION_REPEATABLE_READ"
validate()
}
val pool = HikariDataSource(hikariConfig)
val db = Database.connect(pool, {}, DatabaseConfig { useNestedTransactions = true })
}
I use ktor-server-test-host, Test Containers and junit 5 to test the service. My test looks similar like below:
#Testcontainers
class SampleApplicationTest{
companion object {
#Container
val postgreSQLContainer = PostgreSQLContainer<Nothing>(DockerImageName.parse("postgres:13.4-alpine")).apply {
withDatabaseName("database_test")
}
}
#Test
internal fun `should make request successfully`() {
withTestApplication({
(environment.config as MapApplicationConfig).apply {
put("ktor.database.url", postgreSQLContainer.jdbcUrl)
put("ktor.database.user", postgreSQLContainer.username)
put("ktor.database.password", postgreSQLContainer.password)
}
module(testing = true)
}) {
handleRequest(...)
}
}
}
I observed an issue that if I ran multiple test classes together, some requests ended up using old Exposed db instance that was setup in a previous test class, causing the test case failed because the underlying database was already stopped.
When I ran one test class at a time, all were running fine.
Please refer to the log below for the error stack trace:
2022-10-01 08:00:36.102 [DefaultDispatcher-worker-5 #request#103] WARN Exposed - Transaction attempt #1 failed: java.sql.SQLTransientConnectionException: HikariPool-4 - Connection is not available, request timed out after 30001ms.. Statement(s): INSERT INTO cards (...)
org.jetbrains.exposed.exceptions.ExposedSQLException: java.sql.SQLTransientConnectionException: HikariPool-4 - Connection is not available, request timed out after 30001ms.
at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed_core(Statement.kt:49)
at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:143)
at org.jetbrains.exposed.sql.Transaction.exec(Transaction.kt:128)
at org.jetbrains.exposed.sql.statements.Statement.execute(Statement.kt:28)
at org.jetbrains.exposed.sql.QueriesKt.insert(Queries.kt:73)
at com.example.application.services.CardService$createCard$row$1.invokeSuspend(CardService.kt:53)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt$suspendedTransactionAsyncInternal$1.invokeSuspend(Suspended.kt:127)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.sql.SQLTransientConnectionException: HikariPool-4 - Connection is not available, request timed out after 30001ms.
at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:695)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:100)
at org.jetbrains.exposed.sql.Database$Companion$connect$3.invoke(Database.kt:142)
at org.jetbrains.exposed.sql.Database$Companion$connect$3.invoke(Database.kt:139)
at org.jetbrains.exposed.sql.Database$Companion$doConnect$3.invoke(Database.kt:127)
at org.jetbrains.exposed.sql.Database$Companion$doConnect$3.invoke(Database.kt:128)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction$connectionLazy$1.invoke(ThreadLocalTransactionManager.kt:69)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction$connectionLazy$1.invoke(ThreadLocalTransactionManager.kt:68)
at kotlin.UnsafeLazyImpl.getValue(Lazy.kt:81)
at org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManager$ThreadLocalTransaction.getConnection(ThreadLocalTransactionManager.kt:75)
at org.jetbrains.exposed.sql.Transaction.getConnection(Transaction.kt)
at org.jetbrains.exposed.sql.statements.InsertStatement.prepared(InsertStatement.kt:157)
at org.jetbrains.exposed.sql.statements.Statement.executeIn$exposed_core(Statement.kt:47)
... 19 common frames omitted
Caused by: org.postgresql.util.PSQLException: Connection to localhost:49544 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:303)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:465)
at org.postgresql.Driver.connect(Driver.java:264)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477)
at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:725)
at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:711)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.Net.pollConnect(Native Method)
at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
I tried to add some cleanup code for Exposed's TransactionManager in my application module as following:
fun Application.module(testing: Boolean = false) {
// ...
val db = Database.connect(pool, {}, DatabaseConfig { useNestedTransactions = true })
if (testing) {
environment.monitor.subscribe(ApplicationStopped) {
TransactionManager.closeAndUnregister(db)
}
}
}
However, the issue still happened, and I also observed additional error as following:
2022-10-01 08:00:36.109 [DefaultDispatcher-worker-5 #request#93] ERROR Application - Unexpected error
java.lang.RuntimeException: database org.jetbrains.exposed.sql.Database#3bf4644c don't have any transaction manager
at org.jetbrains.exposed.sql.transactions.TransactionApiKt.getTransactionManager(TransactionApi.kt:149)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt.closeAsync(Suspended.kt:85)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt.access$closeAsync(Suspended.kt:1)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt$suspendedTransactionAsyncInternal$1.invokeSuspend(Suspended.kt:138)
(Coroutine boundary)
at org.mpierce.ktor.newrelic.KtorNewRelicKt$runPipelineInTransaction$2.invokeSuspend(KtorNewRelic.kt:178)
at org.mpierce.ktor.newrelic.KtorNewRelicKt$setUpNewRelic$2.invokeSuspend(KtorNewRelic.kt:104)
at io.ktor.routing.Routing.executeResult(Routing.kt:154)
at io.ktor.routing.Routing$Feature$install$1.invokeSuspend(Routing.kt:107)
at io.ktor.features.ContentNegotiation$Feature$install$1.invokeSuspend(ContentNegotiation.kt:145)
at io.ktor.features.StatusPages$interceptCall$2.invokeSuspend(StatusPages.kt:102)
at io.ktor.features.StatusPages.interceptCall(StatusPages.kt:101)
at io.ktor.features.StatusPages$Feature$install$2.invokeSuspend(StatusPages.kt:142)
at io.ktor.features.CallLogging$Feature$install$2.invokeSuspend(CallLogging.kt:188)
at io.ktor.server.testing.TestApplicationEngine$callInterceptor$1.invokeSuspend(TestApplicationEngine.kt:296)
at io.ktor.server.testing.TestApplicationEngine$2.invokeSuspend(TestApplicationEngine.kt:50)
Caused by: java.lang.RuntimeException: database org.jetbrains.exposed.sql.Database#3bf4644c don't have any transaction manager
at org.jetbrains.exposed.sql.transactions.TransactionApiKt.getTransactionManager(TransactionApi.kt:149)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt.closeAsync(Suspended.kt:85)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt.access$closeAsync(Suspended.kt:1)
at org.jetbrains.exposed.sql.transactions.experimental.SuspendedKt$suspendedTransactionAsyncInternal$1.invokeSuspend(Suspended.kt:138)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Could someone show me what could be the issue here with my application code & test setup?
Thanks and regards.

Can't Import Automation Account Storage Module using Bicep

Hi im trying to import an automaiton account module to an Automatino account that i have Using Bicep
the Module
Az.Storage
version : 2.0.0
Following the Documentation im using this code :
resource znssPSModulesAzStorageName 'Microsoft.Automation/automationAccounts/modules#2020-01-13-preview' = {
name: psModules.azStorage.name
location: location
parent: znssAutomationAccountName
tags:{}
properties: {
contentLink:{
uri: 'https://www.powershellgallery.com/api/v2/packages/Az.Storage/2.0.0'
}
}
}
but im getting this error :
Error importing the module Az.Storage. Import failed with the
following error:
Orchestrator.Shared.AsyncModuleImport.ModuleImportException: No
content was read from the supplied ContentLink.
[ContentLink.Uri=https://www.powershellgallery.com/api/v2/packages/Az.Storage/2.0.0]
at
Orchestrator.Activities.GetModuleContentActivity.ExecuteInternal(CodeActivityContext
context, String contentUri, String contentHashAlgorithm, String
contentHashValue, String contentVersion, String moduleName,
ModuleLanguage moduleLanguage) at
Orchestrator.Activities.GetModuleContentActivity.Execute(CodeActivityContext
context) at
System.Activities.CodeActivity.InternalExecute(ActivityInstance
instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
at
System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor
executor, BookmarkManager bookmarkManager, Location resultLocation)
is there a problem with the link in powershell Gallery
i searched in the internet and couldn't find anything useful
hope someone can help me
the Problem is in the uri instead of :
uri: 'https://www.powershellgallery.com/api/v2/packages/Az.Storage/2.0.0
use : package singular wihout the s
uri: 'https://www.powershellgallery.com/api/v2/package/Az.Storage/2.0.0
it should work :
resource znssPSModulesAzStorageName 'Microsoft.Automation/automationAccounts/modules#2020-01-13-preview' = {
name: psModules.azStorage.name
location: location
parent: znssAutomationAccountName
tags:{}
properties: {
contentLink:{
uri: 'https://www.powershellgallery.com/api/v2/package/Az.Storage/2.0.0'
}
}
}

[Karate Test][Saucelab]: Session disconnecting after launching mobile app installed in Saucelab

I am launching android test on Saucelab, however I can see the script is launching the mobile app on saucelab but not able to perform any action on it and throwing this exception:
12:54:20.931 [main] ERROR com.intuit.karate - java.net.SocketTimeoutException: Read timed out, http call failed after 32303 milliseconds for url: https://oauth-abdulkadir786-684f1:1bd00f8e-392f-4b4c-8f0e-2597cc9912a3#ondemand.eu-central-1.saucelabs.com:443/wd/hub/session
I am using the following steps for execute my test:
Configure driver in karate-config.js
var android = {}
android["desiredConfig"] = {
"accessKey":"1bd00f8e-392f-4b4c-8f0e-2597cc9912a3",
"deviceName":"Android GoogleAPI Emulator",
"app" : "storage:b82d6099-60ad-49dd-b15f-925166e03dcd",
"platformVersion" : "11.0",
"platformName" : "Android",
"newCommandTimeout":300,
"automationName" : "UiAutomator2",
"username": "oauth-abdulkadir786-684f1"
}
config["android"] = android
Then Writing the feature file to call the webDriverSession:
Feature: Calling test from Sauce labs
Background:
* configure driver = { type: 'android', start: false, webDriverUrl: 'https://oauth-abdulkadir786-684f1:1bd00f8e-392f-4b4c-8f0e-2597cc9912a3#ondemand.eu-central-1.saucelabs.com:443/wd/hub' }
Scenario: android mobile app UI tests
Given driver { webDriverSession: { desiredCapabilities : "#(android.desiredConfig)"} }
* delay(2000)
Then click('/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/android.widget.FrameLayout/android.widget.ScrollView/android.widget.LinearLayout/android.widget.LinearLayout/android.widget.LinearLayout[2]/android.widget.Button[3]')
Any help would be highly appreciated.....
It looks like Karate is timing out the session before the emulator and app gets a chance to load fully:
Read timed out, http call failed after 32303 milliseconds for url: https://oauth-abdulkadir786-684f1:1bd00f8e-392f-4b4c-8f0e-2597cc9912a3#ondemand.eu-central-1.saucelabs.com:443/wd/hub/session
The suspicious parts are Read timed out, 32303 milliseconds (which is pretty close to 30 seconds and so is probably a config thing; 30 or 60 seconds is a common default timeout) and the path /wd/hub/session looks like the initial POST request which starts a sesson.
You'll probably have more luck if you increase the connectTimeout and readTimeout, either in your Background or in karate-config.js:
karate.configure('connectTimeout', 60000);
karate.configure('readTimeout', 60000);

How to design worker verticle with infinite blocking loop?

I am trying to compose a worker verticle that will bridge Google cloud PubSub topic subscription with an event-bus of the vert.x by adopting kotlin example of PubSub combined with this answer regarding worker with an infinite blocking loop processing.
It does works but Vert.X keep nagging into the log that Thread blocked by throwing an exception sometime after the message was from PubSub was received (please ignore blocking initialization for now):
9:15:12 AM: Executing task 'run'...
WARNING: You are a using release candidate 2.0.0-rc5. Behavior of this plugin has changed since 1.3.5. Please see release notes at: https://github.com/GoogleCloudPlatform/app-gradle-plugin.
Missing a feature? Can't get it to work?, please file a bug at: https://github.com/GoogleCloudPlatform/app-gradle-plugin/issues.
:compileKotlin UP-TO-DATE
:compileJava NO-SOURCE
:processResources NO-SOURCE
:classes UP-TO-DATE
Mar 10, 2019 9:15:18 AM io.vertx.core.impl.launcher.commands.Watcher
INFO: Watched paths: [/home/username/IdeaProjects/project_name/./src]
Mar 10, 2019 9:15:18 AM io.vertx.core.impl.launcher.commands.Watcher
INFO: Starting the vert.x application in redeploy mode
:run
Starting vert.x application...
f48ba7fd-a52b-487f-b553-2b74473e58ba-redeploy
Creating topic gcs-project-id:vertx.
Mar 10, 2019 9:15:18 AM com.google.auth.oauth2.DefaultCredentialsProvider warnAboutProblematicCredentials
WARNING: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/.
Mar 10, 2019 9:15:21 AM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 2759 ms, time limit is 2000 ms
Topic gcs-project-id:vertx successfully created.
Creating subscription gcs-project-id:kotlin.
Mar 10, 2019 9:15:22 AM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 3759 ms, time limit is 2000 ms
Mar 10, 2019 9:15:23 AM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 4758 ms, time limit is 2000 ms
Mar 10, 2019 9:15:24 AM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 5759 ms, time limit is 2000 ms
io.vertx.core.VertxException: Thread blocked
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:469)
at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:142)
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1309)
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:52)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.pubsub.v1.SubscriptionAdminClient.createSubscription(SubscriptionAdminClient.java:359)
at com.google.cloud.pubsub.v1.SubscriptionAdminClient.createSubscription(SubscriptionAdminClient.java:260)
at com.example.project.MainVerticle.subscribeTopic(MainVerticle.kt:76)
at com.example.project.MainVerticle.init(MainVerticle.kt:46)
at io.vertx.core.impl.DeploymentManager.lambda$doDeploy$8(DeploymentManager.java:492)
at io.vertx.core.impl.DeploymentManager$$Lambda$28/1902260856.handle(Unknown Source)
at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:320)
at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38)
at io.vertx.core.impl.EventLoopContext$$Lambda$29/1640639994.run(Unknown Source)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Subscription gcs-project-id:kotlin successfully created.
Listening to messages on kotlin:
Mar 10, 2019 9:15:25 AM io.vertx.core.impl.launcher.commands.VertxIsolatedDeployer
INFO: Succeeded in deploying verticle
Message Id: 462746807438186 Data: Bazinga
Message Id: 462746750387788 Data: Another message
Mar 10, 2019 9:16:25 AM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-worker-thread-0,5,main] has been blocked for 60171 ms, time limit is 60000 ms
io.vertx.core.VertxException: Thread blocked
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
at com.example.project.MainVerticle$start$1.handle(MainVerticle.kt:32)
at com.example.project.MainVerticle$start$1.handle(MainVerticle.kt:13)
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:272)
at io.vertx.core.impl.ContextImpl$$Lambda$33/1101004004.run(Unknown Source)
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
at io.vertx.core.impl.TaskQueue$$Lambda$26/1213216872.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Mar 10, 2019 9:16:26 AM io.vertx.core.impl.BlockedThreadChecker
WARNING: Thread Thread[vert.x-worker-thread-0,5,main] has been blocked for 61172 ms, time limit is 60000 ms
io.vertx.core.VertxException: Thread blocked
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
at com.example.project.MainVerticle$start$1.handle(MainVerticle.kt:32)
at com.example.project.MainVerticle$start$1.handle(MainVerticle.kt:13)
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:272)
at io.vertx.core.impl.ContextImpl$$Lambda$33/1101004004.run(Unknown Source)
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
at io.vertx.core.impl.TaskQueue$$Lambda$26/1213216872.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
And here is source-code:
package com.example.project_name
import com.google.api.gax.rpc.ApiException
import com.google.cloud.pubsub.v1.*
import com.google.pubsub.v1.ProjectSubscriptionName
import com.google.pubsub.v1.ProjectTopicName
import com.google.pubsub.v1.PubsubMessage
import com.google.pubsub.v1.PushConfig
import io.vertx.core.*
import java.util.concurrent.LinkedBlockingDeque
class MainVerticle : MessageReceiver, AbstractVerticle() {
private val projectId = "gcs-project-id"
private val topicId = "vertx"
private val topic: ProjectTopicName = ProjectTopicName.of(projectId, topicId)
private val subscriptionId = "kotlin"
private val subscription = ProjectSubscriptionName.of(projectId, subscriptionId)
private val messages = LinkedBlockingDeque<PubsubMessage>()
private lateinit var subscriber: Subscriber
override fun receiveMessage(message: PubsubMessage, consumer: AckReplyConsumer) {
messages.offer(message)
consumer.ack()
}
override fun start() {
vertx.executeBlocking<Void>({
try {
println("Listening to messages on $subscriptionId:")
subscriber.awaitRunning()
while (true) {
val message = messages.take()
println("Message Id: ${message.messageId} Data: ${message.data.toStringUtf8()}")
}
} finally {
subscriber.stopAsync()
it.complete()
}
}, { println("done, ${it.cause()}") })
}
override fun init(vertx: Vertx?, context: Context?) {
super.init(vertx, context)
try {
createTopic()
subscribeTopic()
subscriber = Subscriber.newBuilder(subscription, this).build()
subscriber.startAsync()
} catch (e: ApiException) {
// example : code = ALREADY_EXISTS(409) implies topic already exists
println("Failed: $e")
}
}
override fun stop(stopFuture: Future<Void>?) {
super.stop(stopFuture)
try {
deleteSub()
deleteTopic()
} catch (e: ApiException) {
println("Failed: $e")
} finally {
subscriber.stopAsync()
stopFuture!!.complete()
}
}
private fun createTopic() { // expects 1 arg: <topic> to create
println("Creating topic ${topic.project}:${topic.topic}.")
TopicAdminClient.create().use { topicAdminClient -> topicAdminClient.createTopic(topic) }
println("Topic ${topic.project}:${topic.topic} successfully created.")
}
private fun subscribeTopic() { // expects 2 args: <topic> and <subscription>
println("Creating subscription ${subscription.project}:${subscription.subscription}.")
SubscriptionAdminClient.create().use { it.createSubscription(subscription, topic, PushConfig.getDefaultInstance(), 0) }
println("Subscription ${subscription.project}:${subscription.subscription} successfully created.")
}
private fun deleteTopic() {
println("Deleting topic ${topic.project}:${topic.topic}.")
TopicAdminClient.create().use { it.deleteTopic(topic) }
println("Topic ${topic.project}:${topic.topic} successfully deleted.")
}
private fun deleteSub() { // expects 1 arg: <subscription> to delete
println("Deleting subscription ${subscription.project}:${subscription.subscription}.")
SubscriptionAdminClient.create().use { it.deleteSubscription(subscription) }
println("Subscription ${subscription.project}:${subscription.subscription} successfully deleted.")
}
}
fun main(vararg args: String) {
Vertx.vertx().deployVerticle(MainVerticle(), DeploymentOptions().apply {
isWorker = true
})
}
I am clearly missing something. Also if you have a better approach that can integrate/unify Google’s PubSub library (that has its own async loop) with Vert.X I’d be happy to hear over my primitive example approach.
The problem is with your while loop.
"Blocking" in this case does not mean that you can just keep running forever.
Your call to it.complete() is never reached, and at some point Vert.x will complain about that.
See the manual on Running blocking code, specifically the WARNING section.
To solve your problem, you will need to schedule your calls to messages.take() in one way or another, for example using setPeriodic. Inside the interval handler, empty your queue with executeBlocking, then give back control by calling complete(), either before or after you have scheduled the processing of the messages, depending if you care about the result.

How To Retrieve Data From Rest API In Appcelerator

I have the misfortune of further developing an existing mobile application in Appcelerator. The app uses a Rest API on a remote server to read and write data. The API works well in test environments and in production. I need to post data to the API and read the output. Here is an example of what the output of the API looks like after a POST command:
{
"equipment":
{
"result": "create",
"id": 419213
},
"_meta":
{
"offset": 0,
"limit": -1,
"total_results": 1,
"url": "http://localhost:8080/api/v1/equipment",
"utc_start_time": 1459449461115,
"nano_total_time": 74771
}
}
I am able to successfully post the data in Appcelerator. I have verified this in the database that the CRUD operation is acting on. However, I am unable to get the aforementioned data from the httpClient object that makes the call, despite following the directions in the outdated Titanium documentation.
Here is my Appcelerator code:
var payload = "name=atad&asset_number=adtasd&department_id=185080&property_id=10086&designator_id=379828&is_leased=N&is_assignable_asset=N&status=A";
var url = "http://localhost:8080/api/v1/equipment";
var client = Ti.Network.createHTTPClient({
onload : function(e) {
Ti.API.info(e); // {}
Ti.API.info(e.source); // []
Ti.API.info(JSON.stringify(e.source)); // {}
Ti.API.info(JSON.stringify(e.source.reponseText)); // null
Ti.API.info(JSON.stringify(e.source.reponseData)); // null
Ti.API.info(this); // []
console.log(JSON.stringify(this)); // {}
Ti.API.info(JSON.stringify(this.reponseText)); // null
Ti.API.info(this.reponseData); // null
}
,onerror : function(e){
Ti.API.info(e);
alert("error");
}
});
var auth = 'Basic ' + Ti.App.Properties.getString('auth');
client.open("POST", apiUrl);
client.setRequestHeader('Authorization', auth);
client.setRequestHeader('Content-Type', 'text/plain');
client.send(payload);
Here is the console output:
[INFO] : {
[INFO] : code = 0;
[INFO] : source = "[object TiNetworkHTTPClient]";
[INFO] : success = 1;
[INFO] : type = load;
[INFO] : }
[INFO] : [object TiNetworkHTTPClient]
[INFO] : {"method":"POST","url":"http://localhost:8080/api/v1/equipment"}
[INFO] : <null>
[INFO] : <null>
[INFO] : [object TiNetworkHTTPClient]
[INFO] : {"method":"POST","url":"http://localhost:8080/api/v1/equipment"}
[INFO] : <null>
[INFO] : <null>
The documentation here http://docs.appcelerator.com/platform/latest/#!/api/Titanium.Network.HTTPClient explicitly says to use this.responseText, but that is clearly not giving me the results I need. I need that "id" that is being returned from the server.
How do I read the data that is returned from the server after a post API call?
There is just a typo in this.responseText
Have a look at this module: https://github.com/jasonkneen/RESTe
It makes your life easier and gives you a great way to the same syntax for future projects!
If you want to keep your syntax (which is fine too):
have a look at this example: https://github.com/m1ga/titanium-libraries/blob/master/api.js#L49
and the following lines. It shows you how to read the JSON. Basically you have to wait for if (this.readyState === 4) {} inside the onload and then read the this.responseText or JSON.parse() it.
It looks like the apiUrl variable is not defined?
Which could explain a lot or your current frustration.