Send logs directly to Loki without use of agents - api

Question

Is there a way to send logs to Loki directly without having to use one of it's agents?
For example, if I have an API, is it possible to send request/response logs directly to Loki from an API, without the interference of, for example, Promtail?
Thanks in advance!

Answer 1

Loki HTTP API
Loki HTTP API allows pushing messages directly to Grafana Loki server:
POST /loki/api/v1/push
/loki/api/v1/push is the endpoint used to send log entries to Loki.
The default behavior is for the POST body to be a snappy-compressed
protobuf message:
Protobuf definition
Go client library
Alternatively, if the Content-Type header is set to application/json,
a JSON post body can be sent in the following format:
{
"streams": [
{
"stream": {
"label": "value"
},
"values": [
[ "<unix epoch in nanoseconds>", "<log line>" ],
[ "<unix epoch in nanoseconds>", "<log line>" ]
]
}
]
}
You can set Content-Encoding: gzip request header and post gzipped
JSON.
Example:
curl -v -H "Content-Type: application/json" -XPOST -s "http://localhost:3100/loki/api/v1/push" --data-raw \
'{"streams": [{ "stream": { "foo": "bar2" }, "values": [ [ "1570818238000000000", "fizzbuzz" ] ] }]}'
So it is easy to create JSON-formatted string with logs and send it to the Grafana Loki.
Libraries
There are some libraries implementing several Grafana Loki protocols.
There is also (my) zero-dependency library in pure Java 1.8, which implements pushing logs in JSON format to Grafana Loki. Works on Java SE and Android platform:
https://github.com/mjfryc/mjaron-tinyloki-java
Security
Above API doesn't support any access restrictions as written here - when using over public network, consider e.g. configuring Nginx proxy with HTTPS from Certbot and Basic Authentication.

Answer 2

Yes. You can send logs directly from a Java application to loki.
It can be done using the loki4j configuration in your java springboot project. Add these below dependencies to pom.xml
<dependency>
<groupId>com.github.loki4j</groupId>
<artifactId>loki-logback-appender</artifactId>
<version>1.2.0</version>
</dependency>
Run loki either directly or from docker depending on how you have installed loki on your system. I use docker instances of loki and grafana.
Create a logback.xml in your springboot project with the following contents
<property name="HOME_LOG" value="app.log" />
<appender name="FILE-ROLLING"
class="com.github.loki4j.logback.Loki4jAppender">
<http>
<url>http://localhost:3100/loki/api/v1/push</url>
</http>
<format>
<label>
<pattern>app=my-app,host=${HOSTNAME},level=%level</pattern>
</label>
<message>
<pattern>l=%level h=${HOSTNAME} c=%logger{20} t=%thread | %msg %ex
</pattern>
</message>
<sortByTime>true</sortByTime>
</format>
</appender>
<logger name="com.vasanth.loki" level="debug" additivity="false">
<appender-ref ref="FILE-ROLLING" />
</logger>
<root level="error">
<appender-ref ref="FILE-ROLLING" />
</root>
</configuration>
Configure your logger names in the above example and make sure you have given the proper loki URL - You are basically telling the application to write logs into an output stream going directly to the loki URL instead of the traditional way of writing logs to a file through log4j configuration and then using promtail to fetch these logs and load into loki.