Prevent getting old updates from Telegram Bot API using a web hook - telegram-bot

I'm writing a Telegram bot and I'm using the official bot API. I've got a webhook server that handles requests and sends a 200 OK response for every request.
Before the server stops, the webhook is detached so Telegram does not send updates anymore. However, whenever I turn the bot on and set the webhook URL again, Telegram starts flooding the webhook server with old updates.
Is there any way I can prevent this without requesting /getUpdates repeatedly until I reach the last update?
Here's a heavily simplified version of how my code looks like:
var http = require('http'),
unirest = require('unirest'),
token = '***';
// Attach the webhook
unirest.post('https://api.telegram.org/bot' + token + '/setWebhook')
.field('url', 'https://example.com/api/update')
.end();
process.on('exit', function() {
// Detach the webhook
unirest.post('https://api.telegram.org/bot' + token + '/setWebhook')
.field('url', '')
.end();
});
// Handle requests
var server = http.createServer(function(req, res) {
res.writeHead(200, { 'Content-Type': 'text/plain' })
res.end('Thanks!');
});
server.listen(80);
Thanks in advance.

The best way is to use update_id which is a specific number that increases on every new request (i.e. update). How to implement it?
First off, let's start with the following anonymous class (using PHP7):
$lastUpdateId = new class()
{
const FILE_PATH = "last-update-id.txt";
private $value = 1;
public function __construct()
{
$this->ensureFileExists();
$this->value = filesize(self::FILE_PATH) == 0
? 0 : (int)(file_get_contents(self::FILE_PATH));
}
public function set(int $lastUpdateId)
{
$this->ensureFileExists();
file_put_contents(self::FILE_PATH, $lastUpdateId);
$this->value = $lastUpdateId;
}
public function get(): int
{
return $this->value;
}
public function isNewRequest(int $updateId): bool
{
return $updateId > $this->value;
}
private function ensureFileExists()
{
if (!file_exists(self::FILE_PATH)) {
touch(self::FILE_PATH);
}
}
};
What the class does is clear: Handling the last update_id via a plain file.
Note: The class is tried to be as short as possible. It does not provide error-checking. Use your custom implementation (e.g. use SplFileObject instead of file_{get|put}_contents() functions) instead.
Now, there are two methods of getting updates: Long Polling xor WebHooks (check Telegram bot API for more details on each methods and all JSON properties). The above code (or similar) should be used in both cases.
Note: Currently, it is impossible to use both methods at the same time.
Long Polling Method (default)
This way, you send HTTPS requests to Telegram bot API, and you'd get updates as response in a JSON-formatted object. So, the following work can be done to get new updates (API, why using offset):
$botToken = "<token>";
$updates = json_decode(file_get_contents("https://api.telegram.org/bot{$botToken}/getUpdates?offset={$lastUpdateId->get()}"), true);
// Split updates from each other in $updates
// It is considered that one sample update is stored in $update
// See the section below
parseUpdate($update);
WebHook Method (preferred)
Requiring support for HTTPS POST method from your server, the best way of getting updates at-the-moment.
Initially, you must enable WebHooks for your bot, using the following request (more details):
https://api.telegram.org/bot<token>/setWebhook?url=<file>
Replace <token> with you bot token, and <file> with the address of your file which is going to accept new requests. Again, it must be HTTPS.
OK, the last step is creating your file at the specified URL:
// The update is sent
$update = $_POST;
// See the section below
parseUpdate($update);
From now, all requests and updates your bot will be directly sent to the file.
Implementation of parseUpdate()
Its implementation is totally up to you. However, to show how to use the class above in the implementation, this is a sample and short implementation for it:
function parseUpdate($update)
{
// Validate $update, first
// Actually, you should have a validation class for it
// Here, we suppose that: $update["update_id"] !== null
if ($lastUpdateId->isNewRequest($update["update_id"])) {
$lastUpdateId->set($update["update_id"]);
// New request, go on
} else {
// Old request (or possible file error)
// You may throw exceptions here
}
}
Enjoy!
Edit: Thanks to #Amir for suggesting editions made this answer more complete and useful.

When you server starts up you can record the timestamp and then use this to compare against incoming message date values. If the date is >= the timestamp when you started...the message is ok to be processed.
I am not sure if there is a way you can tell Telegram you are only interested in new updates, their retry mechanism is a feature so that messages aren't missed...even if your bot is offline.

In the webhook mode, Telegram servers send updates every minute until receives an OK response from the webhook program.
so I recommend these steps:
Check your webhook program that you specified its address as url parameter of the setWebhook method. Call its address in a browser. It does not produce an output to view, but clears that probably there is no error in your program.
Include a command that produces a '200 OK Status' header output in your program to assure that the program sends this header to the Telegram server.

I have the same issue, then I tried to reset the default webhook with
https://api.telegram.org/bot[mybotuniqueID]/setWebhook?url=
after that, i verified the current getUpdates query were the same old updates but I sent new requests through the telegram's bot chat
https://api.telegram.org/bot[mybotuniqueID]/getUpdates
when I set up my webhook again the webhook read the same old updates. Maybe the getUpdates method is not refreshing the JSON content.
NOTE:
in my case, it was working fine until I decided to change /set privacy bot settings from botfather

Related

Can't perform HTTP Post Action from Logic App to Asp.net Core Web API

I've built many Logic Apps. I've also integrated with the Logic App API. For some reason, a Post request to an Asp.net Core Web API won't work. It works in Postman, but I can't get Logic Apps to complete the request.
The request arrives at my Web API. I can step through it during a remote debug session. I'm using the [FromBody] decorator on the API method. All the string values in the object are null.
Logic App Headers
Accept = "application/json"
ContentType = "application/json"
ContentLength = "35"
Host = "****.centralus.logic.azure.com"
API method
[HttpPost]
[Route("CreateSomething")]
public async Task<IActionResult> CreateSomething([FromBody] MyObject object)
{
//Create something great
}
I think it might have something to do with the Headers. I noticed that the Postman request won't succeed unless I check the Host and Content-Length box in the Headers section. According to this article, Logic Apps ignores those Headers.
https://learn.microsoft.com/en-us/azure/connectors/connectors-native-http
I've built the HTTP Post Action using the API as well as configured it manually using the Logic App UI in Azure.
By the way, does anyone know the Expression that will automatically calculate the ContentLength?
UPDATE:
I finally figured this out. I had to do some Ninja coding crap to make this work. I'll post my solution tomorrow.
Does anyone know how to make this work? Thanks in advance!
When you use the Logic App API to programmatically create Logic Apps, you have to specify the Body class for when you do something like an HTTP Post. When the Body JSON displayed in the designer, it contained a single object with the objects properties. My API method could not handle this. The key was to simply post the properties in the JSON Body. To make matters worse, I'm doing two HTTP Posts in this particular Logic App. When I tried to add my object properties to the existing Body class, it caused my other HTTP Post to stop working. To overcome this, I had to create a Body2 class with the objects properties. I then had to use the following line of code to replace body2 with body before adding the JSON to the Logic App API call.
This did not work.
body = new Body()
{
object = new Object()
{
//Properties
}
}
This worked.
body2 = new Body2()
{
Type = 0,
Description = "#{items('For_each_2')?['day']?['description']}",
Locations = item.Locations,
Cold = "#{items('For_each_2')?['temperature']?['cold']?['value']}",
Hot = "#{items('For_each_2')?['temperature']?['hot']?['value']}",
Hide = 0
}
Notice I used Replace on body2.
var options = new JsonSerializerOptions { WriteIndented = true, IgnoreNullValues = true};
string jsonString = ReplaceFirst(JsonSerializer.Serialize(myApp, options), "schema", "$schema").Replace("_else", "else").Replace("_foreach", "foreach").Replace("body2", "body");

Task Module call from Ms Teams in Bot Framework

I am looking to open a task module (Pop up - iframe with audio/video) in my bot that is connected to Teams channel. I am having issues following the sample code provided on the GitHub page.
I have tried to follow the sample and incorporate to my code by did not succeed.
In my bot.cs file I am creating card action of invoke type:
card.Buttons.Add(new CardAction("invoke", TaskModuleUIConstants.YouTube.ButtonTitle, null,null,null,
new Teams.Samples.TaskModule.Web.Models.BotFrameworkCardValue<string>()
{
Data = TaskModuleUIConstants.YouTube.Id
}));
In my BotController.cs that inherits from Controller
[HttpPost]
public async Task PostAsync()
{
// Delegate the processing of the HTTP POST to the adapter.
// The adapter will invoke the bot.
await _adapter.ProcessAsync(Request, Response, _bot);
}
public async Task<HttpResponseMessage> Post([FromBody] Activity activity)
{
if (activity.Type == ActivityTypes.Invoke)
{
return HandleInvokeMessages(activity);
}
return new HttpResponseMessage(HttpStatusCode.Accepted);
}
private HttpResponseMessage HandleInvokeMessages (Activity activity)
{
var activityValue = activity.Value.ToString();
if (activity.Name == "task/fetch")
{
var action = Newtonsoft.Json.JsonConvert.DeserializeObject<Teams.Samples.TaskModule.Web.Models.BotFrameworkCardValue<string>>(activityValue);
Teams.Samples.TaskModule.Web.Models.TaskInfo taskInfo = GetTaskInfo(action.Data);
Teams.Samples.TaskModule.Web.Models.TaskEnvelope taskEnvelope = new Teams.Samples.TaskModule.Web.Models.TaskEnvelope
{
Task = new Teams.Samples.TaskModule.Web.Models.Task()
{
Type = Teams.Samples.TaskModule.Web.Models.TaskType.Continue,
TaskInfo = taskInfo
}
};
return msg;
}
return new HttpResponseMessage(HttpStatusCode.Accepted);
}
There is more code as per the GitHub sample but I won't paste it here. Can someone point me into the correct direction ?
I have got to the stage that it is displaying a pop up window but the content and title comes from manifest file instead of creating actual iframe also no video is rendering. My goal is to render video within my teams using iframe container.
The important part from the sample:
This sample is deployed on Microsoft Azure and you can try it yourself by uploading Task Module CSharp.zip to one of your teams and/or as a personal app. (Sideloading must be enabled for your tenant; see step 6 here.) The app is running on the free Azure tier, so it may take a while to load if you haven't used it recently and it goes back to sleep quickly if it's not being used, but once it's loaded it's pretty snappy.
So,
Your Teams Admin MUST enable sideloading
Your bot MUST be sideloaded into Teams
The easiest way to do this would be download the sample manifest, open it in App Studio, then edit your bot information in. You then need to make sure Domains and permissions > Valid Domains are set for your bot. Also ensure you change the Tabs URLs to your own.
You also need to make sure that in your Tasks, the URLs they call ALL use https and not http. If anywhere in the chain is using http (like if you're using ngrok and http://localhost), it won't work.

JWT authentication with gundb

Can i use JWT authentication with gundb? And if so, would it dramatically slow down my sync speed? I was going to try and implement a test using the tutorial here but wanted to see if there were any 'gotchas' I should be aware of.
The API has changed to use a middleware system. The SEA (Security, Encryption, Authorization) framework will be published to handle stuff like this. However, you can roll your own by doing something like this on the server:
Gun.on('opt', function(ctx){
if(ctx.once){ return }
ctx.on('in', function(msg){
var to = this.to;
// process message.
to.next(msg); // pass to next middleware
});
});
Registering the in listener via the opt hook lets this middleware become 1st in line (before even gun core), that way you can filter all inputs and reject them if necessary (by not calling to.next(msg)).
Likewise to add headers on the client you would want to register an out listener (similarly to how we did for the in) and modify the outgoing message to have msg.headers = {token: data} and then pass it forward to the next middleware layers (which will probably be websocket/transport hooks) by doing to.next(msg) as well. More docs to come on this as it stabilizes.
Old Answer:
A very late answer, sorry this was not addressed sooner:
The default websocket/ajax adapter allows you to update a headers property that gets passed on every networked message:
gun.opt({
headers: { token: JWT },
});
On the server you can then intercept and reject/authorize requests based on the token:
gun.wsp(server, function(req, res, next){
if('get' === req.method){
return next(req, res);
}
if('put' === req.method){
return res({body: {err: "Permission denied!"}});
}
});
The above example rejects all writes and authorizes all reads, but you would replace this logic with your own rules.

How do i use Behat with Mink and WebApiContext?

The project i'm working on has a api behind a login.
I'm using behat with mink to login:
Scenario: Login
Given I am on "/login/"
And I should see "Login"
When I fill in "_username" with "test"
And I fill in "_password" with "test"
And I press "_submit"
Then I should be on "/"
This works..
However, the login session is not stored whenever i want to do the following using the WebApiContext:
Scenario: Getting the list of pages
When I send a GET request to "/api/pages.json"
Then print response
I'm using both scenarios in the same feature. My FeatureContext class looks something like this:
class FeatureContext extends MinkContext
{
public function __construct(array $parameters)
{
$context = new WebApiContext($parameters['base_url']);
$context->getBrowser()->getClient()->setCookieJar(new \Buzz\Util\CookieJar());
$this->useContext('web', $context);
}
}
I added the cookiejar idea from this issue without success.. When i print the response i just see the HTML page from the login screen..
Does anyone have any idea if i'm going at this totally the wrong way or am i somewhat in the right direction?
I am successfully using the same method. I don't think there's a standard way of doing this. As far as you understand the cookie basics you should be able to implement the solution.
In a common scenario, a client sends an authentication request to the server with some credentials, the servers validates it, starts an authenticated session and sends back a cookie with that session id. All following requests contain that id, so the server can recognise the callee. A specific header can be used instead of the cookie, or a database can be used instead of the session, but the principle is the same and you can (relatively) easily simulate it with Mink.
/**
* Start a test session, set the authenticated user and set the client's cookie.
*
* #Given /^I am signed in$/
*/
signIn()
{
session_start();
$_SESSION['user'] = 'jos';
$this->getSession()->getDriver()->setCookie(session_name(), session_id());
session_commit();
}
The above step definition (Behat 3) is the basics of it, you manually create the authenticated session and set to the client it's id. That must be also what the other example illustrates.
PHP's sessions can be problematic when you start doing more complex things and there are a couple of big underwater rocks with this solution. If you want to run assertions from both perspectives (the client and the server) you might often need to have your sessions synced. This can be done by updating the cookie before all Mink steps and reloading the session after.
/**
* #beforeStep
* #param BeforeStepScope $scope
*/
public function synchroniseClientSession(BeforeStepScope $scope)
{
// Setup session id and Xdebug cookies to synchronise / enable both.
$driver = $this->getSession()->getDriver();
// Cookie must be set for a particular domain.
if ($driver instanceof Selenium2Driver && $driver->getCurrentUrl() === 'data:,') {
$driver->visit($this->getMinkParameter('base_url'));
}
// Also enables the debugging support.
$driver->setCookie(session_name(), session_id());
$driver->setCookie('XDEBUG_SESSION', 'PHPSTORM');
}
/**
* #afterStep
* #param AfterStepScope $scope
*/
public function synchroniseServerSession(AfterStepScope $scope)
{
$driver = $this->getSession()->getDriver();
// Only browser kit driver, only initiated requests, only not repeating requests.
if (!$driver instanceof BrowserKitDriver) {
return;
} elseif (($request = $driver->getClient()->getRequest()) === null) {
return;
} elseif ($request === self::$request) {
return;
}
// Your logic for reloading the session.
self::$request = $request;
}
The biggest problem I had was the session reloading. This might be due to my framework of choice, which I doubt. The very first code snippet has session_commit(), which saves and closes the session. In theory in the following step definitions you must be able to session_id(/* session id from the cookie… */); and session_start();, but in practice that didn't work and no session data was actually loaded from the file, though the session did start. To solve this I created a custom session manager with reload() method using session save handler.
Second problem is where you cannot simply close the session without either writing it or destroying it (the support is added in PHP 5.6) on which relies the reloading itself. I reinvented the wheel with a flag for the session manager which tells it whether to write or just to close it.
:)

Grails how to post out to someone else's API

I am writing a Grails app, and I want the controller to hit some other API with a POST and then use the response to generate the page my user sees. I am not able to Google the right terms to find anything about posting to another page and receiving the response with Grails. Links to tutorials or answers like "Thats called..." would me much appreciated.
Seems like you are integrating with some sort of RESTful web service. There is REST client plugin, linked here.
Alternatively, its quite easy to do this without a plugin, linked here.
I highly recommend letting your controller just be a controller. Abstract your interface with this outside service into some class like OtherApiService or some sort of utility. Keep all the code that communicates with this outside service in one place; that way you can mock your integration component and make testing everywhere else easy. If you do this as a service, you have room to expand, say in the case you want to start storing some data from the API in your own app.
Anyway, cutting and posting from the linked documentation (the second link), the following shows how to send a GET to an API and how to set up handlers for success and failures, as well as dealing with request headers and query params -- this should have everything you need.
#Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.5.0-RC2' )
import groovyx.net.http.*
import static groovyx.net.http.ContentType.*
import static groovyx.net.http.Method.*
def http = new HTTPBuilder( 'http://ajax.googleapis.com' )
// perform a GET request, expecting JSON response data
http.request( GET, JSON ) {
uri.path = '/ajax/services/search/web'
uri.query = [ v:'1.0', q: 'Calvin and Hobbes' ]
headers.'User-Agent' = 'Mozilla/5.0 Ubuntu/8.10 Firefox/3.0.4'
// response handler for a success response code:
response.success = { resp, json ->
println resp.statusLine
// parse the JSON response object:
json.responseData.results.each {
println " ${it.titleNoFormatting} : ${it.visibleUrl}"
}
}
// handler for any failure status code:
response.failure = { resp ->
println "Unexpected error: ${resp.statusLine.statusCode} : ${resp.statusLine.reasonPhrase}"
}
}
You might also want to check out this, for some nifty tricks. Is has an example with a POST method.