I have a problem about YouTube API with ESP8266 - api

I try to make youtube subscribe counter but it a problem with youtube api library here the error message
Arduino: 1.8.12 (Windows 10), Board: "NodeMCU 1.0 (ESP-12E Module), 80 MHz, Flash, Legacy (new can return nullptr), All SSL ciphers (most compatible), 4MB (FS:2MB OTA:~1019KB), 2, v2 Lower Memory, Disabled, None, Only Sketch, 115200"
The sketch name had to be modified.
Sketch names must start with a letter or number, followed by letters,
numbers, dashes, dots and underscores. Maximum length is 63 characters.
C:\Users\Um Sythat\Documents\Arduino\libraries\arduino-youtube-api-master\src\YoutubeApi.cpp:95:11: error: DynamicJsonBuffer is a class from ArduinoJson 5. Please see arduinojson.org/upgrade to learn how to upgrade your program to ArduinoJson version 6
DynamicJsonBuffer jsonBuffer;
^
C:\Users\Um Sythat\Documents\Arduino\libraries\arduino-youtube-api-master\src\YoutubeApi.cpp: In member function 'bool YoutubeApi::getChannelStatistics(String)':
C:\Users\Um Sythat\Documents\Arduino\libraries\arduino-youtube-api-master\src\YoutubeApi.cpp:95:20: error: 'jsonBuffer' was not declared in this scope
DynamicJsonBuffer jsonBuffer;
^
C:\Users\Um Sythat\Documents\Arduino\libraries\arduino-youtube-api-master\src\YoutubeApi.cpp:97:10: error: 'ArduinoJson::JsonObject' has no member named 'success'
if(root.success()) {
^
exit status 1
Error compiling for board NodeMCU 1.0 (ESP-12E Module).
Invalid library found in C:\Program Files (x86)\Arduino\libraries\libraries: no headers files (.h) found in C:\Program Files (x86)\Arduino\libraries\libraries
Invalid library found in C:\Program Files (x86)\Arduino\libraries\youtube_control_arduino: no headers files (.h) found in C:\Program Files (x86)\Arduino\libraries\youtube_control_arduino
Invalid library found in C:\Program Files (x86)\Arduino\libraries\libraries: no headers files (.h) found in C:\Program Files (x86)\Arduino\libraries\libraries
Invalid library found in C:\Program Files (x86)\Arduino\libraries\youtube_control_arduino: no headers files (.h) found in C:\Program Files (x86)\Arduino\libraries\youtube_control_arduino
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.
I already download youtube api library and arduino json library and import it to arduino ide I always get error from it i dont know why it gone like this someone who know please help me. I like to hear from you.
And here my code :
/*******************************************************************
* Read YouTube Channel statistics from the YouTube API *
* *
* By Brian Lough *
* https://www.youtube.com/channel/UCezJOfu7OtqGzd5xrP3q6WA *
*******************************************************************/
#include <YoutubeApi.h>
#include <ESP8266WiFi.h>
#include <WiFiClientSecure.h>
#include <ArduinoJson.h> // This Sketch doesn't technically need this, but the library does so it must be installed.
//------- Replace the following! ------
char ssid[] = "xxx"; // your network SSID (name)
char password[] = "yyyy"; // your network key
#define API_KEY "zzzz" // your google apps API Token
#define CHANNEL_ID "UCezJOfu7OtqGzd5xrP3q6WA" // makes up the url of channel
WiFiClientSecure client;
YoutubeApi api(API_KEY, client);
unsigned long api_mtbs = 60000; //mean time between api requests
unsigned long api_lasttime; //last time api request has been done
long subs = 0;
void setup() {
Serial.begin(115200);
// Set WiFi to station mode and disconnect from an AP if it was Previously
// connected
WiFi.mode(WIFI_STA);
WiFi.disconnect();
delay(100);
// Attempt to connect to Wifi network:
Serial.print("Connecting Wifi: ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
Serial.print(".");
delay(500);
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
IPAddress ip = WiFi.localIP();
Serial.println(ip);
}
void loop() {
if (millis() - api_lasttime > api_mtbs) {
if(api.getChannelStatistics(CHANNEL_ID))
{
Serial.println("---------Stats---------");
Serial.print("Subscriber Count: ");
Serial.println(api.channelStats.subscriberCount);
Serial.print("View Count: ");
Serial.println(api.channelStats.viewCount);
Serial.print("Comment Count: ");
Serial.println(api.channelStats.commentCount);
Serial.print("Video Count: ");
Serial.println(api.channelStats.videoCount);
// Probably not needed :)
//Serial.print("hiddenSubscriberCount: ");
//Serial.println(api.channelStats.hiddenSubscriberCount);
Serial.println("------------------------");
}
api_lasttime = millis();
}
}

I would ditch the libraray - it uses
#include <ArduinoJson.h>
and the error tells us whats wrong
error: DynamicJsonBuffer is a class from ArduinoJson 5 <=== you have probably version 6.x.x installed
which is a memory hog. Often one single function is used by libraries and the rest is useless. To solve your problem, you have to
downgrade ArduinoJson.h to version 5.13.5
The other reason I do not like it, breaking changes in nearly all major releases (real big breaking). So another option you have (if skilled enough) replace the ArduinoJson functions with
a lighter JSON library
or replace it with self written JSON functions - often a simple buffer and some char handling does the trick
Read the issues and PRs on github to inform yourself about missing updates and other problems. Development stopped March 2018 since then no adaption to the (breaking) changes in the youtube API

Related

Boost asio crashes

I have a program using cpprestsdk for http querying and websocketpp for subscribing a data stream. The program will crash immediately(it says Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)). But if I comment either of the http querying or subcribing data stream, the program won't crash.
#include <websocketpp/config/asio_client.hpp>
#include <websocketpp/client.hpp>
#include "json.hpp"
#include <iostream>
#include <ctime>
#include <iostream>
#include <cpprest/http_client.h>
#include <cpprest/filestream.h>
#include <vector>
#include <string>
using std::string;
using namespace web;
using std::cout, std::endl;
using std::vector;
using websocketpp::lib::placeholders::_1;
using websocketpp::lib::placeholders::_2;
using websocketpp::lib::bind;
typedef websocketpp::client<websocketpp::config::asio_tls_client> client;
typedef websocketpp::config::asio_client::message_type::ptr message_ptr;
void on_stream_data(websocketpp::connection_hdl hdl, message_ptr msg) {
}
class OrderBook {
public:
void initialize() {
web::http::client::http_client_config cfg;
std::string uri = string("https://fapi.binance.com/fapi/v1/depth?symbol=btcusdt&limit=1000");
web::http::client::http_client client(U(uri), cfg);
web::http::http_request request(web::http::methods::GET);
request.headers().add("Content-Type", "application/x-www-form-urlencoded");
web::http::http_response response = client.request(request).get();
}
int start_stream() {
client c;
std::string uri = string("wss://fstream.binance.com/ws/btcusdt#depth#100ms");
try {
c.set_access_channels(websocketpp::log::alevel::all);
c.clear_access_channels(websocketpp::log::alevel::frame_payload);
c.init_asio();
c.set_message_handler(bind(on_stream_data, ::_1, ::_2));
websocketpp::lib::error_code ec;
client::connection_ptr con = c.get_connection(uri, ec);
if (ec) {
std::cout << "could not create connection because: " << ec.message() << std::endl;
return 0;
}
c.connect(con);
c.run();
} catch (websocketpp::exception const &e) {
std::cout << e.what() << std::endl;
}
}
};
int main(int argc, char *argv[]) {
OrderBook ob;
ob.initialize(); // comment either of these two lines, the program won't crash, otherwise the program will crash once start
std::this_thread::sleep_for(std::chrono::milliseconds(10000000));
ob.start_stream(); // comment either of these two lines, the program won't crash, otherwise the program will crash once start
}
When I run this program in Clion debug mode, Clion show that the crash comes from function in /opt/homebrew/Cellar/boost/1.76.0/include/boost/asio/ssl/detail/impl/engine.ipp
int engine::do_connect(void*, std::size_t)
{
return ::SSL_connect(ssl_);
}
It says Exception: EXC_BAD_ACCESS (code=1, address=0xf000000000)
What's wrong with it? is it because I run two pieces of code using boost::asio, and something shouldn't be initialized twice?
I can compile this and run it fine.
My best bet is that you might be mixing versions, particularly boost versions. A common mode of failure is caused when ODR violations lead to Undefined Behaviour.
Note that these header-only libraries depend on a number of boost libraries that are not header-only (e.g. Boost System, Thread and/or Chrono). You need to compile against the same version as the libraries you link.
If you use distribution packaged versions of any library (cpprestsdk, websocketpp or whatever json library that is you're using) then you'd be safest also using the distribution packaged version of Boost.
I'd personally simplify the situation by just using Boost (Beast for HTTP/websocket, Json for, you guessed it).
Running it all on a test Ubuntu 18.04 the OS Boost 1.65 version, the start_stream sequence triggers this informative error:
[2022-05-22 13:42:11] [fatal] Required tls_init handler not present.
could not create connection because: Connection creation attempt failed
While being UBSAN/ASAN clean. Perhaps that error helps you, once you figure out the configuration problems that made your program crash.

ESP32-CAM how to publish large binary payload to AWS IOT ssl mqtt topic, tested many libs without success :-(

I'm currently working on an ESP32CAM project to publish on AWS IOT topic some captures from the camera in high resolution (UXGA).
I've managed to publish some short json payloads with attributes to different AWS IOT certificate protected topics but I'm facing an annoying issue to do the same with large payload as the capture binary file.
I've browsed many sites, forums, tested differents libs like MQTT, PubSubClient, AsyncMQTTClient... but I've not found a true working solution for large payload around 100KB size.
For example with the PubSubClient lib, I try to fragment my binary payload with the BeginPublish, write, endPublish scheme as below :
bool publishBinary(const uint8_t *buffer,size_t len, const char *topicPubName)
{
Serial.print("publishing binary ["+(String)len+"] ...");
if (len == 0) {
// Empty file
Serial.println("Error : binary payload is empty!");
return(false);
}
if (!client.beginPublish(topicPubName,len,false)) {
Serial.println("MQTT beginPublish failed.");
return(false);
}
size_t max_transfer_size=80;
size_t n=0;
size_t size_send;
size_t offset=0;
while ((len-offset)>0) {
n=(len-offset);
if (n > max_transfer_size)
n=max_transfer_size;
size_send=client.write((const uint8_t *)(buffer+offset),n);
Serial.printf("%d/%d : %.02f %%\n",offset,len,(double)((100*offset)/len));
//Serial.println("n: "+(String)n+" - send: "+(String)size_send);
if(size_send != n) {
// error handling. this is triggered on write fail.
Serial.println("Error during publishing..."+(String)size_send+" instead of "+(String)n);
client.endPublish();
return(false);
} else {
offset+=size_send;
}
}
client.endPublish();
Serial.println("ok");
return(true);
}
client is defined as PubSubClient client(net) where net is WiFiClientSecure object with validated CA_cert, cert and private key.
The MQTT connection is working well but when I try to publish the large binary payload, the function fragments buffer into chunks till the end but there is quite always an error like UNKNOWN ERROR CODE (0050) or when it succeeds to publish, only a small part of payload is published on the destination. In this case, my jpeg file is truncated on my S3 bucket where the payload lands.
I have to say that sometimes, I managed to publish a 65K payload but like a stroke of luck... :-)
I've looked for some examples on the web but very often it is for small payload. As mentioned in a post, I've tested the Publish_P(...) from PubSubClient... but same result, it aborts during transfer.
I begin to ask myself if it really possible by mqtt topic or do I have to create an API gateway with a lambda to handle such large payload. Tell me I'm wrong :-)
If you know a good solution for a true working large payload publishing, I would be delighted to discuss with you :-)
Thanks !
#include <PubSubClient.h>
void setup() {
...
boolean res = mqttClient.setBufferSize(50*1024); // ok for 640*480
if (res) Serial.println("Buffer resized."); else Serial.println("Buffer resizing failed");
...
}
I'm working with 50kB buffer and it works well, haven't tried beyond that bu it should work with 100kB as well.
After you've resized the buffer publish as you normally would.
BTW, setBufferSize function was only recently added IIRC.
Hey I had problems with the PubSubClient library and large files too. In the end I figured out, that I have to update the PubSubClient.h as follows:
//128000 = 128 kB is the maximum size for AWS I think..
#define MQTT_MAX_PACKET_SIZE 100000
// It takes a long time to transmit the large files
// maybe even more than 200 seconds...
#define MQTT_KEEPALIVE 200
#define MQTT_SOCKET_TIMEOUT 200
I had the same problem, and I found that PubSubClient's bufferSize is defined as uint16_t.
https://github.com/knolleary/pubsubclient/blob/v2.8/src/PubSubClient.h#L92
So, we can't extend the buffer size over 64 kB, and can't publish a large payload.
Michael's comment might help you.

Traffic modelling on ns-3 network Simulator

I am trying to simulate traffic of voice, video, and data in order to check the differences between EDCF and traditional CSMA/CA.
I am using an on off helper based on the example of http://www.scielo.org.co/pdf/dyna/v84n202/0012-7353-dyna-84-202-00055.pdf
for the video but on setAttribute(“OffTime”, StringValue(“ns3::LogNormalRandomVariable[Mu=0.4026,Sigma=0.0352]”));
I get error 'Attribute value "0.4026,Sigma=0.0352" is not properly formatted'
Any ideas for the error or suggestions on how to model the three types of traffic?
Thanks for your time
I couldn't find the version of ns3 in your provided reference. Also, the reference paper was published in September 2017. I think, they have used ns3.26 probably.
So, I've tried with ns3.29 and couldn't reproduce the above error. I've got somewhat different error. But I could fixed it by putting a space before '['.
The second solution I found by using Mu and Sigma separately.
onoff.SetAttribute ("OffTime", StringValue("ns3::LogNormalRandomVariable[Mu=0.4026]"));
onoff.SetAttribute ("OffTime", StringValue("ns3::LogNormalRandomVariable[Sigma=0.0352]"));
a4arshad's answer's compile and run, but they are not correct. The reason a4arshad's solution does not work is that the second SetAttribute call sets the OffTime attribute as a LogNormalRandomVariable with Sigma=0.0352. The second call does not add to the existing LogNormalRandomVariable with Mu=0.4026. The second call overwrites the first one.
a4arshad also suggested putting a space between LogNormalRandomVariable and [, but that doesn't work either; this will just cause ns-3 to ignore all parameters.
The correct way to set the attribute as the linked paper is to delimit parameters using a pipe, |. You should do something like
onoff.SetAttribute("OffTime", StringValue("ns3::LogNormalRandomVariable[Mu=0.4026|Sigma=0.0352]"));
I don't know when ns-3 switched to using a pipe to instead of a comma, but I am using the most recent release version, ns-3.30.1.
You can use the following program to check whether setting an attribute worked. The last lines are the relevant ones; the rest is boilerplate to set up the simulation. In the last lines, we extract the Attribute from the Application, and print the Mu and Sigma values. Try commenting/uncommenting the correct/incorrect methods and look at the different outputs. According to the documentation of LogNormalRandomVariable, the default values of Mu and Sigma are 0 and 1, respectively.
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
// setting a LogNormalRandomVariable with multiple parameters as the OffTime
// of an OnOffApplication
// https://stackoverflow.com/questions/60791009
// Author: Sagar
#include "ns3/core-module.h"
#include "ns3/internet-module.h"
#include "ns3/point-to-point-module.h"
#include "ns3/ipv4-global-routing-helper.h"
#include "ns3/applications-module.h"
using namespace ns3;
NS_LOG_COMPONENT_DEFINE("RandomVariableAttirbuteTest");
int
main(int argc, char *argv[]) {
NodeContainer nodes;
nodes.Create(2);
PointToPointHelper p2pLink;
NetDeviceContainer devices = p2pLink.Install(nodes);
InternetStackHelper stack;
stack.InstallAll();
// assign IP addresses to NetDevices
Ipv4AddressHelper address ("10.1.1.0", "255.255.255.0");
Ipv4InterfaceContainer ipInterfaces = address.Assign(devices);
Ipv4GlobalRoutingHelper::PopulateRoutingTables();
// configure and install server app
int serverPort = 8080;
UdpServerHelper serverApp (serverPort);
serverApp.Install(nodes.Get(0));
Address serverAddress = InetSocketAddress(ipInterfaces.GetAddress(0), serverPort);
// configure and install client apps
OnOffHelper onoff ("ns3::UdpSocketFactory", serverAddress);
// incorrect (two separate SetAttribute calls)
// onoff.SetAttribute("OffTime", StringValue("ns3::LogNormalRandomVariable[Mu=0.4026]"));
// onoff.SetAttribute("OffTime", StringValue("ns3::LogNormalRandomVariable[Sigma=0.0352]"));
// also incorrect (adding a space)
// onoff.SetAttribute("OffTime", StringValue("ns3::LogNormalRandomVariable [Mu=0.4026,Sigma=0.0352]"));
// correct
onoff.SetAttribute("OffTime", StringValue("ns3::LogNormalRandomVariable[Mu=0.4026|Sigma=0.0352]"));
onoff.Install(nodes.Get(1));
// print Attirbutes to check if they were correctly set
Ptr<Application> app = nodes.Get(1)->GetApplication(0);
PointerValue pv;
app->GetAttribute("OffTime", pv);
Ptr<LogNormalRandomVariable> rv = pv.Get<LogNormalRandomVariable>();
NS_LOG_UNCOND("Mu = " << rv->GetMu());
NS_LOG_UNCOND("Sigma = " << rv->GetSigma());
return 0;
}

How to find the entry point(or base address) of a process - take care of ASLR

Because of ASLR(Address space layout randomization, since Windows Vista), the base address of an exe is random, so it can't be found in PE file anymore.
In Visual C++ now the /DYNAMICBASE option is default enabled, so the base address
of an exe is random - everytime the loader loads it, it happens.
After did some research on google, I am trying to use this pattern,
But it doesn't work.
Please have a look at this simple code sample:
#include <iostream>
#include <vector>
#include <stdio.h>
#include <windows.h>
#include <psapi.h>
int main()
{
STARTUPINFOA startupInfo = {0};
startupInfo.cb = sizeof(startupInfo);
PROCESS_INFORMATION processInformation = {0};
if (CreateProcessA("UseCase01.exe", NULL, NULL, NULL, FALSE, CREATE_SUSPENDED, NULL, NULL, &startupInfo, &processInformation))
{
std::vector<HMODULE> buf(128);
DWORD needed = 0;
for (;;) {
if (EnumProcessModulesEx(processInformation.hProcess, &buf[0], DWORD(buf.size()*sizeof(HMODULE)), &needed, LIST_MODULES_ALL) == FALSE) {
DWORD ec = GetLastError();
std::cout << ec << std::endl;
break;
}
else if (needed <= buf.size() * sizeof(HMODULE)) {
break;
}
else {
const size_t oldSize = buf.size();
buf.resize(oldSize * 2);
}
}
ResumeThread(processInformation.hThread);
}
}
My OS is Windows 7 64bit pro, my compiler is VS2013, this is a 32bit console program, and the UseCase01.exe is also a 32bit console program too.
EnumProcessModulesEx always fails, the error code returned by GetLastError() is 299, MSDN says what about this error code: ERROR_PARTIAL_COPY, "Only part of a ReadProcessMemory or WriteProcessMemory request was completed."
About this error code, on the EnumProcessModules's page of MSDN, "If this function is called from a 32-bit application running on WOW64, it can only enumerate the modules of a 32-bit process. If the process is a 64-bit process, this function fails and the last error code is ERROR_PARTIAL_COPY (299)."
But I am sure my program is 32bit, And, I tested on 64bit program, it fails with error 299 too, so it doesn't make sence.
"The handle returned by the CreateProcess function has PROCESS_ALL_ACCESS access to the process object." - from MSDN, so it can't be a access rights problem ?
Then I try to use CreateToolhelp32Snapshot, it fails with error code 299 too, both 32bit and 64bit.
I just can't figure it out.
My goal is find the entry point of the sub-process in a safe way, whatever it's 32bit or 64bit process.
I found this is the "deepest" answer about this question: http://winprogger.com/getmodulefilenameex-enumprocessmodulesex-failures-in-wow64/
Unfortunately, 64bit program will fails too, not only for Wow64, so it doesn't make sence.
If this is infeasible, what is the good way (find base address or entry point of a suspended sub-process)?
You are creating the process suspended. While the key kernel data structures will be created, no modules will be loaded (that would involve executing code in module entry points (dllmain)).
Thus the error makes sense: the data structures to track modules loaded will be empty, and quite possibly not allocated at all.
Put some wait it will help you it looks currently resource is not available.
On all Windows operating systems (32/64bit):
DWORD ImageBaseAddress = ((LPDWORD)PEB)[2]

OpenNI 1.5::Could not run code from documentation

I am trying to run a sample code from the OpenNI 1.5 documentation.I have imported the library required XnCppWrapper.h so that I can use C++.The code has only one error on a particular variable "bshouldrun".I know that it should be declared as something but since I am new at this and the documentation does not contain anything above the main, I dont know what to declare it as..Please help!!
And thanks in advance.
#include <XnOpenNI.h>
#include <XnCppWrapper.h>
#include <stdio.h>
int main()
{
XnStatus nRetVal = XN_STATUS_OK;
xn::Context context;
// Initialize context object
nRetVal = context.Init();
// TODO: check error code
// Create a DepthGenerator node
xn::DepthGenerator depth;
nRetVal = depth.Create(context);
// TODO: check error code
// Make it start generating data
nRetVal = context.StartGeneratingAll();
// TODO: check error code
// Main loop
while (bShouldRun) //<-----------------------------**ERROR;bShouldRun Undefined**
{
// Wait for new data to be available
nRetVal = context.WaitOneUpdateAll(depth);
if (nRetVal != XN_STATUS_OK)
{
printf("Failed updating data: %s\n", xnGetStatusString(nRetVal));
continue;
}
// Take current depth map
const XnDepthPixel* pDepthMap = depth.GetDepthMap();
// TODO: process depth map
}
// Clean-up
context.Shutdown();
}
Here's what I did to run a sample from Visual Studio 2010 Express on Windows (8):
Opened the NiSimpleViewer.vcxproj VS2010 project from C:\Program Files (x86)\OpenNI\Samples\NiSimpleViewer
Edited OpenNI.rc to comment out #include "afxres.h" on line 10(might be missing this because I'm using Express version, not sure. Your machine might compile this fine/not complain about the missing header file)
Enabled Tools > Options > Debugging > Symbols > Microsoft Symbol Servers (to get past missing pdb files issue)
Optionally edit the SAMPLE_XML_PATH to "SamplesConfig.xml" rather than the default "../../../Data/SamplesConfig.xml", otherwise you need to run the sample executable from ..\Bin\Debug\NiSimpleViewer.exe by navigating to there rather than using the Ctrl+F5. A;so copy the SamplesConfig.xml file into your sample folder as you can see bellow
Here are a few images to illustrate some of the above steps:
You can also compile the NiHandTracker sample, which sounds closer to what you need.
So this explains the setup for OpenNI 1.5 which is what your question is about.
I've noticed your OpenNI 2 lib issue in the comments. It should be a matter of linking against SimpleHandTracker.lib which you can do via Project Properties (right-click project->select Properties) > Linker > Input > Additional Dependencies > Edit.
I don't have OpenNI2 setup on this machine, but assuming SimpleHandTracker.lib would be in OpenNI_INSTALL_FOLDER\Lib. Try a file search in case I might be wrong.