I am really new to the Airtable API and for some reason connecting the API this way did not work.
at = airtable.Airtable('Base_Key', 'Airtable_Key')
But I got it working this way -
get_url = ‘https://api.airtable.com/v0/BASE_ID/TABLE_NAME’
get_headers = {
‘Authorization’: ‘Bearer API_KEY’ }
Response = requests.get(get_url, headers=get_headers)
Response_Table = Response.json()
However, this fetches only the first 100 records and am reading about offset and pagination but I am unable to figure how to incorporate it into this code.
Thank you for the time!
After a lot of issues, I found this solution. Posting it for anyone else facing the same problem.
global offset
offset = '0'
result = []
while True :
url = "https://api.airtable.com/v0/BASE_ID/TABLE_NAME"
querystring = {
"view":"Published View",
"api_key":"YOUR_KEY",
"offset": offset}
try :
response= requests.get(url, params=querystring)
response_Table = response.json()
records = list(response_Table['records'])
result.append(records)
#print(records[0]['id'] , len(records))
try :
offset = response_Table['offset']
#print(offset)
except Exception as ex:
#print(ex , offset)
break
except error as e:
print(e)
Related
It's me again, Van.
Hi everyone,
I hope you're doing well!
I am doing a tutorial on a chat application using Codeigniter 4 with Ajax.
Everything worked fine until I applied the following code in the Model below
public function load_chat_data($sender_id,$receiver_id) {
// $where = ['sender_id' => $sender_id, 'receiver_id' => $receiver_id];
$where1 = "sender_id = $sender_id OR sender_id = $receiver_id";
$where2 = "receiver_id = $receiver_id OR receiver_id = $sender_id";
$builder = $this->db->table('chat_messages');
// $builder->where($where);
$builder->where($where1);
$builder->where($where2);
$builder->orderBy('chat_messages_id','ASC');
$results = $builder->get();
$rows = $results->getResultArray();
if($rows > 0)
{
return $rows;
}
else
{
return false;
}
}
The lines that I commented worked well before they were commented but it was not enough data I wanted to get so I tried to get both data of sender and receiver to display on the view by adding more code. However, when I tried $where1 and $where2 for the WHERE clauses, it didn't work. I think it must be the syntax error. Please correct my codes or any ideas on how the codes work with the same meaning supposed.
Thank you so much!!!
I tried as below, but it still didn't work.
$where1 = "sender_id={$sender_id} OR sender_id={$receiver_id}";
$where2 = "receiver_id={$receiver_id} OR receiver_id={$sender_id}";
Also, I tried:
$where1 = "'sender_id'=$sender_id OR 'sender_id'=$receiver_id";
$where2 = "'receiver_id'=$receiver_id OR 'receiver_id'=$sender_id";
I think you are trying to receive the messages of the conversation between two people. Can you try the following code for this?
$builder->groupStart(); // or $builder->orGroupStart();
$builder->where('sender_id', $sender_id);
$builder->where('receiver_id', $receiver_id);
$builder->groupEnd();
$builder->orGroupStart();
$builder->where('sender_id', $receiver_id);
$builder->where('receiver_id', $sender_id);
$builder->groupEnd();
Hello i want asking somethin that i got when i tried to run my streamlit, so i got error like this on my frontend page, i import it from backend page:
i will show my code too
Predict = st.button('Predict Satisfacion Rate')
if Predict:
r = requests.post(URL, json=data)
res = r.json()
if res['code'] == 200:
res2 = (res['result']['description'])
if res2 == 'Not Satisfied':
st.markdown('**The Passenger is not Satisfied**')
col4,col5,col6 = st.columns([1,1,1])
with col5 :
st.image('Happy.jpg')
else:
st.markdown('**The Passenger is Satisfied**')
col7,col8,col9 = st.columns([1,1,1])
with col8 :
st.image('NotHappy.jpg')
else:
st.write('**Error**')
st.write(f"Details : {res['result']['description']}")
so how can i do to solve this error?
thank you.
In your code, the conversion res = r.json() is unnecessary. Unless you really need this as JSON somewhere else, you can test the status code directly from the r object as r.status_code.
After if r.status_code == 200:, you can then convert to JSON if you really need to, as you should be confident that the server returned a valid response.
I'm trying to pull data from the Crunchbase Open Data Map to a Google Spreadsheet. I'm following Ben Collins's script but it no longer works since the upgrade from v3 to v3.1. Anyone had any luck modifying the script for success?
var USER_KEY = 'insert your API key in here';
// function to retrive organizations data
function getCrunchbaseOrgs() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheetByName('Organizations');
var query = sheet.getRange(3,2).getValue();
// URL and params for the Crunchbase API
var url = 'https://api.crunchbase.com/v/3/odm-organizations?query=' + encodeURI(query) + '&user_key=' + USER_KEY;
var json = getCrunchbaseData(url,query);
if (json[0] === "Error:") {
// deal with error with fetch operation
sheet.getRange(5,1,sheet.getLastRow(),2).clearContent();
sheet.getRange(6,1,1,2).setValues([json]);
}
else {
if (json[0] !== 200) {
// deal with error from api
sheet.getRange(5,1,sheet.getLastRow(),2).clearContent();
sheet.getRange(6,1,1,2).setValues([["Error, server returned code:",json[0]]]);
}
else {
// correct data comes back, filter down to match the name of the entity
var data = json[1].data.items.filter(function(item) {
return item.properties.name == query;
})[0].properties;
// parse into array for Google Sheet
var outputData = [
["Name",data.name],
["Homepage",data.homepage_url],
["Type",data.primary_role],
["Short description",data.short_description],
["Country",data.country_code],
["Region",data.region_name],
["City name",data.city_name],
["Blog url",data.blog_url],
["Facebook",data.facebook_url],
["Linkedin",data.linkedin_url],
["Twitter",data.twitter_url],
["Crunchbase URL","https://www.crunchbase.com/" + data.web_path]
];
// clear any old data
sheet.getRange(5,1,sheet.getLastRow(),2).clearContent();
// insert new data
sheet.getRange(6,1,12,2).setValues(outputData);
// add image with formula and format that row
sheet.getRange(5,2).setFormula('=image("' + data.profile_image_url + '",4,50,50)').setHorizontalAlignment("center");
sheet.setRowHeight(5,60);
}
}
}
This code no longer pulls data as expected.
I couldn't confirm about the error messages when you ran the script. So I would like to show about the clear difference point. It seems that the endpoint was changed from https://api.crunchbase.com/v/3/ to https://api.crunchbase.com/v3.1/. So how about this modification?
From :
var url = 'https://api.crunchbase.com/v/3/odm-organizations?query=' + encodeURI(query) + '&user_key=' + USER_KEY;
To :
var url = 'https://api.crunchbase.com/v3.1/odm-organizations?query=' + encodeURI(query) + '&user_key=' + USER_KEY;
Note :
From your script, I couldn't also find query. So if the script doesn't work even when you modified the endpoint, please confirm about it. You can see the detail of API v3 Compared to API v3.1 is here.
References :
API v3 Compared to API v3.1
Using the API
If this was not useful for you, I'm sorry.
I want to reverse engineering the contents generated by scrolling down in the webpage. The problem is in the url https://www.crowdfunder.com/user/following_page/80159?user_id=80159&limit=0&per_page=20&screwrand=933. screwrand doesn't seem to follow any pattern, so the reversing the urls don't work. I'm considering the automatic rendering using Splash. How to use Splash to scroll like browsers? Thanks a lot!
Here are the codes for two request:
request1 = scrapy_splash.SplashRequest(
'https://www.crowdfunder.com/user/following/{}'.format(user_id),
self.parse_follow_relationship,
args={'wait':2},
meta={'user_id':user_id, 'action':'following'},
endpoint='http://192.168.99.100:8050/render.html')
yield request1
request2 = scrapy_splash.SplashRequest(
'https://www.crowdfunder.com/user/following_user/80159?user_id=80159&limit=0&per_page=20&screwrand=76',
self.parse_tmp,
meta={'user_id':user_id, 'action':'following'},
endpoint='http://192.168.99.100:8050/render.html')
yield request2
ajax request shown in browser console
To scroll a page you can write a custom rendering script (see http://splash.readthedocs.io/en/stable/scripting-tutorial.html), something like this:
function main(splash)
local num_scrolls = 10
local scroll_delay = 1.0
local scroll_to = splash:jsfunc("window.scrollTo")
local get_body_height = splash:jsfunc(
"function() {return document.body.scrollHeight;}"
)
assert(splash:go(splash.args.url))
splash:wait(splash.args.wait)
for _ = 1, num_scrolls do
scroll_to(0, get_body_height())
splash:wait(scroll_delay)
end
return splash:html()
end
To render this script use 'execute' endpoint instead of render.html endpoint:
script = """<Lua script> """
scrapy_splash.SplashRequest(url, self.parse,
endpoint='execute',
args={'wait':2, 'lua_source': script}, ...)
Thanks Mikhail, I tried your scroll script, and it worked, but I also notice that your script scroll too much one time, some js have no time too render and is skipped, so I do some little change as follow:
function main(splash)
local num_scrolls = 10
local scroll_delay = 1
local scroll_to = splash:jsfunc("window.scrollTo")
local get_body_height = splash:jsfunc(
"function() {return document.body.scrollHeight;}"
)
assert(splash:go(splash.args.url))
splash:wait(splash.args.wait)
for _ = 1, num_scrolls do
local height = get_body_height()
for i = 1, 10 do
scroll_to(0, height * i/10)
splash:wait(scroll_delay/10)
end
end
return splash:html()
end
I do not think that setting the number of scrolls hard coded is a good idea for infinite scroll pages, so I modified the above-mentioned code like this:
function main(splash, args)
current_scroll = 0
scroll_to = splash:jsfunc("window.scrollTo")
get_body_height = splash:jsfunc(
"function() {return document.body.scrollHeight;}"
)
assert(splash:go(splash.args.url))
splash:wait(3)
height = get_body_height()
while current_scroll < height do
scroll_to(0, get_body_height())
splash:wait(5)
current_scroll = height
height = get_body_height()
end
splash:set_viewport_full()
return splash:html()
end
I need a way to search via the eventbrite api past events, by organizer, that are private, but I also need to be able to limit the date range. I have not found a viable solution for this search. I assume the organizer_list_events api would be the preferred method, but the request paramaters don't seem to allow for the date range, and I am getting FAR too many returns.
I'm having some similar issues I posted a question to get a response about parsing the timezone, here's the code I'm using to get the dates though and exclude any events before today (unfortunately like you said I'm still getting everything sent to me and paring things out client side)
Note this is an AngularJS control but the code is just using the EventBrite javascript API.
function EventCtrl($http, $scope)
{
$scope.events=[];
$scope.noEventsDisplay = "Loading events...";
Eventbrite({'app_key': "EVC36F6EQZZ4M5DL6S"}, function(eb){
// define a few parameters to pass to the API
// Options are listed here: http://developer.eventbrite.com/doc/organizers/organizer_list_events/
//3877641809
var options = {
'id' : "3588304527",
};
// provide a callback to display the response data:
eb.organizer_list_events( options, function( response ){
validEvents = [];
var now = new Date().getTime();
for(var i = 0; i<response.events.length; i++)
{
var sd = response.events[i].event.start_date;
var ed = response.events[i].event.end_date;
var parsedSD = sd.split(/[:-\s]/);
var parsedED = ed.split(/[:-\s]/);
var startDate = new Date(parsedSD[0], parsedSD[1]-1, parsedSD[2], parsedSD[3], parsedSD[4], parsedSD[5]);
var endDate = new Date(parsedED[0], parsedED[1]-1, parsedED[2], parsedED[3], parsedED[4], parsedED[5]);
if(endDate.getTime()<now)
continue;
response.events[i].event.formattedDate = date.toDateString();
validEvents.push(response.events[i])
}
if(validEvents.length == 0)
{
$scope.$apply(function(scope){scope.noEventsDisplay = "No upcoming events to display, please check back soon.";});
}
else
{
$scope.$apply(function(scope){scope.noEventsDisplay = "";});
}
$scope.$apply(function(scope){scope.events = validEvents;});
//$('.event_list').html(eb.utils.eventList( response, eb.utils.eventListRow ));
});
});
}