How to download a `geojson` file with Panda from Google Colab? - pandas

I'm a newbie in python and panda and I use Google Colab.
I have a dataframe I manipulate in many ways. This is OK.
At the end of my manipulations I have a returned geojson formatted data:
for record in json_result:
geojson['features'].append({
'type': 'Feature',
'geometry': {
'type': 'Polygon',
'coordinates': [
[
[ record['tlx'], record['tly'] ],
[ record['blx'], record['bly'] ],
[ record['brx'], record['bry'] ],
[ record['trx'], record['try'] ],
[ record['tlx'], record['tly'] ],
]],
},
'properties': {"surface": record['SURFACE']},
})
The output (geojson) is good; I don't think I have any problem here.
However I want to download this geojson file to my computer.
I've tried numerous ways, without success.
from google.colab import files
......
How to do?
Thanks

Mount your google drive:
from google.colab import drive
drive.mount('/content/drive')
Save your geojson object:
import json
# create json object from dictionary
geojson_str = json.dumps(geojson)
# open file for writing, "w"
f = open("drive/MyDrive/geojson_export.json","w")
# write json object to file
f.write(geojson_str)
# close file
f.close()

Related

Bazel extension that allows to load file from S3 into BUILD

Currently I have a list of dictionaries in a .bzl file:
test_data = [
{ "name": "test", "data": "test_data"}
]
That I load in a BUILD file and perform some magic with list comprehension...
[
foo(name=data["name"], data=data["data"])
for data in test_data
]
I need to be able to pull this file in from S3 and provide the contents of the BUILD file the same way I do with the static .bzl file.

Import data from the api URL (for example from Google Analytics) to clickhouse

Using curl the url to import data from Google's API, for example to Google Analytics, you can use this URL to get data
curl 'https://www.googleapis.com/analytics/v3/data/ga?access_token=[YOUR_TOKEN]&ids=ga%3A[ID_GA_VIEW]&dimensions=ga%3Adate&metrics=ga%3Ausers&start-date=2021-12-01&end-date=2021-12-10'
This return me a JSON
"totalsForAllResults": {
"ga:users": "214377"
},
"rows": [
[
"20211201",
"28818"
],
[
"20211202",
"27421"
],
[
"20211203",
"23758"
],
[
"20211204",
"12453"
],
[
"20211205",
"12514"
],
[
"20211206",
"21857"
],
[
"20211207",
"23770"
],
[
"20211208",
"17458"
],
[
"20211209",
"24278"
],
[
"20211210",
"22050"
]
]
Could I import to clickhouse using data directly from the url of the api?
Desired result, inserting data into table dirctly from API URL
date
users
2021-12-01
28818
2021-12-02
27421
2021-12-03
23758
2021-12-04
12453
....
....

How can i add a module in odoo15

I use mac to study odoo,
And i use this document to build my first module with command
./odoo-bin --addons-path=/Users/xuhongxin/src/custom,addons
And i successfully start the odoo,
But i can not find the estate module;
How can i solve this problem?
the document is :
https://www.odoo.com/documentation/15.0/developer/howtos/rdtraining/03_newapp.html
my manifest.py is :
{
'name': 'estate',
'summary': 'estate',
'description': "estate",
'website': 'https://www.odoo.com/page/crm',
'depends': [
'base_setup',
'sales_team',
'mail',
'calendar',
'resource',
'fetchmail',
'utm',
'web_tour',
'contacts',
'digest',
'phone_validation',
],
'data': [
'security/crm_security.xml',
'security/ir.model.access.csv',
'data/crm_lead_prediction_data.xml',
'data/crm_lost_reason_data.xml',
'data/crm_stage_data.xml',
'data/crm_team_data.xml',
'data/digest_data.xml',
'data/mail_data.xml',
'data/crm_recurring_plan_data.xml',
'wizard/crm_lead_lost_views.xml',
'wizard/crm_lead_to_opportunity_views.xml',
'wizard/crm_lead_to_opportunity_mass_views.xml',
'wizard/crm_merge_opportunities_views.xml',
'views/assets.xml',
'views/calendar_views.xml',
'views/crm_recurring_plan_views.xml',
'views/crm_menu_views.xml',
'views/crm_lost_reason_views.xml',
'views/crm_stage_views.xml',
'views/crm_lead_views.xml',
'views/digest_views.xml',
'views/mail_activity_views.xml',
'views/res_config_settings_views.xml',
'views/res_partner_views.xml',
'views/utm_campaign_views.xml',
'report/crm_activity_report_views.xml',
'report/crm_opportunity_report_views.xml',
'views/crm_team_views.xml',
],
'demo': [
'data/crm_team_demo.xml',
'data/mail_activity_demo.xml',
'data/crm_lead_demo.xml',
],
'css': ['static/src/css/crm.css'],
'installable': True,
'application': True,
'auto_install': False
}
You must pay attention to the path that odoo is having while searching for addons which is where all modules are located. You can check this via running the server in the command line and looking for the path that says is running or taking the info from, mostly is addons directory, make sure your module is running inside that exact path and if not, just add the path to the variable that stores the addons paths.

Using CDSAPI in google colab

I installed a python library called cdsapiin google colab.
To use it I need to locate its config file (which in a general Linux system is $HOME/.cdsapirc) and add my account key to it.
More details can be found here (https://cds.climate.copernicus.eu/api-how-to).
I am having a problem with this step
Copy the code displayed beside, in the file $HOME/.cdsapirc (in your
Unix/Linux environment): url: {api-url} key: {uid}:{api-key}
I tried using !cd /home/ in colab notebook but it doesn't contain this file.
I have also tried !cat /home/.cdsapirc, it gave error:
cat: /home/.cdsapirc: No such file or directory
I achieved this successfully. My code in Colab is as follows:
First, create '.cdsapirc' and write your key in root dir:
url = 'url: https://cds.climate.copernicus.eu/api/v2'
key = 'key: your uid and key'
with open('/root/.cdsapirc', 'w') as f:
f.write('\n'.join([url, key]))
with open('/root/.cdsapirc') as f:
print(f.read())
Then, install cdsapi:
!pip install cdsapi
Run example:
import cdsapi
c = cdsapi.Client()
c.retrieve("reanalysis-era5-pressure-levels",
{
"variable": "temperature",
"pressure_level": "1000",
"product_type": "reanalysis",
"year": "2008",
"month": "01",
"day": "01",
"time": "12:00",
"format": "grib"
}, "/target/dir/download.grib")
The target dir could be your google drive folder.
You can specify your UID, API key and CDS API endpoint directly as arguments into the constructor:
uid = <YOUR UID HERE>
apikey = <YOUR APIKEY>
c = cdsapi.Client(key=f"{uid}:{apikey}", url="https://cds.climate.copernicus.eu/api/v2")

TemplateDoesNotExist at/

here is my site url http://webtrick.heliohost.org/
my template directory settings:
TEMPLATE_DIRS = (
os.path.join(os.path.dirname(__file__) , 'templates').replace('\\','/')
)
view.py
from django.http import HttpResponse
from django.template import Template , Context
from django.shortcuts import render_to_response
def index(request):
return render_to_response('index.html')
url.py
from django.conf.urls.defaults import patterns, include, url
from webtrickster.views import index
urlpatterns = patterns('',
url(r'^$', index)
)
i don't know what's wrong with this code
any help appreciated
Make sure your Django app is in the INSTALLED_APPS in settings.py. That was my problem with it. So if you have the templates folder in your polls app folder, you need to add the 'polls' at the end of the installed apps in the settings file.
Add you application into setting.py end of the INSTALLED_APPS like below:
INSTALLED_APPS = [
...
'django.contrib.staticfiles',
'mysite'
]
and your templates dir should be in mysite like
mysite
-settings.py
-templates/
I wasted a lot of time trying to figure out what was wrong with my 'templates' directory and found out that :
You may have forgotten to add TEMPLATE_DIR in the following segment of settings.py :
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
So it should have 'DIRS': [TEMPLATE_DIR,], instead of 'DIRS': [],
Also see answers for : TemplateDoesNotExist at /
os.path.join(os.path.dirname(__file__) ,'../templates').replace('\\','/')
That worked for me.
First you need to make one folder named 'templates' in your django project folder then, you can add template directory by two ways open your settings file then add any of the following code
1) You can specify your template path directly by
TEMPLATE_DIRS = (
'D:/Django/web/Kindset/templates' #your file path ,this is my template directory path
)
2) Or if you want to use generic path means use
TEMPLATE_DIRS = (
'os.path.join(BASE_DIR, "templates"),'
)
You may forgot to install you app
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'tutorials'
]
Make sure you also add dirs properly in your templates
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR, 'templates')],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
TEMPLATE_DIRS = (
"mysite/templates"
)
TemplateDoesNotExist - file exists, no permissions issue
My templates folder wasn't a python package because it was missing the __init__.py file.
This might have been the reason why Django was not locating my templates.
Many good answers here, but on updating an old project (nightmare=zinnia, django_cms, AUTH_USER_MODEL = 'accounts.CustomUser') that was not maintained for many moons from django 1.6 to django 1.7 as stage one of a very long process, I found nothing worked. Then I noticed that the errors had quotes around the template it was trying to load:
/home/vagrant/venv/crowd88/local/lib/python2.7/site-packages/djangocms_admin_style/templates/'core/includes/ga.html' (File does not exist)
So I looked at the source and found that quotes are stripped this way:
# lib/python2.7/site-packages/cms/utils/plugins.py
...
template = get_template(force_unicode(node.template).strip('"'))
...
Then I noticed that single quotes were being used in the loaders
{% include core/includes/ga.html' %}
So the fix was
{% include "core/includes/ga.html" %}
using a regex process
Move your "templates" folder to the main and not in any sub folder. Or on the same level as other projects folder. This might have been the reason why Django was not locating my templates.
My project worked after doing this.
make sure your dirs directory in settings.py looks like this
'DIRS': [os.path.join(BASE_DIR, 'templates')],
By default, inside the settings.py file, the 'DIRS' array is empty as you can see below.
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
So to solve the issue, just add BASE_DIR / 'templates' to the array to look like below. And you are good to go.
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [BASE_DIR / 'templates'],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
Go to the settings.py file and check the TEMPLATES section. Notice the DIR value, by default it should be []. Add "os.path.join(BASE_DIR, 'templates')" inside this. It should look like this :
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR, 'templates')],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
I encountered the same error. After checking TEMPLATE_DIR and INSTALLED_APPS a thousand times I noticed a small typo in one .html template:
{% extends "my_app/base.html " %}
should have been
{% extends "my_app/base.html" %}
(notice the white space in the first command)
If you have tried some of the solutions above and it not worked try checking whether the template is within the correct folder.
Template outside of the templates/appname folder:
appname/
|
|__templates/
| |
| |__appname/
| |__appname_form.html
|__appname_list.html <------ outside of the appname/ folder (WRONG)
Like this if you have a url that send to this template, Django will through a TemplateDoesNotExist since the template does not follow the path you placed in TEMPLATES.DIRS.
Try fixing like:
Template inside of the templates/appname folder:
appname/
|
|__templates/
| |
| |__appname/
| |__appname_form.html
| |__appname_list.html <------ inside of the appname/ folder (RIGHT)
Therefore, like this you may fix this exception.
if {% extends "blog/base.html" %} make sure base.html is in the blog folder and not in Templates or any other directory. That solved it for me.
This worked for me:
1.Make a folder in your app called static and create the css file in the static folder
2.Make another folder still in your app called templates and create the html file there
On the first line of your html file write this: {% load static %}
Link the css file inside the head tag as follows: href="{% static 'index.css' %}"
Thank me later.
Got the same problem !
I've already followed all solutions provided above but pfff ! I were lost and maybe someone could face it too!
So I solved it by only
return render(request, 'profile.html')
and not
return redirect(request, "profile.html")
Happy debugging !!!
After debugging a lot, i found out that the path which i was giving in views.py to render the template is wrong so it was showing TemplateDoesNotExist at/ error
for Example the mistake i was making is this
Wrong code :-
return render(request, 'show/index.html', {'product_objects': product_objects})
Right code :-
return render(request, 'shop/index.html', {'product_objects': product_objects})
the app name was shop not show.
Hope it will help someone
I think the answer is pretty simple just give the right path
Use this if you have your templates folder in your app
BASE_DIR = Path(__file__).resolve().parent.parent
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'DIRS': [BASE_DIR,"templates"],