I have an url: https://api.kite.trade/instruments
And this is my code to fetched data from url and write into excel
import pandas as pd
url = "https://api.kite.trade/instruments"
df = pd.read_json(url)
df.to_excel("file.xlsx")
print("Program executed successfully")
but, when I run this program I'm getting error like this_
AttributeError: partially initialized module 'pandas' has no attribute 'read_json' (most likely due to a circular import)
It's not a json, it's csv. So you need to use read_csv. Can you please try this?
import pandas as pd
url = "https://api.kite.trade/instruments"
df = pd.read_csv(url)
df.to_excel("file.xlsx",index=False)
print("Program excuted successfully")
I added an example how I converted the text to a csv dump on your local drive.
import requests
url = "https://api.kite.trade/instruments"
filename = 'test.csv'
f = open(filename, 'w')
response = requests.get(url)
f.write(response.text)
Related
Code:
import urllib.request
from bs4 import BeautifulSoup
from requests import get
import requests
import dload
import pandas as pd
pd.set_option('display.max_colwidth', None)
week_11_picURL = "https://www.packers.com/photos/game-photos-packers-at-vikings-week-11-2021#9258618e-e793-41ae-8d9a-d3792366dcbb"
response = get(week_11_picURL)
print(response)
html_page = requests.get(week_11_picURL)
soup = BeautifulSoup(html_page.content, 'html.parser')
image = soup.findAll('div', class_="nfl-c-photo-album__picture-wrapper")
data = []
for x in soup.select('.nfl-c-photo-album__picture-wrapper picture source:first-child'):
try:
data.append(x['srcset'].split(',')[0])
except:
data.append(x['data-srcset'].split(',')[0])
data
test_url = "https://static.clubs.nfl.com/image/private/t_new_photo_album/f_auto/packers/f6jcqnmhbzs2dyvepa8z.jpg"
df = pd.DataFrame(data)
replace = df.replace(["/t_lazy", "1x"], "", regex=True)
folder = "f:/nfl pics/packers/week 11 - at vikings"
save = dload.save_multi(url_list=replace, dir=folder, max_threads=1, tsleep=0.05)
replace data:
0
0 https://static.clubs.nfl.com/image/private/t_new_photo_album/f_auto/packers/hjmcucejx2vmfshjkdkj.jpg
1 https://static.clubs.nfl.com/image/private/t_new_photo_album/f_auto/packers/rgsvjp6sxu89ditolacv.jpg
2 https://static.clubs.nfl.com/image/private/t_new_photo_album/f_auto/packers/zsogvqrqgaauqcdgejde.jpg
3 https://static.clubs.nfl.com/image/private/t_new_photo_album/f_auto/packers/jyegqthuab2hsuygirqp.jpg
4 https://static.clubs.nfl.com/image/private/t_new_photo_album/f_auto/packers/kwsq1fvn41f6kzqo4nkl.jpg
etc.
The error I get from using my "save" function is:
Traceback (most recent call last):
File "location", line 174, in save_multi
with open(url_list) as f:
TypeError: expected str, bytes or os.PathLike object, not DataFrame
I'm trying to find away to automatically open all the links from the data in "replace" and save the respected images in the directory labeled "folder". When I try to use my "save" function I get the error above. How do I fix this issue or is there a more efficient way to go about this?
I used the code ask below to load the csv.gz file but I got the error
OSError: Not a gzipped file (b'NU')
How can I solve it?
Code:
import pandas as pd
data = pd.read_csv('climat.202010.csv.gz', compression='gzip')
print(data)
Or:
import gzip
import pandas as pd
filename = 'climat.202010.csv.gz'
with gzip.open(filename, 'rb') as f:
data = pd.read_csv(f)
Try
import gzip
with gzip.open(filename, 'rb') as fio:
df = pd.read_csv(fio)
This works for me:
import gzip
import pandas as pd
with gzip.open(r'C:\Users\MyUser\OneDrive - Company\Data\Wiser\Files\WiserWeeklyReport.csv.gz') as f:
wiser_report = pd.read_csv(f)
wiser_report.head()
If you're still getting an error, it may be the file or the file name. Have you tried taking out the extra period in the file name?
I am trying to upload and show a dataframe by flask and when I want to show it, it says
AttributeError: 'builtin_function_or_method' object has no attribute 'replace'.
I found this code on YT and I don't know if it is correct. Can somebody help me?
from flask import Flask, render_template, request
from werkzeug.utils import secure_filename
import pandas as pd
import csv
def reencode(file):
for line in file:
yield line.decode('windows-1250').encode('utf-8')
#app.route("/data")
def data():
df = pd.read_csv("Sistema_de_Stock.csv", encoding='latin-1')
df = df.drop(df.loc['stock al cargar':].columns, axis=1)
df.to_html('data.html')
with open("data.html", 'r', encoding='latin-1') as file:
file = file.read
**file = file.replace("<table","<table class='rwd-table'")**
with open("data.html","w") as file_write:
file_write.write(html + file)
data = os.startfile("data.html")
return data
file.read is a method, so you should call the method. Furthermore you might want to rename the variable to make it clear that this is not a file handler:
with open('data.html', 'r', encoding='latin-1') as file:
# call the method ↓
file_data = file.read().replace('<table', "<table class='rwd-table'")
with open('data.html', 'w') as file_write:
file_write.write(html + file_data)
data = os.startfile('data.html')
I have a csv file with my data to plot (x, y, and other fields) and want to import it using the new FileInput widget. I don't have sufficient knowledge to manipulate the "base64" strings coming from FileInput to pass it to a ColumnDataSource of dataframe.
from bokeh.io import curdoc
from bokeh.models.widgets import FileInput
def update_cds(attr, old, new):
#A code here to extract column names and data
#from file_input and pass it to a ColumnDataSource or a DataFrame
file_input = FileInput(accept=".csv")
file_input.on_change('value', update_cds)
doc=curdoc()
doc.add_root(file_input)
Thanks for your help!
Here is a working solution: the code will upload the csv file on the server side in a 'data' folder (to be created before). Then it is easy to open the csv and pass it to a ColumnDataSource for instance.
#widget
file_input = FileInput(accept=".csv")
def upload_csv_to_server(attr, old, new):
#decode base64 format (from python base24 website)
base64_message = file_input.value
base64_bytes = base64_message.encode('ascii')
message_bytes = base64.b64decode(base64_bytes)
message = message_bytes.decode('ascii')
#convert string to csv and save it on the server side
message_list = message.splitlines()
with open('data/' + file_input.filename, 'w', newline='') as file:
writer = csv.writer(file)
for i in range(len(message_list)):
writer.writerow(message_list[i].split(','))
file_input.on_change('value', upload_csv_to_server)
If you see a nicer way please let me know. It is easy with the csv structure to do that way but what about any other file format?
Python has a built in base64 standard library module:
import base64
data = base64.b64decode(encoded)
I'm trying to write a dataframe file to a csv using pandas. I'm getting the following error AttributeError: 'list' object has no attribute 'to_csv'. I believe I'm writing the syntax correctly, but could anyone point out where my syntax is incorrect in trying to write a dataframe to a csv?
This is link the link of the file: https://s22.q4cdn.com/351912490/files/doc_financials/quarter_spanish/2018/2018.02.25_Release-4Q18_ingl%C3%A9s.pdf
Thanks for your time!
import tabula
from tabula import read_pdf
import pandas as pd
from pandas import read_json, read_csv
a = read_pdf(r"C:\Users\Emege\Desktop\micro 1 true\earnings_release.pdf",\
multiple_tables= True, pages = 15, output_format = "csv",\
)
print(a)
a.to_csv("a.csv",header = False, index = False, encoding = "utf-8")
enter image description here