TypeError: 'NoneType' object is not iterable Youtube Live Chat - beautifulsoup

Question

I am trying to get names and messages from yt live chat.
Here is the code:
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
url ='https://www.youtube.com/live_chat?is_popout=1&v=EEIk7gwjgIM'
browser = webdriver.Firefox()
browser.get(url)
r = requests.get(url)
soup = BeautifulSoup(r.content,"html.parser")
contents =soup.find("yt-live-chat-text-message-renderer")
for content in contents:
author = content.find("span",{"id":"author-name"}).text
message_content = content.find("span",{"id":"message"}).text
print(author+message_content)
It gives me this error:
TypeError: 'NoneType' object is not iterable
How can i solve that?

Answer 1

Use selenium to extract the html code of the website
Use soup.find_all() instead of soup.find() to retrieve all messages
Here is the full code:
from bs4 import BeautifulSoup
from selenium import webdriver
import time
url ='https://www.youtube.com/live_chat?is_popout=1&v=EEIk7gwjgIM'
browser = webdriver.Chrome()
browser.get(url)
time.sleep(3)
soup = BeautifulSoup(browser.page_source,"html.parser")
contents = soup.find_all("yt-live-chat-text-message-renderer")
for content in contents:
author = content.find("span",{"id":"author-name"}).text.strip()
message_content = content.find("span",{"id":"message"}).text.strip()
print(author ,message_content)
Output:
Plague Doctor #Yellow Rose I don't what you have to say.All people like you do is jump into a conversation and act like you are important
Yellow Rose Haha #Plague Doctor Whatever you say
racheal naidoo hi
NoMade lion Hi yellow rose what is going on in your life
半杯苦茶 Hallo
racheal naidoo hello everyone
Arifa Waheed hi
Plague Doctor Hello 1/2 cup bitter tea
putri zahrani ada orang Indonesia gak?
racheal naidoo hi night bot! 🌃
Yellow Rose Not much #NoMade Lion how about you?
Sath Sitivili hi
racheal naidoo bye everyone 😇🌃
Yellow Rose !iss12
Nightbot The Water Recovery System on ISS reduces crew dependence on “delivered” water by 75% – from about 4 litre a day to 1 litre. Learn more:https://youtu.be/cR_jQ4Is8t0
Arifa Waheed yellow rose what 12?
.
.
.
.
Aniket Srivastava clear sky
C Moore music is so right for the image😊
δεσποινα ζαχου you calm when you look at this view
kaminari fala aeeeeee galera
Kazouchan hello c'est moi je suis quantique
Note: Your output might be different from mine as the time during when we scraped the data would be different.