Beautifulsoup turning <link> into <link/> - beautifulsoup

Question

I'm trying to download and parse text from some RSS feeds, such as http://rss.sciencedirect.com/publication/science/03043878. Here's a simple example:
import urllib.request
import urllib.parse
import requests
from bs4 import BeautifulSoup
def main():
soup = BeautifulSoup(urllib.request.urlopen('http://rss.sciencedirect.com/publication/science/03043878'),"html.parser").encode("ascii")
print(soup)
if __name__ == '__main__':
main()
In the original html (if you look at the website directly), links are preceded by <link> and followed by </link>. But what beautifulsoup prints out replaces <link> with <link/> and drops </link> completely. Any ideas what I could be doing wrong or is this a bug?
PS Tried changing encoding to utf-8 and it still happens.

Answer 1

You are parsing RSS. RSS is XML. So pass features="xml" into the BeautifulSoup constructor.
import urllib.request
from bs4 import BeautifulSoup
def main():
doc = BeautifulSoup(urllib.request.urlopen('http://rss.sciencedirect.com/publication/science/03043878'), "xml")
# If you want to print it as ascii (as per your original post).
print (doc.prettify('ascii'))
# To write it to an file as ascii (as per your original post).
with open("ascii.txt", "wb") as file:
file.write(doc.prettify('ascii'))
# To write it to an file as utf-8 (as the original RSS).
with open("utf-8.txt", "wb") as file:
file.write(doc.prettify('utf-8'))
# If you want to print the links.
for item in doc.findAll('link'):
print(item)
if __name__ == '__main__':
main()
Outputs in both files and terminal:
... <link>
http://rss.sciencedirect.com/action/redirectFile?&zone=main&currentActivity=feed&usageType=outward&url=http%3A%2F%2Fwww.sciencedirect.com%2Fscience%3F_ob%3DGatewayURL%26_origin%3DIRSSSEARCH%26_method%3DcitationSearch%26_piikey%3DS0304387817300512%26_version%3D1%26md5%3D16ed8e2672e8048590d3c41993306b0f
</link> ...

Answer 2

The parser can't evaluate the links correctly. For this issue you should use xml as your parser instead of html.parser.
soup = BeautifulSoup(urllib.request.urlopen('http://rss.sciencedirect.com/publication/science/03043878'),"xml")
print(len(soup.find_all("link")))
Outputs 52 links.