How do i handle this dropdown using selenium with no select class? - selenium

Question

I am trying to access this website and after that, I want to click on the Status dropdown and select Active from the dropdown. I think there are no select tags to be used hence the traditional select method is not working. Would be really helpful if anyone can have a look at it!
"""
PATH = "/usr/local/bin/chromedriver"
driver = webdriver.Chrome(PATH)
driver.get("https://www.silveroaksp.com/portfolio")
driver.find_element_by_xpath("//div[#class='btn-group show']").click()
time.sleep(2)
driver.close()
"""

Answer 1

*To capture all the Webelements in dropdown we use driver.findElements for multiple elements(this return element in the list).
*You can use to method for this. click on Feild and then click on active Web element, Second is you put it in a for each loop and in IFELSE condition put click
1.
PATH = "/usr/local/bin/chromedriver"
driver = webdriver.Chrome(PATH)
driver.get("https://www.silveroaksp.com/portfolio")
driver.find_element_by_xpath("//select[#id='StatusMultiselect']/following-
sibling::div").click()
time.sleep(2)
driver.find_element_by_xpath("//select[#id='StatusMultiselect']/following-
sibling::div/ul/li[2]/a/label/input").click()
time.sleep(2)
driver.close()
*This'll work I guess. IF doesn't let me know I'll provide a second option

Answer 2

This problem can be solved with Requests & Pandas:
import requests
from bs4 import BeautifulSoup
import pandas as pd
final_list = []
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:23.0) Gecko/20131011 Firefox/23.0',
'Host': 'www.silveroaksp.com',
'Accept': 'text/html, */*; q=0.01',
'Accept-Encoding': 'gzip, deflate, br',
'Referer': 'https://www.silveroaksp.com/portfolio',
'Content-Type': 'application/json; charset=utf-8',
'X-Requested-With': 'XMLHttpRequest'
}
data = '{"prmStatusIds":"PortStatus1","prmIndustryIds":"","prmFundIds":"","AllStatusClickChecker":"0","AllIndustryClickChecker":"1","AllFundClickChecker":"1"}'
with requests.Session() as s:
s.get('https://www.silveroaksp.com/portfolio')
r = s.post('https://www.silveroaksp.com/CommonItem/FilterPort', data=data, headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')
actives = soup.select('div.col-lg-4')
for active in actives:
business = active.select_one('img')['alt']
link = active.get('id')
r = s.post('https://www.silveroaksp.com/CommonItem/PortfolioInfo', data = '{"PortUrl":"' + link + '","Status":"all"}', headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')
try:
headq = soup.find("label", string="Headquarters").find_next_sibling('span').text.strip()
except Exception as e:
headq = 'unkown'
try:
industry = soup.find("label", string="Industry").find_next_sibling('span').text.strip()
except Exception as e:
industry = 'unkown'
try:
aquired = soup.find("label", string="Acquired").find_next_sibling('span').text.strip()
except Exception as e:
aquired = 'unkown'
try:
website = soup.find("label", string="Website").find_next_sibling('span').text.strip()
except Exception as e:
website = 'unkown'
try:
title = soup.select_one('div.about-author').select_one('h4').text.strip()
except Exception as e:
title = 'unkown'
try:
description = ' '.join([x.text.strip() for x in soup.select_one('div.about-author').select('p')])
except Exception as e:
description = 'unkown'
try:
headline = soup.select_one('div.about-author').select_one('strong').text.strip()
except Exception as e:
headline = 'unkown'
final_list.append([headq, industry, aquired, website, title, description, headline])
final_df = pd.DataFrame(final_list, columns = ['headq', 'industry', 'aquired', 'website', 'title', 'description', 'headline'])
final_df.head()
This returns:
headq industry aquired website title description headline
0 Chicago, Illinois Business Services March 2018 www.brilliantfs.com Brilliant Brilliant is a leading provider of temporary a... Brilliant is actively looking for add-on acqui...
1 New York, New York Consumer Services November 2017 caringpeopleinc.com Caring People Founded in 1998, Caring People is a leading ho... Caring People is actively looking for add-on a...
2 Denver, Colorado Business Services December 2021 ccsbts.com CCS Facility Services CCS Facility Services is a leading provider of... CCS is actively looking for add-on acquisition...
3 Atlanta, GA Consumer Services August 2021 unkown Drive Automotive Services Drive, headquartered in Atlanta, GA, is a lead... Drive is actively looking for add-on acquisiti...
4 Woburn, Massachusetts Healthcare Services December 2020 www.geriatricmedical.com Geriatric Medical Founded in 1945, Geriatric Medical is a leadin... Geriatric Medical is actively looking for add-...