python - How to find all elements with a custom html attribute regardless of html tag using Beautiful Soup?

Question

I have two cases where i want to scrape html tags with custom html attributes This is the example of the html. How do you scrape all the elements with the custom attribute "limit".

<div class="names" limit="10">Bar</div> 
<div id="30" limit="20">Foo</div> 
<li limit="x">Baz</li>

The second case is similar but with all the same html tags

<div class="names" limit="10">Bar</div> 
<div class="names" limit="20">Bar</div> 
<div class="names" limit="30">Bar</div> 

My question is different than How to find tags with only certain attributes - BeautifulSoup because the latter targets attribute values with a specific tag whereas my question finds attributes only regardless of tag or value

Answer 1

# First case:
soup.find_all(attrs={"limit":True})

# Second case:
soup.find_all("div", attrs={"limit":True})

Reference:

• http://www.crummy.com/software/BeautifulSoup/bs4/doc/#kwargs
• http://www.crummy.com/software/BeautifulSoup/bs4/doc/#find-all

---

If your attribute name doesn't collide with either Python keywords or soup.find_all named args, the syntax is simpler:

soup.find_all(id=True)