python - href with 2 different links but with same content

Question

I wanted to create a small program using tutorials on youtube to scrape download links on a video website.

but the problem is that the 'href' that hold the download links sometimes hold the same content. In fact it's a forum, there's the link to download a video and the same video in smaller size just below. If we click on the video we are redirected to the same page that the link above redirects us.

Here is the small program:

import requests
from bs4 import BeautifulSoup

links =[]

for p in range(21):
    url = 'website url' + str(p)
    response = requests.get(url)
    print(response)

    if response.ok:
        print('Page: ' + str(p))
        soup = BeautifulSoup(response.text, "html.parser")
        iS = soup.findAll('i')

        for i in iS:
            a = i.find('a')
            link = a['href']
            links.append(link)

print(len(links))

with open('urls.txt', 'w') as file:
    for link in links:
        file.write(link + '
')


with open('urls.txt', 'r') as file:
    for row in file:
        print(row)

Answer 1

等待大神答复