Need help with Python script on importing user-posted genres tag from dlsite.com by html scrapping.

sonicstream

New member
May 18, 2022
5
1
I have hundreds of doujin folders to manage so I wrote python script that output excel file where all doujin folders are hyperlinked for easier access and can be sort by author and name.

I'm a novice in Python programming so I compiled the script with help of ChatGPT, but then while the script is able to extract genres from official tag, the script cannot extract genres that is voted by user reviews.

My code is compiled so to collect text embedded in <a href="/maniax/fsr/=genre/\d+/"></a> and <li class="meny selected "></li>, combining them into one line of output string text.

While <a href="/maniax/fsr/=genre/\d+/"></a> worked as intended in extracting work genres, <li class="meny selected "></li> failed to extract review voted work genres.

What else do i need to modify?

for url in product_urls:
product_link = url['href']
if "product_id" in product_link: # product_idが含まれるリンクだけを対象とする
product_response = requests.get(product_link)
product_soup = BeautifulSoup(product_response.content, "html.parser")

# /maniax/fsr/=/genre/*番号*/ のパターンを持つリンクを取得(class="btn_default"の有無に関わらず)
genre_links = product_soup.find_all("a", href=re.compile(r'/maniax/fsr/=/genre/\d+/'))

# 取得したリンクから<a>タグのテキスト内容を抽出し、genres集合に追加
for genre_link in genre_links:
genres.add(genre_link.text)

# <ul class="meny_selected_list">の中の<li class="many_selected_item">を探す
selected_items = product_soup.select("ul.meny_selected_list > li.many_selected_item")

# 各<li>タグ内の<a>タグのテキストをgenres集合に追加
for item in selected_items:
if item.a: # <a>タグが存在する場合
genres.add(item.a.text)

# ジャンル名をカンマで区切って1つの文字列に連結
if genres:
genres_text = ", ".join(genres)
 

Attachments

  • 同人作品ファイルリスト整理_タグ追加.zip
    2.5 KB · Views: 45
Last edited:

Users who are viewing this thread

Latest profile posts

thebreaker2 wrote on Otokonoko's profile.
Could you please reupload RJ155505? Thank you so much!
mobius0417 wrote on Shine's profile.
[141010][ジャクシークリエイト] ドラマラクエスト (Ver 2.12) [RJ142825]
Please re-upload
Mr Gandhi wrote on Lebedev's profile.
Motto! Haramase! Honoo no Oppai Isekai Ero Mahou Gakuen!(もっと!孕ませ!炎のおっぱい異世界エロ魔法学園!)

[2019][Waffle] Hentai Elf Shimai to Majime Orc(変態エルフ姉妹と真面目オーク) [Save + uncensored]

The link is broken. Please repair
fishnets1901 wrote on Ryzen111's profile.
Hello, could you please reupload Rj01133991 ?