Need help with Python script on importing user-posted genres tag from dlsite.com by html scrapping.

sonicstream

New member
May 18, 2022
5
1
I have hundreds of doujin folders to manage so I wrote python script that output excel file where all doujin folders are hyperlinked for easier access and can be sort by author and name.

I'm a novice in Python programming so I compiled the script with help of ChatGPT, but then while the script is able to extract genres from official tag, the script cannot extract genres that is voted by user reviews.

My code is compiled so to collect text embedded in <a href="/maniax/fsr/=genre/\d+/"></a> and <li class="meny selected "></li>, combining them into one line of output string text.

While <a href="/maniax/fsr/=genre/\d+/"></a> worked as intended in extracting work genres, <li class="meny selected "></li> failed to extract review voted work genres.

What else do i need to modify?

for url in product_urls:
product_link = url['href']
if "product_id" in product_link: # product_idが含まれるリンクだけを対象とする
product_response = requests.get(product_link)
product_soup = BeautifulSoup(product_response.content, "html.parser")

# /maniax/fsr/=/genre/*番号*/ のパターンを持つリンクを取得(class="btn_default"の有無に関わらず)
genre_links = product_soup.find_all("a", href=re.compile(r'/maniax/fsr/=/genre/\d+/'))

# 取得したリンクから<a>タグのテキスト内容を抽出し、genres集合に追加
for genre_link in genre_links:
genres.add(genre_link.text)

# <ul class="meny_selected_list">の中の<li class="many_selected_item">を探す
selected_items = product_soup.select("ul.meny_selected_list > li.many_selected_item")

# 各<li>タグ内の<a>タグのテキストをgenres集合に追加
for item in selected_items:
if item.a: # <a>タグが存在する場合
genres.add(item.a.text)

# ジャンル名をカンマで区切って1つの文字列に連結
if genres:
genres_text = ", ".join(genres)
 

Attachments

  • 同人作品ファイルリスト整理_タグ追加.zip
    2.5 KB · Views: 70
Last edited:

Users who are viewing this thread

Latest profile posts

ballball1110 wrote on Ryzen111's profile.
Hi, do you have this one?
[whisp]【限定特典截止至2025年05月26日/动画版】在和耳学园的医务室里,总是挤满了发情的兽耳娘♪〜学校里最优秀的天才少女一旦发情就会在我面前做出发情母狗般的动作♪ [RJ01372975]
or
[whisp]【2025年05月26日迄限定特典/アニメ版】和耳学園の保健室は、発情期のケモミミッ娘でいっぱい♪〜学園最高の天才才女は、発情したら腰ヘコメス犬ムーブを見せちゃいます♪ [RJ01365064]
Thank you
Cri wrote on Otokonoko's profile.
Hello, please update to new version (should be 1.1, was released 23rd February 2024)
https://www.anime-sharing.com/threads/🔥new⚡-231216-くるはくる-催眠デカ尻jk陸上部-rj01106925.1409429/