Need help with Python script on importing user-posted genres tag from dlsite.com by html scrapping.

sonicstream

New member
May 18, 2022
5
1
I have hundreds of doujin folders to manage so I wrote python script that output excel file where all doujin folders are hyperlinked for easier access and can be sort by author and name.

I'm a novice in Python programming so I compiled the script with help of ChatGPT, but then while the script is able to extract genres from official tag, the script cannot extract genres that is voted by user reviews.

My code is compiled so to collect text embedded in <a href="/maniax/fsr/=genre/\d+/"></a> and <li class="meny selected "></li>, combining them into one line of output string text.

While <a href="/maniax/fsr/=genre/\d+/"></a> worked as intended in extracting work genres, <li class="meny selected "></li> failed to extract review voted work genres.

What else do i need to modify?

for url in product_urls:
product_link = url['href']
if "product_id" in product_link: # product_idが含まれるリンクだけを対象とする
product_response = requests.get(product_link)
product_soup = BeautifulSoup(product_response.content, "html.parser")

# /maniax/fsr/=/genre/*番号*/ のパターンを持つリンクを取得(class="btn_default"の有無に関わらず)
genre_links = product_soup.find_all("a", href=re.compile(r'/maniax/fsr/=/genre/\d+/'))

# 取得したリンクから<a>タグのテキスト内容を抽出し、genres集合に追加
for genre_link in genre_links:
genres.add(genre_link.text)

# <ul class="meny_selected_list">の中の<li class="many_selected_item">を探す
selected_items = product_soup.select("ul.meny_selected_list > li.many_selected_item")

# 各<li>タグ内の<a>タグのテキストをgenres集合に追加
for item in selected_items:
if item.a: # <a>タグが存在する場合
genres.add(item.a.text)

# ジャンル名をカンマで区切って1つの文字列に連結
if genres:
genres_text = ", ".join(genres)
 

Attachments

  • 同人作品ファイルリスト整理_タグ追加.zip
    2.5 KB · Views: 57
Last edited:

Users who are viewing this thread

Latest profile posts

xorthalndor wrote on Ryzen111's profile.
Hi Ryzen!
Can you please up
戦巫女ホノカ ―コンプリートパック―
to the latest 1.0.7? Thanx!
Deladela2000 wrote on Ryzen111's profile.
Hello, Ryzen.
Can you reup this game?
Deladela2000 wrote on Shine's profile.
Hello, Shine. Can you reup RJ081433?
ヨコシマ wrote on Shine's profile.
こんにちはshineさん
ラボスイーパー~ドロシーの秘密研究記録~ https://www.dlsite.com/maniax/work/=/product_id/RJ01282013.html
新しいバージョン 1 .04 が公開されました
更新をよろしくお願いいたします