[WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

The Preface

When I first come to the eroge world, I've always wondered how do people rip the encrypted CG out of the package files. I've come to learn that the easy way is to use ExtractFile or Crass to do it. Unfortunately, not all the eroge works with those two overpopular software, especially newer title, so I tried to google the way to rip CG.

Unfortunately, most of the tutorial is in Chinese. However, I've come across a blog from a person called Haibara, although most of his blog is in Chinese, some very useful article were available in English. So I've decided to trial and error with his guide, and here I am, writing an introduction to Ripping and Processing CG.

The Intro

Kii.re isn't about just ripping the CG and dumping everything we ripped on it and the truth is ripping is just one of the task of being an uploader for Kii.re. There are actually 3 stages before a package is being uploaded on Kiire: Extracing the CG from encrypted game files, Processing the CG, and Optimization.

Extracting the CG involve in using appropriate tools to extract the game CG without loss of quality. Different game engines requires different tools. For example, Kirikiri2 engine can be extracted using xp3viewer, or Minori Musica engine can be extracted using asmodean's exef2paz. Most game can be done using Westside's tool, a commercial solution packages.

Once you dumped out the CG, you would find out that most of them have separated faces, bodies, and probably other parts as well (especially genitals and body fluid) to save space and processing power for the game to load fast and efficiently. Processing the CG meaning that one would merge these file into something usable for everyone.

Once you have merged these files together, you will need to delete all the smaller resolution files, trim out all of the unnecessary pixels, blank spaces for example, converting any bmp format into png format that have much better compression while retaining the quality, and optimizing the png (using PNGOUT) before uploading it to Kii.re

In this tutorial, I'll also include a link to download those tool, so you won't have them yourself. I'll also share most of my tools available (which is taken from Haibara's box.net), some other tools I got from ShareEX (Westside packages) and whatever I collected along the way. I'll also share my setup, how did I rip & process the eroge that I did in the upload sheet as I go.

This is the first tutorial I write for Kii.re, also, the first eroge to be uploaded onto Kii.re:
Kurano-kunchi no Futago Jijou
倉野くんちのふたご事情
http://vndb.org/v10793

The game shall be referred as kurafuta hence forth.

The Details

Before you continue, I have to warn you that is not an easy task at all, you would probably need a fairly powerful PC with a bunch of software. Personally, I make use of an independent Core i5 2500K, 16GB of RAM, about 3TB of HDDs and a cached 120GB SSD dedicate to this task. You must use Windows: either XP or Windows 7, nothing middle, nothing newer. I control this machine through Remote Desktop Connection. It also connect to a ZFS Storage Server - so that I can safely store the RAW files and the Packaged files. RAW files are original, unprocessed files, and Packaged files are processed, optimized, ready to be uploaded to Kii.re.

You would probably need loads of software aside of the ripping tools on it: Photoshop CS5 or CS6 for some post processing. I always go for the latest - CS6. ImageMagick (for post processing), PNGOUT (for optimization), Java Runtime (JRE) (certain processing software requires Java, Kii.re mass uploader also requires Java), ACDSEE Pro (for quickly previewing and assets the files) and dependencies that those above software requires (dotNetFX 3.5, 4.0 for example).

There are many alternative to those software but that's what I use. Feel free to provide suggestion of the alternatives.

kurafuta use Kirikiri2 engine, and the images is in xp3 package format. Kirikiri2 engine is an opensource visual novel engine in Japan, it's free to use, so you would find many eroge companies and doujinshi circles use it as the start. Most (if not all, although I don't rip enough to say all) Kirikiri2 game package the files in XP3. The best tools for kirikiri2 and xp3 packages is XP3Viewer. For older game (before 2011), you can probably use ExtractData too.

You have to have the kurafuta game installed.

Download XP3Viewer.
Extract it to somewhere.
Drag the kurafuta executable file or shortcut onto XP3Viewer.exe
The game should run along with XP3Viewer, you must leave it alone, there are music but you may want to mute it so, it won't be looping over and over again.
Drag the xp3 files you want to extract into XP3Viewer, set it to 32-bit PNG, don't use TLG Decoder.
char.xp3 = character sprites, most post processing works shall be this
event.xp3 = event (or ev) CG , most fapper want this.
data.xp3 = scripts, game engine files, may contain useful sprite such as game_logo
parts.xp3 = may contain many elements, in this case, backgrounds.
voice.xp3 = contain voice or music file, you can extract this and use it as mobile phone notification, or alarm sound. Oniichan Daisuki! for example works pretty well to make my day better.

Other games may have other files, it's best to extract them all.

I'd probably go grab something to eat or play something while it processes the files. This shouldn't take too long if your PC is strong enough. However, it might lags you use it for something else (like watching youtube). This is probably my number one reason to build another PC for ripping and processing.

Once you're done, you will notice that: the background and the event CG is fine, you don't have to merge anything, this is great. However, you will need to merge the faces in the char CG.

There are also a bunch of csv files in the char CG. These contain position value for the face, you will need these files to merge the faces together. For this, I use Kirikiri2CSV.jar to make it work.

You will need JRE 7 and ImageMagick 6.7.2 binary for Windows or newer, always use 32-bit version even if you're on Windows 64bit. So before continue, make sure you have them installed.

There are a few other things you need to adjust before JRE work on your system: go to "Environment Variables" in the System Setting of Windows, Find the PATH in System Variables and append the following line in the last:
Code:
;C:\Program Files (x86)\Java\jre7\bin

If you had Java installed else where, change the path as necessary. the ; must stay there or else you will break your system totally. Press Ok.


Now run cmd.exe as Admin, type java, if it display a bunch of switches such as /a /b etc etc then it works, if it say java is not recognized as an internal or external command, then you probably did it wrong somewhere, check the path, relaunch cmd (you must relaunch cmd everytime you change the PATH value)

Go to the folder where you store your Kirikiri2CSV.jar, the command is:
Code:
kirikiri2CSV.jar -C "path_to_csv_folder" -d "path_to_char_folder"

Mind the cap. For me, it's
Code:
kirikiri2CSV.jar -C "D:\Ripping\Kurafuta\char" -d "D:\Ripping\Kurafuta\char"

I extracted the char in D:\Ripping\Kurafuta. The csv is located inside char folder, and the folder I want it to merge is also char folder. Now make sure the files are in PNG format and not BMP format, this requirement however differs from tools to tools, games to games.

kirikiri2CSV.jar will create a bat file inside its folder named merge.bat, if you look at it, it will show:

Code:
composite "D:\Ripping\Kurafuta\char\CA01002L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +263+192 "D:\Ripping\Kurafuta\char\merged\CA01002L.png"

composite "D:\Ripping\Kurafuta\char\CA01003L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +263+191 "D:\Ripping\Kurafuta\char\merged\CA01003L.png"

composite "D:\Ripping\Kurafuta\char\CA01004L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +257+193 "D:\Ripping\Kurafuta\char\merged\CA01004L.png"

composite "D:\Ripping\Kurafuta\char\CA01005L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +261+193 "D:\Ripping\Kurafuta\char\merged\CA01005L.png"

composite "D:\Ripping\Kurafuta\char\CA01002M.png" "D:\Ripping\Kurafuta\char\CA01001M.png" -geometry +147+109 "D:\Ripping\Kurafuta\char\merged\CA01002M.png"

These are ImageMagick command to merge the files according to the position value in CSV. Run the bat as Admin and wait for it to finish it work. If you have any error, check if you had ImageMagick installed properly.

Once it's done, you can find the merged folder inside the char folder, all the faces are merged with the sprites nicely. Now, you have done with processing the images. Next step is Optimizing the Package.

The First step of Optimization is to filter which files are going to be uploaded. If you take a look at the char folder for example, you will see the file list like this:
Code:
CA01002L.png
CA01002M.png
CA01003L.png
CA01003M.png

Upon further investigation, those L and M files are identical except that the L has a higher resolution than M. Before we conclude that we should discard all M and only keep L. Note that many games, the L files doesn't have the same counts as M. What it means is that in certain game, there are only a few files that are available in higher resolution, not all.

To be able to tell if this was the case, I'll actually sort all M and L files and count it. My method is to key in *L.png in the Windows search bar - Windows will automatically search out all files that has L in the end. Then I'll cut all the search result, make a new folder call Large, and place those L files inside there.

My results are: M files count: 532, L files count: 532. It means that we can safely discard the M files and only keep L files for upload. However, discarding the files doesn't mean you delete, discarding mean you're not going to include them in the final upload package. Remember, all RAW files should be kept in case of a ripping error.

The same process is apply to all other folders. In this particular rip, I only see the event CG that has similar format. So I count the L files to make sure it has the same number of the non L one before conclude that I won't be uploading them onto Kii.re

So the result are:
char: only L files in merged folders, plus L files that has full body in char folders - these are already there and the kirikiri2CSV left them untouched, you have to take a look at them and copy them out manually (666 files)
event: only files that has 1900 x 1080 dimensions (754 files)
parts/bg: all files
parts/face: looks like a bunch of faces again, though files counts are 666, char files counts are 666. We conclude that the we already had them all but I suggest you take a look a bit to make sure. In this case, we got them all. So this folder is discarded
parts/frame: this is the game interface package, you can scoop anything you deem useful here. To me, the company logo and game logo is the two that could be very useful, so I take two of them out.
parts/rule: these are effects mask to be used in game engine, currently, I don't see how it's being useful.
parts/sd: sd art, saved them all
parts/se: sound fx, I'd love to collect them all but not for kii.re
parts/thumb: save game thumbnail, discard it.
voice: voice file, all the moaning, ferachio, etc etc, enjoy them at your convenience.

Now what I'd do is that I'd copy all of the files into a master Folder call Upload, my Upload folder will have the following folder:
Code:
bg
char
event
frame
sd

Checking char folder, I see there are a lot of blank space from all sides of the sprites images to be trimmed, hence I run the char through a trim batch automation command in Photoshop. It will take an hour or two depending on how many it has to process. You can use imagemagick command too but I don't know how to batch a lot of files yet, I'll update when I figure that out. It should be faster than Photoshop.

Finally, I run all of the images through PNGOUT. This process is long and brutal: it takes all 100% processing power from your computer for several hours: anywhere from 6-24 hours. This is the second reason why I built myself a reasonably powerful computer seperated from my workstation. During the process, you won't be able to use your computer for anything.

The process shall reduce the PNG file to smallest, out of my test, a 1.7MB PNG when run through PNGOUT will have a size of 1.15MB, that's about 550KB saving per file, and there are 1500 files to run through, that could be potentially 800MB of space to save for per game, much more on bandwidth in the long run. In this case, I saved 600MB of space after nearly 20 hours of running everything through PNGOUT.

THIS IS A WORK IN PROGRESS
 
Last edited:
For some reason kirikiri2csv.jar failed to work for me (I think java problem because ImageMagick works after all the trouble I went to install it).

Good thing I found a better way to take care of those CSVs and merging operations (Before I found that I made do by taking the ImageMagick merge code from here and manually input the x/y value from the CSV into the code. Tedious, but at least it works).
 
I haven't had much success in extracting the data. But I need character sprites, background pictures and effect pictures extracted from Tsujidou-san no Jun'ai road and Tsujidou-san no Virgin road. I'm thinking of making an animated video of the drama cd and translating it while I go. But I need the extracted image data from the games. I've used every program I can think of and I haven't had much success. I'd really appreciate it someone could be able to extract the picture data from the games for me.
 
I haven't had much success in extracting the data. But I need character sprites, background pictures and effect pictures extracted from Tsujidou-san no Jun'ai road and Tsujidou-san no Virgin road. I'm thinking of making an animated video of the drama cd and translating it while I go. But I need the extracted image data from the games. I've used every program I can think of and I haven't had much success. I'd really appreciate it someone could be able to extract the picture data from the games for me.

Check this out.

Wish I could help more, but those two games aren't in my line of interest and with KO'd internet I can't check them out anytime soon.
 
I hope this is the right section to ask for help regarding my problem :D

I'm working on a German translation of Yosuga no Sora and I'm looking for a way to decompress the archives and then compress it back (mostly for the menu of the game). The archives have the file extension .NOA and I tried several "all-purpose tools" like AE Tools, ExtractData etc. but without success.

I would be grateful if someone knows how to decompress and compress them back. Thx in advance for an answer :)
 
Not sure if this is the right place to post it. Been trying to extract 月に寄りそう乙女の作法 CGs. I believe it's using the QLiE engine. I've found and tried to use ExtractData however everything is showing up as corrupted (encrypted). I'm wondering if there's a way to decrypt or is this not possible.
 
Not sure if this is the right place to post it. Been trying to extract 月に寄りそう乙女の作法 CGs. I believe it's using the QLiE engine. I've found and tried to use ExtractData however everything is showing up as corrupted (encrypted). I'm wondering if there's a way to decrypt or is this not possible.

ExtractData can only get you so far...

My source says you need a jk5cv for that one.
 
hello all.. i hope it isn't the wrong thread to post this, but i've been trying to try extract psbpack.dat file from one of Anim games. i think there are chara files in that DAT file because i can't find the chara files in cg folder of the game (i can only find the bg+event cgs)... crass and extractdata didn't work to extract the file, though... any solutions?
 
Hello,

With the recent release of 催眠学園1年生 (Saimin Gakuen 1-nensei) I'm trying to extract the characters files set.

The game's files are in .med. I have seached all around the web and older games from Fudegaki Soft in .med should be extracted using purgodcv(westside) or exmed(asmodean) so it should be the same here.

My problem is I can't figure out how using MediaExtractor from Westside since it is fully in japanese. I can launch a scan but it stop after few seconds with nothing new.

Same goes for Exmed witch is in command line (I'm far more familiar with that). I'm doing something like .../exmed.exe .../*.med , something happen for few seconds and, again, nothing more. :reallyconfused:

Can someone explain me how using Exmed or Westside ? That would be much appreciated :bigeyes:
 

Users who are viewing this thread

Latest profile posts

xorthalndor wrote on Ryzen111's profile.
Hi Ryzen!
Can you please up
戦巫女ホノカ ―コンプリートパック―
to the latest 1.0.7? Thanx!
Deladela2000 wrote on Ryzen111's profile.
Hello, Ryzen.
Can you reup this game?
Deladela2000 wrote on Shine's profile.
Hello, Shine. Can you reup RJ081433?
ヨコシマ wrote on Shine's profile.
こんにちはshineさん
ラボスイーパー~ドロシーの秘密研究記録~ https://www.dlsite.com/maniax/work/=/product_id/RJ01282013.html
新しいバージョン 1 .04 が公開されました
更新をよろしくお願いいたします