The Preface
When I first come to the eroge world, I've always wondered how do people rip the encrypted CG out of the package files. I've come to learn that the easy way is to use ExtractFile or Crass to do it. Unfortunately, not all the eroge works with those two overpopular software, especially newer title, so I tried to google the way to rip CG.
Unfortunately, most of the tutorial is in Chinese. However, I've come across a blog from a person called Haibara, although most of his blog is in Chinese, some very useful article were available in English. So I've decided to trial and error with his guide, and here I am, writing an introduction to Ripping and Processing CG.
The Intro
Kii.re isn't about just ripping the CG and dumping everything we ripped on it and the truth is ripping is just one of the task of being an uploader for Kii.re. There are actually 3 stages before a package is being uploaded on Kiire: Extracing the CG from encrypted game files, Processing the CG, and Optimization.
Extracting the CG involve in using appropriate tools to extract the game CG without loss of quality. Different game engines requires different tools. For example, Kirikiri2 engine can be extracted using xp3viewer, or Minori Musica engine can be extracted using asmodean's exef2paz. Most game can be done using Westside's tool, a commercial solution packages.
Once you dumped out the CG, you would find out that most of them have separated faces, bodies, and probably other parts as well (especially genitals and body fluid) to save space and processing power for the game to load fast and efficiently. Processing the CG meaning that one would merge these file into something usable for everyone.
Once you have merged these files together, you will need to delete all the smaller resolution files, trim out all of the unnecessary pixels, blank spaces for example, converting any bmp format into png format that have much better compression while retaining the quality, and optimizing the png (using PNGOUT) before uploading it to Kii.re
In this tutorial, I'll also include a link to download those tool, so you won't have them yourself. I'll also share most of my tools available (which is taken from Haibara's box.net), some other tools I got from ShareEX (Westside packages) and whatever I collected along the way. I'll also share my setup, how did I rip & process the eroge that I did in the upload sheet as I go.
This is the first tutorial I write for Kii.re, also, the first eroge to be uploaded onto Kii.re:
Kurano-kunchi no Futago Jijou
倉野くんちのふたご事情
http://vndb.org/v10793
The game shall be referred as kurafuta hence forth.
The Details
Before you continue, I have to warn you that is not an easy task at all, you would probably need a fairly powerful PC with a bunch of software. Personally, I make use of an independent Core i5 2500K, 16GB of RAM, about 3TB of HDDs and a cached 120GB SSD dedicate to this task. You must use Windows: either XP or Windows 7, nothing middle, nothing newer. I control this machine through Remote Desktop Connection. It also connect to a ZFS Storage Server - so that I can safely store the RAW files and the Packaged files. RAW files are original, unprocessed files, and Packaged files are processed, optimized, ready to be uploaded to Kii.re.
You would probably need loads of software aside of the ripping tools on it: Photoshop CS5 or CS6 for some post processing. I always go for the latest - CS6. ImageMagick (for post processing), PNGOUT (for optimization), Java Runtime (JRE) (certain processing software requires Java, Kii.re mass uploader also requires Java), ACDSEE Pro (for quickly previewing and assets the files) and dependencies that those above software requires (dotNetFX 3.5, 4.0 for example).
There are many alternative to those software but that's what I use. Feel free to provide suggestion of the alternatives.
kurafuta use Kirikiri2 engine, and the images is in xp3 package format. Kirikiri2 engine is an opensource visual novel engine in Japan, it's free to use, so you would find many eroge companies and doujinshi circles use it as the start. Most (if not all, although I don't rip enough to say all) Kirikiri2 game package the files in XP3. The best tools for kirikiri2 and xp3 packages is XP3Viewer. For older game (before 2011), you can probably use ExtractData too.
You have to have the kurafuta game installed.
Download XP3Viewer.
Extract it to somewhere.
Drag the kurafuta executable file or shortcut onto XP3Viewer.exe
The game should run along with XP3Viewer, you must leave it alone, there are music but you may want to mute it so, it won't be looping over and over again.
Drag the xp3 files you want to extract into XP3Viewer, set it to 32-bit PNG, don't use TLG Decoder.
char.xp3 = character sprites, most post processing works shall be this
event.xp3 = event (or ev) CG , most fapper want this.
data.xp3 = scripts, game engine files, may contain useful sprite such as game_logo
parts.xp3 = may contain many elements, in this case, backgrounds.
voice.xp3 = contain voice or music file, you can extract this and use it as mobile phone notification, or alarm sound. Oniichan Daisuki! for example works pretty well to make my day better.
Other games may have other files, it's best to extract them all.
I'd probably go grab something to eat or play something while it processes the files. This shouldn't take too long if your PC is strong enough. However, it might lags you use it for something else (like watching youtube). This is probably my number one reason to build another PC for ripping and processing.
Once you're done, you will notice that: the background and the event CG is fine, you don't have to merge anything, this is great. However, you will need to merge the faces in the char CG.
There are also a bunch of csv files in the char CG. These contain position value for the face, you will need these files to merge the faces together. For this, I use Kirikiri2CSV.jar to make it work.
You will need JRE 7 and ImageMagick 6.7.2 binary for Windows or newer, always use 32-bit version even if you're on Windows 64bit. So before continue, make sure you have them installed.
There are a few other things you need to adjust before JRE work on your system: go to "Environment Variables" in the System Setting of Windows, Find the PATH in System Variables and append the following line in the last:
If you had Java installed else where, change the path as necessary. the ; must stay there or else you will break your system totally. Press Ok.
Now run cmd.exe as Admin, type java, if it display a bunch of switches such as /a /b etc etc then it works, if it say java is not recognized as an internal or external command, then you probably did it wrong somewhere, check the path, relaunch cmd (you must relaunch cmd everytime you change the PATH value)
Go to the folder where you store your Kirikiri2CSV.jar, the command is:
Mind the cap. For me, it's
I extracted the char in D:\Ripping\Kurafuta. The csv is located inside char folder, and the folder I want it to merge is also char folder. Now make sure the files are in PNG format and not BMP format, this requirement however differs from tools to tools, games to games.
kirikiri2CSV.jar will create a bat file inside its folder named merge.bat, if you look at it, it will show:
These are ImageMagick command to merge the files according to the position value in CSV. Run the bat as Admin and wait for it to finish it work. If you have any error, check if you had ImageMagick installed properly.
Once it's done, you can find the merged folder inside the char folder, all the faces are merged with the sprites nicely. Now, you have done with processing the images. Next step is Optimizing the Package.
The First step of Optimization is to filter which files are going to be uploaded. If you take a look at the char folder for example, you will see the file list like this:
Upon further investigation, those L and M files are identical except that the L has a higher resolution than M. Before we conclude that we should discard all M and only keep L. Note that many games, the L files doesn't have the same counts as M. What it means is that in certain game, there are only a few files that are available in higher resolution, not all.
To be able to tell if this was the case, I'll actually sort all M and L files and count it. My method is to key in *L.png in the Windows search bar - Windows will automatically search out all files that has L in the end. Then I'll cut all the search result, make a new folder call Large, and place those L files inside there.
My results are: M files count: 532, L files count: 532. It means that we can safely discard the M files and only keep L files for upload. However, discarding the files doesn't mean you delete, discarding mean you're not going to include them in the final upload package. Remember, all RAW files should be kept in case of a ripping error.
The same process is apply to all other folders. In this particular rip, I only see the event CG that has similar format. So I count the L files to make sure it has the same number of the non L one before conclude that I won't be uploading them onto Kii.re
So the result are:
char: only L files in merged folders, plus L files that has full body in char folders - these are already there and the kirikiri2CSV left them untouched, you have to take a look at them and copy them out manually (666 files)
event: only files that has 1900 x 1080 dimensions (754 files)
parts/bg: all files
parts/face: looks like a bunch of faces again, though files counts are 666, char files counts are 666. We conclude that the we already had them all but I suggest you take a look a bit to make sure. In this case, we got them all. So this folder is discarded
parts/frame: this is the game interface package, you can scoop anything you deem useful here. To me, the company logo and game logo is the two that could be very useful, so I take two of them out.
parts/rule: these are effects mask to be used in game engine, currently, I don't see how it's being useful.
parts/sd: sd art, saved them all
parts/se: sound fx, I'd love to collect them all but not for kii.re
parts/thumb: save game thumbnail, discard it.
voice: voice file, all the moaning, ferachio, etc etc, enjoy them at your convenience.
Now what I'd do is that I'd copy all of the files into a master Folder call Upload, my Upload folder will have the following folder:
Checking char folder, I see there are a lot of blank space from all sides of the sprites images to be trimmed, hence I run the char through a trim batch automation command in Photoshop. It will take an hour or two depending on how many it has to process. You can use imagemagick command too but I don't know how to batch a lot of files yet, I'll update when I figure that out. It should be faster than Photoshop.
Finally, I run all of the images through PNGOUT. This process is long and brutal: it takes all 100% processing power from your computer for several hours: anywhere from 6-24 hours. This is the second reason why I built myself a reasonably powerful computer seperated from my workstation. During the process, you won't be able to use your computer for anything.
The process shall reduce the PNG file to smallest, out of my test, a 1.7MB PNG when run through PNGOUT will have a size of 1.15MB, that's about 550KB saving per file, and there are 1500 files to run through, that could be potentially 800MB of space to save for per game, much more on bandwidth in the long run. In this case, I saved 600MB of space after nearly 20 hours of running everything through PNGOUT.
THIS IS A WORK IN PROGRESS
When I first come to the eroge world, I've always wondered how do people rip the encrypted CG out of the package files. I've come to learn that the easy way is to use ExtractFile or Crass to do it. Unfortunately, not all the eroge works with those two overpopular software, especially newer title, so I tried to google the way to rip CG.
Unfortunately, most of the tutorial is in Chinese. However, I've come across a blog from a person called Haibara, although most of his blog is in Chinese, some very useful article were available in English. So I've decided to trial and error with his guide, and here I am, writing an introduction to Ripping and Processing CG.
The Intro
Kii.re isn't about just ripping the CG and dumping everything we ripped on it and the truth is ripping is just one of the task of being an uploader for Kii.re. There are actually 3 stages before a package is being uploaded on Kiire: Extracing the CG from encrypted game files, Processing the CG, and Optimization.
Extracting the CG involve in using appropriate tools to extract the game CG without loss of quality. Different game engines requires different tools. For example, Kirikiri2 engine can be extracted using xp3viewer, or Minori Musica engine can be extracted using asmodean's exef2paz. Most game can be done using Westside's tool, a commercial solution packages.
Once you dumped out the CG, you would find out that most of them have separated faces, bodies, and probably other parts as well (especially genitals and body fluid) to save space and processing power for the game to load fast and efficiently. Processing the CG meaning that one would merge these file into something usable for everyone.
Once you have merged these files together, you will need to delete all the smaller resolution files, trim out all of the unnecessary pixels, blank spaces for example, converting any bmp format into png format that have much better compression while retaining the quality, and optimizing the png (using PNGOUT) before uploading it to Kii.re
In this tutorial, I'll also include a link to download those tool, so you won't have them yourself. I'll also share most of my tools available (which is taken from Haibara's box.net), some other tools I got from ShareEX (Westside packages) and whatever I collected along the way. I'll also share my setup, how did I rip & process the eroge that I did in the upload sheet as I go.
This is the first tutorial I write for Kii.re, also, the first eroge to be uploaded onto Kii.re:
Kurano-kunchi no Futago Jijou
倉野くんちのふたご事情
http://vndb.org/v10793
The game shall be referred as kurafuta hence forth.
The Details
Before you continue, I have to warn you that is not an easy task at all, you would probably need a fairly powerful PC with a bunch of software. Personally, I make use of an independent Core i5 2500K, 16GB of RAM, about 3TB of HDDs and a cached 120GB SSD dedicate to this task. You must use Windows: either XP or Windows 7, nothing middle, nothing newer. I control this machine through Remote Desktop Connection. It also connect to a ZFS Storage Server - so that I can safely store the RAW files and the Packaged files. RAW files are original, unprocessed files, and Packaged files are processed, optimized, ready to be uploaded to Kii.re.
You would probably need loads of software aside of the ripping tools on it: Photoshop CS5 or CS6 for some post processing. I always go for the latest - CS6. ImageMagick (for post processing), PNGOUT (for optimization), Java Runtime (JRE) (certain processing software requires Java, Kii.re mass uploader also requires Java), ACDSEE Pro (for quickly previewing and assets the files) and dependencies that those above software requires (dotNetFX 3.5, 4.0 for example).
There are many alternative to those software but that's what I use. Feel free to provide suggestion of the alternatives.
kurafuta use Kirikiri2 engine, and the images is in xp3 package format. Kirikiri2 engine is an opensource visual novel engine in Japan, it's free to use, so you would find many eroge companies and doujinshi circles use it as the start. Most (if not all, although I don't rip enough to say all) Kirikiri2 game package the files in XP3. The best tools for kirikiri2 and xp3 packages is XP3Viewer. For older game (before 2011), you can probably use ExtractData too.
You have to have the kurafuta game installed.
Download XP3Viewer.
Extract it to somewhere.
Drag the kurafuta executable file or shortcut onto XP3Viewer.exe
The game should run along with XP3Viewer, you must leave it alone, there are music but you may want to mute it so, it won't be looping over and over again.
Drag the xp3 files you want to extract into XP3Viewer, set it to 32-bit PNG, don't use TLG Decoder.
char.xp3 = character sprites, most post processing works shall be this
event.xp3 = event (or ev) CG , most fapper want this.
data.xp3 = scripts, game engine files, may contain useful sprite such as game_logo
parts.xp3 = may contain many elements, in this case, backgrounds.
voice.xp3 = contain voice or music file, you can extract this and use it as mobile phone notification, or alarm sound. Oniichan Daisuki! for example works pretty well to make my day better.
Other games may have other files, it's best to extract them all.
I'd probably go grab something to eat or play something while it processes the files. This shouldn't take too long if your PC is strong enough. However, it might lags you use it for something else (like watching youtube). This is probably my number one reason to build another PC for ripping and processing.
Once you're done, you will notice that: the background and the event CG is fine, you don't have to merge anything, this is great. However, you will need to merge the faces in the char CG.
There are also a bunch of csv files in the char CG. These contain position value for the face, you will need these files to merge the faces together. For this, I use Kirikiri2CSV.jar to make it work.
You will need JRE 7 and ImageMagick 6.7.2 binary for Windows or newer, always use 32-bit version even if you're on Windows 64bit. So before continue, make sure you have them installed.
There are a few other things you need to adjust before JRE work on your system: go to "Environment Variables" in the System Setting of Windows, Find the PATH in System Variables and append the following line in the last:
Code:
;C:\Program Files (x86)\Java\jre7\binIf you had Java installed else where, change the path as necessary. the ; must stay there or else you will break your system totally. Press Ok.
Now run cmd.exe as Admin, type java, if it display a bunch of switches such as /a /b etc etc then it works, if it say java is not recognized as an internal or external command, then you probably did it wrong somewhere, check the path, relaunch cmd (you must relaunch cmd everytime you change the PATH value)
Go to the folder where you store your Kirikiri2CSV.jar, the command is:
Code:
kirikiri2CSV.jar -C "path_to_csv_folder" -d "path_to_char_folder"Mind the cap. For me, it's
Code:
kirikiri2CSV.jar -C "D:\Ripping\Kurafuta\char" -d "D:\Ripping\Kurafuta\char"I extracted the char in D:\Ripping\Kurafuta. The csv is located inside char folder, and the folder I want it to merge is also char folder. Now make sure the files are in PNG format and not BMP format, this requirement however differs from tools to tools, games to games.
kirikiri2CSV.jar will create a bat file inside its folder named merge.bat, if you look at it, it will show:
Code:
composite "D:\Ripping\Kurafuta\char\CA01002L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +263+192 "D:\Ripping\Kurafuta\char\merged\CA01002L.png"
composite "D:\Ripping\Kurafuta\char\CA01003L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +263+191 "D:\Ripping\Kurafuta\char\merged\CA01003L.png"
composite "D:\Ripping\Kurafuta\char\CA01004L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +257+193 "D:\Ripping\Kurafuta\char\merged\CA01004L.png"
composite "D:\Ripping\Kurafuta\char\CA01005L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +261+193 "D:\Ripping\Kurafuta\char\merged\CA01005L.png"
composite "D:\Ripping\Kurafuta\char\CA01002M.png" "D:\Ripping\Kurafuta\char\CA01001M.png" -geometry +147+109 "D:\Ripping\Kurafuta\char\merged\CA01002M.png"These are ImageMagick command to merge the files according to the position value in CSV. Run the bat as Admin and wait for it to finish it work. If you have any error, check if you had ImageMagick installed properly.
Once it's done, you can find the merged folder inside the char folder, all the faces are merged with the sprites nicely. Now, you have done with processing the images. Next step is Optimizing the Package.
The First step of Optimization is to filter which files are going to be uploaded. If you take a look at the char folder for example, you will see the file list like this:
Code:
CA01002L.png
CA01002M.png
CA01003L.png
CA01003M.pngUpon further investigation, those L and M files are identical except that the L has a higher resolution than M. Before we conclude that we should discard all M and only keep L. Note that many games, the L files doesn't have the same counts as M. What it means is that in certain game, there are only a few files that are available in higher resolution, not all.
To be able to tell if this was the case, I'll actually sort all M and L files and count it. My method is to key in *L.png in the Windows search bar - Windows will automatically search out all files that has L in the end. Then I'll cut all the search result, make a new folder call Large, and place those L files inside there.
My results are: M files count: 532, L files count: 532. It means that we can safely discard the M files and only keep L files for upload. However, discarding the files doesn't mean you delete, discarding mean you're not going to include them in the final upload package. Remember, all RAW files should be kept in case of a ripping error.
The same process is apply to all other folders. In this particular rip, I only see the event CG that has similar format. So I count the L files to make sure it has the same number of the non L one before conclude that I won't be uploading them onto Kii.re
So the result are:
char: only L files in merged folders, plus L files that has full body in char folders - these are already there and the kirikiri2CSV left them untouched, you have to take a look at them and copy them out manually (666 files)
event: only files that has 1900 x 1080 dimensions (754 files)
parts/bg: all files
parts/face: looks like a bunch of faces again, though files counts are 666, char files counts are 666. We conclude that the we already had them all but I suggest you take a look a bit to make sure. In this case, we got them all. So this folder is discarded
parts/frame: this is the game interface package, you can scoop anything you deem useful here. To me, the company logo and game logo is the two that could be very useful, so I take two of them out.
parts/rule: these are effects mask to be used in game engine, currently, I don't see how it's being useful.
parts/sd: sd art, saved them all
parts/se: sound fx, I'd love to collect them all but not for kii.re
parts/thumb: save game thumbnail, discard it.
voice: voice file, all the moaning, ferachio, etc etc, enjoy them at your convenience.
Now what I'd do is that I'd copy all of the files into a master Folder call Upload, my Upload folder will have the following folder:
Code:
bg
char
event
frame
sdChecking char folder, I see there are a lot of blank space from all sides of the sprites images to be trimmed, hence I run the char through a trim batch automation command in Photoshop. It will take an hour or two depending on how many it has to process. You can use imagemagick command too but I don't know how to batch a lot of files yet, I'll update when I figure that out. It should be faster than Photoshop.
Finally, I run all of the images through PNGOUT. This process is long and brutal: it takes all 100% processing power from your computer for several hours: anywhere from 6-24 hours. This is the second reason why I built myself a reasonably powerful computer seperated from my workstation. During the process, you won't be able to use your computer for anything.
The process shall reduce the PNG file to smallest, out of my test, a 1.7MB PNG when run through PNGOUT will have a size of 1.15MB, that's about 550KB saving per file, and there are 1500 files to run through, that could be potentially 800MB of space to save for per game, much more on bandwidth in the long run. In this case, I saved 600MB of space after nearly 20 hours of running everything through PNGOUT.
THIS IS A WORK IN PROGRESS
Last edited:
