If you can't find the lists containing the emails/passwords, you can check your email/username here: http://haveibeenpwned.com/



No offense, Batoto I can understand (since their online reader doesn't compress images and there a few Batoto-based scanlators there), but why in the world would you rip and archive scans from other LQ online readers when you can download the .zip/.rar in HQ directly from the scanlator's sites?!? Doesn't that defeat the whole purpose of archiving and saving files that should be in HQ and upload it in a place that's easily accessible for many people?



I think Harudaki should be considered an exception because NOBODY except the scanlators themselves has access to read or download chapter 42 of Crimson Spell or the latest Haru wo Daiteita/Viewfinder as they don't let you download it until about 2 weeks after the date it is listed here on MU on IRC for whatever reason.
If you see any other Crimson Spell releases on online readers, they are most likely from Mangafox-based scanlators who used parts from whole Chinese raw chapters to release in super LQ. And one really shouldn't trust the chapter listings on online readers as I've often found many mistakes on MangaFox's series pages.

For all other release except Crimson Spell, Haru wo Daiteita and Viewfinder I've explained previously in another thread why nobody can download/read their releases:

It's not just these. For example, the most recent releases of Tsugumomo aren't on the Madokami site either. They were put up on Batoto a few days ago, and since my normal downloader program wasn't working for some reason, I tried to grab them off Madokami's site, only to find they weren't yet uploaded. Had to go and grab each page manually, which for 3 chapters at over 30 pages per chapter was a real pain!

@tart
ohhh ok. yeah i didn't know what the situation with harudaki was until now.


@fake-name
i agree with tart that releases shouldn't be grabbed from online unless its completely impossible to find them from the scanlator so thats a no go in my book. you should probably make a list of everything you've grabbed from online readers so that they can be replaced with the actual hq originals.

________________
Fujoshis rule!

Because it's a completely automated system, and I can't write discrete scrapers for every site under the sun?

I do pull content from a number of original sources, and MangaJoy doesn't appear to re-compress files, at least the majority of the time.

Each new site is 2-4 hours of work to add, so I try to chose the highest-volume sites, while balancing against image quality. I do also have a system for automatic deduplication, even in the cases where the images are in a different archive format, and I have plans to eventually do perceptual-hashing based deduplication, but that needs more work.

Eventually, assuming I get p-hash duplicate detection working, I should be able to automatically sort for highest-resolution copies, and preserve just those. I think I'm going to have to implement a custom indexing system in PostgreSQL to get it to work, though (I need to be able to search by hamming distance across hash values).

If you are interested, Contribute! It's a Open-Source project: http://github.com/fake-name/MangaCMS/, and I'll happily accept pull-requests for new scrapers, and even run them using my own bandwidth.

---

FYI: P-hashing, or perceptual hashing is "approximate" image searching - think google image search or tin-eye. It's a way of matching images even if they have been recompressed.

Last edited by fake-name at 9:55 pm, Sep 10 2014

Not really, just checked the scanlator's official page for Tsugumomo, and it seems like they don't provide DL for the latest chapter until maybe later (latest I could find was chapter 33.1). Looks like it's online reader at Batoto for newest chapter only unless you try asking them to add the new DL to their site?

I realize that I can't get the downloads from the scanlator's site. However, they've been up on Batoto and up till now I've used a program called Free Manga Downloader to grab the chapters off Batoto when they're released as I hate using online readers.

For some reason though, the program wasn't working, and when I tried another similar program I ran into the same problem. So I tried using the Madokami site to get the chapters from there, only to find they didn't have the latest 3 chapters released either. They have up until 35.1, but not the latest 3 chapters even though it's been days since they were put onto Batoto (just checked now, and as of this post that's still the case).

They have up to ch 43, and have for at least the last 18 days.

Note that they're labelled '(JP to KOR to ENG)', but they're the same scans as on batoto (or every single image has a MD5 hash collision with the batoto files, which is functionally impossible).

I'm not sure why batoto is so many chapters behind.


EDIT: It looks like there is drama going on on batoto regarding uploads. things have been uploaded, and then taken down a number of times. At some point, someone uploaded both chapter 39, and chapter *46*, that have subsequently been removed.

I have no idea what is going on there.

Last edited by fake-name at 5:19 am, Sep 11 2014

I don't think those are from the same group though. All the other releases have been labeled with the correct group name, and those ones you are talking about are not, which makes me believe they are not from /a/nimango scans.

Plus, all the recent releases from /a/nimango scans are in the "Needs Sorting - Auto Uploaded from Batoto" folder, where as the one's you're referring to are not, again making me think they are older releases from some other individual/group.

Their sharing folder is based in MEGA's cloud sharing system. You will need the either Chrome or a recent version of Mozilla order to download from the site (you can also get the MEGA extension add-on if you want faster loading/performance).

I'm using Chrome, so I'm not sure what you're trying to say. The scanlator's don't have their most recent uploads in their MEGA folder, and they don't have an IRC channel. They've been uploading their chapters to Batoto, and since I don't like reading online I've been using a tool to grab the chapters from Batoto and download them to my computer. For some reason, this wasn't working with the latest release (3 chapters released a couple of days ago), so I tried using the Madokami site, but even though they say, as posted earlier in this thread, that they get new releases from Batoto within 8 hours, it's still not up even as of now.

So, without any other recourse, I had to manually grab each chapter from Batoto, which is a real pain, but there you are.

All I've been saying all along is that it's not just the one exception that you said, and gave another example of a title that's not been updated on Madokami.

@fake-name
its not a big deal but Real Account only has chapters 1,2,3,6, and 7 on madokami despite all of the chapters being on batoto and other sites. there are currently 8 chapters(chapter 8 is only on batoto right now as far as i can tell).
((again this isnt really a big deal but its just another title as an example.))

________________
Fujoshis rule!

What I was trying to say unlike Harudaki where old and new releases are not made available to those outside of the LJ community and IRC channel, all of /a/nimango scans's releases have no restrictions and is free for anybody to download/upload from MadoKami. Though /a/nimango scans is being slow and not providing DL for their most recent releases (why not try emailing them or visiting 4chan?), l don't consider them them as an exception along the same veins as Harudaki.

(I just mentioned Chrome because when you previously said you couldn't "get the downloads from the scanlator's site," I merely assumed it was because you were using another broswer)



Just checked IRST's site and it looks like they only have chapter 8 of Real Account available for download. I assume it is because they're a new group and upload directly Batoto (which later gets ripped by other online reader sites). They also don't seem to have any restrictions on sharing their releases so I wouldn't consider them an exception unlike Harudaki.

tart, you seem to be having difficulty understanding what myself and others are trying to say. We are referring to this statement, made on page 3 of this thread:



I, and others, have given examples where this is not the case. Madokami does not only put up files that they can directly download from scanlator's sites. They also grab chapters from Batoto and other online reader sites. And as has been pointed out, this does not always happen in a timely manner. Whether or not you think our examples fit some sort of criteria of yours, the fact remains that no, not all recent releases are put up on the Madokami site as quickly as has been stated.

You seem to have missed the fact that the post *you quoted* was edited to say I had found an issue that was disrupting the uploads, and I think had been doing so for a number of days. I also added the upload facility a fair bit of time after the main Madokami site went active (I needed to mirror it completely so I could avoid uploading duplicates - that took about a month).

The actual quote:



This forum does not seem to have any way to do a strike-through text. ~~~{text}~~~ is markdown for <strike>{text}</strike>, though neither that, or [strike]{text}[/strike] works.

You also seem to be missing the fact that I'm the person doing a lot of the uploading, with software I wrote to do it! It certainly has issues, but it's better then what anyone else seems to be doing. This isn't what "they're" doing, it's what I'm doing, or at least trying to do. I'm not speaking for anyone else, here.

I actually have that file already, but it was downloaded during the period of time that my uploader was shitting itself, so it didn't get mirrored properly. It was first seen at 14-09-09 17:46.

I'm going to do a bunch of reorganizing this evening, and see about adding a module to do historical uploads. It's much easier to just upload new things, then try to upload older files only if they're not duplicates (whooo! set intersections). I think I can probably at least walk back in time a week or two without breaking anything.