hopefully with a strong/smart female lead, but not necessary. Anti-heroines and villainesses are my faves.

It can be an island criminals are sent to, or a spaceship, somewhere else with a large amount of criminals,or just a regular prison works too.

Historical/present day/futuristic all are wanted. if futuristic, dystopian would be excellent.

It could also be a delinquent school/juvenile detention center/ or something, but only if it has a somewhat dark tone and isn't comedy. There can be a little comedy, but overall tone has to be serious.

If there's a mastermind behind the scenes messing with everybody, that would be great

No Ecchi/hentai/adult/yaoi/yuri please.

________________
Perdition Prison- where there are no guards, no rules and no escape. Only the strong survive: http://perditionrp.freeforums.net/

Stone Ocean is pretty much what you are looking for

This one : Rainbow . There's also an anime based on it (the opening more or less shows the atmosphere), and I really recommend it.
Takes place in a juvenile detention center in the 50's, it's really, really dark and serious.

Seinen.

KANGOKU GAKUEN well it is of ecchi genre but really funny series.