One of the first computer games I ever played was a game on the VIC-20 called "Night Driving". A black screen, with colored rectangles representing the reflectors on the side of the road. It was a celebration of the limitations of the platform - there was no way that computer could display anything like a real road, so it turned that weakness into a selling point. Now, It was hardly a great game - it's other main selling point was that it was one of maybe 3 games available - but I always loved the sheer chutzpah of it.
I think of this sometimes when I look at runaway successes in literature, and I'm specifically looking at the Harry Potter and Twilight franchises, with a little bit of an eye towards Dan Brown. I look at those and one common thread runs between them: the manner in which they inspire nerdfury. Specifically, they're genre books that are fairly indifferent to the rules of genre. You'd think that would be a problem, but I sometimes suspect that it's actually a big advantage.
At the most obvious explanation is a trifle cynical. For any given genre, there are a certain number of people who are interested, and a vastly smaller number of people who care a whole hell of a lot. If you write for those enthusiasts then there's a good chance that you're not going to be too interesting to the larger populace because they've already demonstrated that they don't really care that much.
A slightly less cynical explanation is that the basics of a genre are usually very quickly grasped, and it's those basics that really matter. Sticking to those basics (whatever they are) is more likely to appeal. Moving away from them, even if it's "more" into the genre, is moving away from what people came for.
There's a nice lesson there - if you're enthusiastic about a genre and willing to just use it to launch into a story you're excited about, then there are examples of how well it can work. Naturally, some readers immediately want to point out that these examples of how things can work are reprehensible affronts against all that is literature, so we can just take that rage as a given.
Now, the counterpoint is that the more purely you serve a genre, the more you create an audience for yourself. The people who are REALLY into a genre are always looking out for something to serve their level of interest, and they constitute a pre-existing audience. You can do well by playing to that, even in a market as small as the RPG industry (some would say you are already doing so by writing RPGs but, hey, into every life a little recursion must fall).
The problem comes, I think, when the choice is not made, and you end up kind of half-falling into genre. If you make the choice consciously, you can work with it. If you make it accidentally then you're a lot more dependent on luck than you are on your talent and effort, and that's a pretty tenuous position.