As I said in my dwarf fortress post, sometimes players have to challenge themselves. Maybe there was no clear goal from the start, or the intended goal isn't the only thing that can be achieved.
Finding additional challenges that were or weren't intended by game developers is as old as gaming itself. Even in the earliest days, people were seeing how big of a streak, speedrun or extra difficulty they could put into a game.
Game developers often provide ways to challenge a player with difficult content or a difficult boss. Sometimes though, that's not enough. You mastered every aspect of the game but feel like you want even more. Some people eventually find something else to occupy themselves with, contend themselves to waiting for the next release or just start again from scratch.
Others though, feel they should try and test their true skills at a game and seek out challenge to do that.
Often these challenges deviate from the original purpose of the game and come up with interesting strategies not even the developers could predict.
Types of challenges
There are several types of challenges you or the developer can impose upon you. I'll put them in different categories:
-Limiting: limiting challenges are challenges that (as the name says) have the player put certain limits on what features they use. For example: not using items, magic, party members, only 1 gun type (usually the weakest or most interesting),...
-No advancement: Several genres (especially rpgs, but also shooters or rts'es) have one way of advancement. Challenges like these involve the act of abstaining from any progression of the sort. This usually means your character is ridiculously weak compared to the difficulty of the content. Resulting is the player needing to devise several complicated strategies to finish (or get past) the more difficult fights.
-Speedrun: Exactly what it says on the tin. A player will rush through the game's main story as fast as possible. This usually involves skipping as much of the content as possible. In games with a big level ramp, this is almost always combined with a low-level run.
-Permadeath: The act of removing your character/save once you die once. Done a lot in online games but also possible in offline games. A lot of people find this hard because they get attached to a character they've put a lot of time in, but others find this one of the draws. Your playstyle will evolve to be more careful, the tension is a lot bigger as the stakes are higher and the sense of achievement upon completing this challenge is immense.
-Soloing group content/superbosses: Most games have a few hidden superbosses or (in the case of online play) content that is designed for more than one player. These are usually intended by the developer.
-Physical limitations: An out-of-game challenge. You limit yourself to using a limited set of your motoric skills. Playing blind, using one hand, keyboard/mouse-only,... are examples of these challenges.
-Combination: Not really a challenge type, but you can combine any of the above types (or anything else you can think of) to form some kind of superchallenge.
As an example, here are some ways to challenge yourself in lotro:
-reach cap while playing permadeath (highly challenging for some classes, for others not so much)
-reach cap in as fast a time as possible
-play your class while only using a limited set of your skills
-solo combat elites, masters,... (easier for some classes than other)
-solo an instance (again easier for some classes)
-only do orange or higher quests (within reason of course)
-only use mediocre or bad equipment, while not playing to the strength of your class (e.g. cappy, using sword and board, wearing no barter armour)
-only use what you find/make yourself (no trading)
-any combination of the above
Turbine has (afaik) only put 2 challenges in the game:
-Permadeath: the undying titles, reaching lvl 20 without dying (which isn't that hard).
-No advancement/Limiting: chicken play: where you get to face all the dangers of the world ... but as a lvl 1 chicken.
Also: (Warning! TvTropes link!) Tvtropes has a bunch of pages on the subject: http://tvtropes.org/pmwiki/pmwiki.php/Main/SelfImposedChallenge.