The best games I've seen on it are:
Swords and Sworcery - Yawn, boring (I'm not really into RPG games...)
Plants vs Zombies - Fun, but loses value after 20-30 hours.
World of Goo - Fun, time-consuming, gets very difficult as you progress
Comments in red.
Now, as for my list of a few other games to try:
Risk - Hours of fun, but after you figure out how to do your strategery (sic), you can win four out of five times on expert with maximum computers. (Hint: let the other computers weaken one of them, then pounce and you can steamroll from there)
Tower Madness - Absolutely fantastic tower defense game, gets extremely hard.
Angry Birds/Seasons/Rio: Fantastic games, the level of continuing support is fantastic
Implode - Demolitions game, needs more challenges though.
AirAttack - WWII Galaga, unfortunately only takes a few hours to beat
Moto X Mayhem - Sidescroller motorcycle. I'm hoping for a version of Trials HD in the future, but this is pretty decent for now.
Osmos - Physics game, really fun and gets insanely difficult.
BTW, for those great iPhone games that you want to look good on the iPad, if you jailbreak that is: RetinaPad. It makes iPhone apps display at iPhone4 res, hence better looking.