1. Can I show the galleries either Alphabetically or in the order of latest created showing at the top. Currently it is showing as the oldest album on top. As the albums keep growing it will get cumbersome always scrolling down to see the latest gallery.
2. Can there be a pagination in the galleries as well so that after say 10,20 galleries per page a link gets added at the bottom saying something like "older galleries" or "more galleries", just like the simple blog addon?
If you are asking for features to be added to gpEasy then see the feature request off the contact page.
If you are asking if it can be done . . . no. gpEasy currently does not have those settings.
The galleries page is designed with drag-n-drop for ease of use. . . not necessarily high volume gallery sites. I understand what you would like; however, I like the exact opposite thus the developers can't do everything. It's just one of those things. Some want it ordered one way and others another. Some want all galleries on one page some only 10, etc. . .
It seems that the developers went with easy of use in mind and the idea that gpEasy is a lightweight CMS primarily designed for small to medium sites thus options to organize 100+ gallery pages doesn't seem like it's there. . . maybe in the future?
Other Options A suggestion for you is to create a page and add thumbnail images that link to your gallery pages. In effect you would be creating your own galleries page(s). It certainly will be a little more work than letting gpEasy handle it all but you will get the effect you want.