Java
Indonesia is a vast country. For many people Bali is the only part they bother to see, which is almost equivalent to, for example, visiting Pondicherry & thinking you've seen India.
Java is the easiest alternative, and has a plethora of worthwhile, even spectacular, sights that are relatively easy for an independent traveller to access, with many linked by a reliable rail network.
Yogyakarta
The cultural capital, self proclaimed, of Java, Yogya sees by far the most foreign tourists. These days it's a bit of a sprawling, multi-centred city, with places to stay often a fair distance from either the main sites, or even to access Yogya's busy street food scene. The major point of interest is probably the Kraton, or former royal palace. This is worthwhile in it's own way, but a walk around the surrounding area, once home to the support players of the royal household is equally interesting. Taman Sari, the former royal baths & garden complex is about 1/2 a kilometre to the south of the Kraton, via winding streets and old walled mansions. To the north of the Kraton lies the old colonial administrative district, and the start of Jalan Marlioboro, Yogya's main street, particularly lively of an evening.
Yogyakarta has a reputation as a contemporary artistic hub, but this can be hard to access amid the plethora of tourist focused painting & batik galleries, with their sunset views of Borobudur, and glamorous apsara dancers in rainbow colours. It's also not a city particularly conducive to walking outside the old colonial area, which is a pity as many little kampong areas are tucked away off the main drags.
Outside Yogyakarta.
The vast majority of people come to Yogya to see Borobudur. It is a pretty stupendous edifice, now heavily restored. But be prepared for the full tourist experience these days, be that the crowds clambering round, the entrance & exits through the gift shops, and the small army of touts looking to make a living.
Unfortunately far fewer people bother to visit Prambanan, the great Hindu temple complex that's even closer to Yogya. Even if you're a veteran of Indian temples, Prambanan is absolutely worth visiting. There are also multiple other fascinating sites surrounding the main Prambanan temple, so it's well worth while to hire a car & driver to make a full day of it. Some notable others in the area include Ratu Boku, very pleasant to walk around, & Plaosan Temple, an ornate Buddhist complex.
The third significant travel option from Yogya is engaging with Mt Merapi. As a very active volcano just how you do this will depend on how Merapi is behaving. There are numerous options for visiting this area, though all are best accessed by car, ranging from mountain treks to jeep tours across recent lava flows. You might not want to engage with the full Indonesian kitsch of the "Lost World Castle", but the Merapi Volcano Museum should figure in your itinerary.
Surakarta (Solo)
Surakarta, or Solo as it was known, is a medium sized city an easy hour away from Yogya by train. Long in competition with Yogya for cultural supremacy, and notwithstanding a reputation for being socially conservative, Surakarta is possibly ahead of Yogya these days in terms of being an easy, pleasant, and interesting city to visit.
Surakarta has three Kratons or royal palaces, none quite as grand as in Yogya, and a bit more frayed around the edges, but much more alive in terms of cultural stuff. You're more likely to catch a dance or gamelan rehearsal in Surakarta's Kratons these days, and enjoy it without the crowds & touts. Surakarta is also still home to a thriving batik cottage industry, centred in a quite walkable neighbourhood, plus the House of Danar Hardi, Indonesia's best & most stunning batik museum.
As per Yogyakarta, there are also fascinating day trips available from Surakarta. Candi Ceto (or Cetho), a Hindu temple on the slopes of Mt Lawu, an hour & a bit outside town, is a superb place to visit, both for the extremely scenic views on the way, and the magical, often misty or cloud wrapped site itself. Candi Sukuh, 1/2 an hour down the road offers superb views, and a good loop trip back to Surakarta.
Surabaya
Surabaya is a major Indonesian city & transport hub on the north east coast of Java. A long time trading entrepot, and industrial hub, it's undergone a significant clean up in recent years. Back in the late 70's it welcomed travellers with a fog of sulphurous smog, belching from oil, gas & other industries, and a chaotic bus station that was a compulsory pit stop between Jakarta & Bali. Nowadays it's still traffic choked, but it's much prouder of it's historic buildings & independence heritage. A couple of interesting sights include the kretek factory museum & gallery, the House of Sampoerna, and the Majapahit Hotel, a wonderful 1910 hotel built by one of the Sarkies family. You could do worse when travelling in Asia than to try and collect the set of Sarkies family hotels, which also include Raffles in Singapore, the Strand Hotel in Yangon, and the Eastern & Oriental in Penang.
Being firmly in East Java, the food takes a noticeable turn away from from sugar laced dishes of Central Java. One speciality worth tucking into is any stew using the black nut or keluak, (or kenari, or kluwak), a potentially toxic ingredient that lends a lovely aromatic earthiness to some East Javanese dishes.
Malang
Malang is a pleasant small city nestled in the hills of East Java. As such it has a cooler climate, and was therefore a favourite retreat of the Dutch, with a good stock of historic buildings still standing, mostly around the old City hall area. One of these buildings is now the hotel Tugu Malang, which contains a fascinating private art collection of historic Javanese objects. It is possible to see this without necessarily staying there, but the bonus traditional afternoon teas, common to all the Tugu hotels, make a stay worthwhile anyway. Incidentally, Malang is also infamous for it's hardcore football supporters, evidence of which is hard to miss around town.
Between Surabaya & Malang, but closer to Malang are the Purwodadi Botanical Gardens. A division of the Bogor Botanic Gardens, one of the worlds great botanic gardens, these are well worth an excursion. Malang is also in the centre of several surrounding small historic Hindu temple sites.
The final reason for stopping in Malang is that it makes a great alternative route to visit Mount Bromo. While a bit further than the more common route to Bromo via Probollingo, this way takes you past high altitude grass plains, multiple twisting valleys, and some other major volcanoes in the Bromo area, before depositing you in the moonscape of the Bromo caldera. Given the state of the roads, a four wheel drive is a necessity but a great day out.
Blitar
Blitar is a small, very pleasant city, about 2 hours by train to the south west of Malang. It has a few picturesque colonial buildings, including another Tugu Hotel in an old mansion, but inside town the most well known landmark is the tomb of the late President Sukarno. An imposing modern complex of buildings, this is notable for the way it manages to reflect those twin intertwined traits of Indonesia - Islam & mysticism, with many people making pilgrimages to his graveside.
Blitar is also a good base for visiting a string of ancient Hindu sites encircling it. Penataran Temple is the largest of these, set in manicured gardens.
Nature wise, Blitar is also a good base for day trips to beaches to the south, or an excursion to Mount Kelud. Kelud is one of Indonesia's most explosive volcanoes - when it goes it really pops it's top. It's 2014 eruption managed to close Borobudur 200 kms to the west, not to mention 76,000 people evacuated and the closure of all airports in Central & East Java. In consequence it's a pretty amazing shattered & twisted landscape to visit, with grand views across the coastal plains to the south as a bonus.
Sumatra
A quick note on Sumatra, the third largest island. Renowned for it's beauty, it's jungles and wildlife, Sumatra is home to quite distinct cultures as you traverse it's length. From the conservative Muslim area of Banda Aceh in the north, to the animist hold outs around Lake Toba, on to the matriarchal Minangkabau around Bukittinggi, with their elegant "buffalo horned "buildings, and on to the foodie feast that is Padang, there's an enormous variation.
The down side is that transport & travel in Sumatra is still difficult & very time consuming, especially if you're trying to avoid unnecessary internal flying. It's been many years since I visited Sumatra, so tales of buses catching fire, and roadside knife fights may no longer be relevant. But it's a world away from people's Bali stories.