This 4th-century structure houses an important branch of the Celje Regional Museum. In the cellar is the City under the City exhibition: partial remains of Roman Celeia's 3rd-century decumanus maximus (main west–east road) brilliantly displayed with excavated statuary, Roman villa ruins, parts of the old city walls and traces of the medieval buildings built on top, all aided by thorough explanation panels. On the 1st floor you'll find beautifully curated displays of neolithic, Bronze Age and Iron Age finds.
Half of the 1st floor and the entire 2nd floor are devoted to (rather sparse) exhibits tracing the history of the Counts of Celje. One room houses a display of 18 of the noble skulls spookily set out in back-lit cases.
The southern wing of the building is home to the Gallery of Contemporary Art, with thematic exhibitions of work by both local and foreign artists.