The building was designed in the neoclassical style, along the lines of Porta Cristina and of the Royal military arsenal's main entrance, these also designed by the Marquis of Putifigari.

The building consists of a marble parapet adorned by four statues representing the Seasons, in the central part the gentle coat of arms of the House is carved. Below the inscription that reminds the year when the building was erected: "COMES KAROLUS PILO BOYL EX MARCHIO PUTIFIGARI INSTAURAVIT ANNO MDCCCXL".

A Pisan tower, which served as access to Castello district and was contemporary with those of San Pancrazio and Elephant, was absorbed by the building. The tower suffered three devastating attacks: the British in 1708, the Spanish siege of 1717, when it lost its upper part and the French attack in 1793. Three cannonballs stuck on the façade of the building are a vivid testimony of the three attacks suffered. Beyond the gate, on the left side, you can see a Roman funeral stele, which had been walled up in the tower along with other inscriptions. At the end of the nineteenth century, the palace was acquired by the Barons Rossi's family. The current owners are the Earls from Marche, Tomassini Barbarossa.