Most films say in the closing credits what music was used in the film, but this is one of the few films I've seen where they say what the paintings were.
What were the paintings?
http://www.spanisharts.com/reinasofia/reinasofia.htm
This is the museum the fellow visits anyway. The readymade with the sheet is Antoni Tapies.
This is what you are looking for:
http://limitedcontrol.posterous.com/a-cultural-glossary-to-the-limits-of-control
where it says
"...For Jarmusch, the paintings the Lone Man sees at the Madrid art museum Museo Reina Sofia – El Violin (1916) by Juan Gris, Desnudo (1922) by Roberto Fernández Balbuena, Madrid Desde Capitán Haya (1987 – 1994) by Antonio López and Gran Sábana (1968) by Antoni Tapies..."