The three “opened the door” to an unknown world where matter takes unusual states or phases, the Royal Swedish Academy of Sciences said.
They were awarded for their “theoretical discoveries of topological phase transitions and topological phases of matter”.
Thouless, 82, is a professor emeritus at the University of Washington, Haldane, 65, is a physics professor at Princeton University in New Jersey, and Kosterlitz, 73, is a physics professor at Brown University in Providence, Rhode Island.
Their research was conducted in the 1970s and 80s. Nobel judges often award discoveries made decades ago, to make sure they withstand the test of time.
This year’s Nobel Prize announcements started on Monday with the medicine award going to Japanese biologist Yoshinori Ohsumi for discoveries on autophagy, the process by which a cell breaks down and recycles content.
The chemistry prize will be announced on Wednesday and the Nobel Peace Prize on Friday. The economics and literature awards will be announced next week.
Each prize has a purse £730,000. The winners also collect a medal and a diploma at the award ceremonies on December 10, the anniversary of prize founder Alfred Nobel’s death in 1896.
DOWNLOAD THE SCOTSMAN APP ON ITUNES OR GOOGLE PLAY