Police officer Vijay Singh said seven fishermen had been arrested for facilitating the American’s visit to North Sentinel Island, where the killing apparently occurred.
Visits to the island are heavily restricted by the government.
The Sentinelese people live on the small forested island. They are known to resist all contact with outsiders, often attacking anyone who comes near.
North Sentinel is in the Andaman Islands – a group of islands at the juncture of the Bay of Bengal and Andaman Sea.
Indian media reports said the American was on an adventure trip to the islands and his body had been spotted by the fishermen.
Mr Singh said police were in the process of recovering the body.
He said the man was killed on Saturday and he was investigating the case.
Local media say he was a missionary. He has been identified as John Allen Chau.
Kathleen Hosie, spokeswoman for the US Consulate in Chennai, the capital of southern Tamil Nadu state, said it was aware of reports concerning an American in the islands.
“When a US citizen is missing, we work closely with local authorities as they carry out their search efforts,” Ms Hosie said.
She said she could not comment further due to privacy considerations.
Survival International, an organisation that works for the rights of tribal people, said the killing of the American should prompt Indian authorities to properly protect the lands of the Sentinelese and other Andaman tribes.
“The British colonial occupation of the Andaman Islands decimated the tribes living there, wiping out thousands of tribespeople and only a fraction of the original population now survives,” Stephen Corry, the group’s director, said in a statement.
“So the Sentinelese fear of outsiders is very understandable.”
Shiv Viswanathan, a social scientist and a professor at Jindal Global Law School, said North Sentinel Island was a protected area and not open to tourists.
“The exact population of the tribe is not known, but it is declining,” he said.
“The government has to protect them.”
Poachers are known to fish illegally in the waters around the island, catching turtles and diving for lobsters and sea cucumbers.
Tribespeople killed two Indian fishermen in 2006 when their boat broke loose and drifted onto the shore.