The Swedish Academy praised Gurnah for for his "uncompromising and compassionate penetration of the effects of colonialism".
The prize is awarded by the Swedish Academy and is worth 10 million Swedish crowns ($1.14m / £840,000).
Gurnah, 73, is the author of 10 novels, including Paradise and Desertion.
He said how grateful he was to the academy, adding: "It's just great - its just a big prize, and such a huge list of wonderful writers - I am still taking it in.
"It was such a complete surprise that I really had to wait until I heard it announced before I could believe it."
Paradise, published in 1994, told the story of a boy growing up in Tanzania in the early 20th Century and was nominated for the Booker Prize, marking his breakthrough as a novelist.
"Abdulrazak Gurnah's dedication to truth and his aversion to simplification are striking," the Nobel Committee for Literature said in a statement.
"His novels recoil from stereotypical descriptions and open our gaze to a culturally diversified East Africa unfamiliar to many in other parts of the world."
"[His] characters find themselves in a hiatus between cultures and continents, between a life that was and a life emerging; it is an insecure state that can never be resolved."
Born in Zanzibar in 1948, Gurnah arrived in England as a refugee in the late 1960s.
He was Professor of English and Postcolonial Literatures at the University of Kent, Canterbury, until his recent retirement.
Gurnah is the first black African author to have won the award since Wole Soyinka in 1986.
He said his award would mean ssues such as the refugee crisis and colonialism, which he has experienced, will be "discussed".
"These are things that are with us every day. People are dying, people are being hurt around the world - we must deal with these issues in the most kind way," he said.
"I came to England when these words, such as asylum-seeker, were not quite the same - more people are struggling and running from terror states.
"The world is much more violent than it was in the 1960s, so there is now greater pressure on the countries that are safe, they inevitably draw more people."
In an interview in 2016, when asked if he would call himself an "author of postcolonial and/or world literature", Gurnah replied: "I would not use any of those words. I wouldn't call myself a something writer of any kind.
"In fact, I am not sure that I would call myself anything apart from my name. I guess, if somebody challenges me, that would be another way of saying, 'Are you a... one of these...?' I would probably say 'no'. Precisely, I don't want that part of me having a reductive name."
The Nobel Prizes, which have been awarded since 1901, recognise achievement in literature, science, peace and latterly economics.
Past winners have included novelists such as Ernest Hemingway, Gabriel Garcia Marquez and Toni Morrison, poets such as Pablo Neruda, Joseph Brodsky and Rabindranath Tagore, and playwrights including Harold Pinter and Eugene O'Neill.
Former UK Prime Minister Winston Churchill won for his memoirs, Bertrand Russell for his philosophy and Bob Dylan for his lyrics.
Last year's award was won by American poet Louise Gluck.
