Oxford English Dictionnary.

Cook and Banks believed it to be the name given to the animal by the natives at Endeavour River, Queensland, and there is later affirmation of its use elsewhere :

⇒ 1770 Cook jrnl. (1893) 224 (Morris) (Aug. 4) The animals which I have before mentioned, called by the Natives Kangooroo or Kanguru. 1770 J. Banks Jrnl. (1896) 301 (Aug. 26) The largest [quadruped] was called by the natives kangooroo. 1773 J. hawkesworth Voy. III. 578 (1st Voy. Cook) The next day our Kangaroo was dressed for dinner and proved most excellent meat. 1774 Goldsm. Nat. Hist. VII. xvi. II. 434 The kanguroo of New Holland, where only it is to be found, is often known to weigh above 60 pounds. 1787 Anderson in Cook's Voy. (1790) IV. 1295 We found, that the animal called kangooroo, at Endeavour River, was known under the same name here [in Tasmania]. 1796 Gentl. Mag. LXVI. I. 467 The Gamgarou, or as Pennant calls it Kangaroo, is a native of New South Wales. 1835 T. B. Wilson Narr. Voy. World. They [natives of the Darling Range, W.A.] distinctly pronounced 'kangaroo' without having heard any of us utter the sound. 1845 Darwin Voy. Nat. xix. (1852) 441 Now the emu is banished to a long distance and the kangaroo is become scarce.

On the other hand, there are express statements to the contrary, showing that the word, if ever current in this sense, was merely local, or had become obsolete :

⇒ 1792 J. Hunter, "Port Jackson" (1793) 54 The animal called the kangaroo (but by the natives patagorong) we found in great numbers. 1793 W. Tench Compl. Acc. Port Jackson 171 The large, or grey kanguroo, to which the natives [of Port Jackson] give the name of Pat-ag-a-ran. Note, Kanguroo was a name unknown to them for any animal, until we introduced it. 1834 Threlkeld Austral. Gram. (Hunter's River) 87 (Morris) Kóng-go-róng, the Emu, likely the origin of the barbarism, kangaroo, used by the English, as the name of an animal called Mo-a-ne. 1850 Jrnl. Ind. Archipelago IV. 188 Kangaroo: it is very remarkable that this word, supposed to be Australian, is not to be found as the name of this singular marsupial animal in any language of Australia. I have this on the authority of my friend Captain King.

The common assertion that it really means 'I don't understand' (the supposed reply of the native to his questioner) seems to be of recent origin and lacks confirmation. (See Morris Austral English s.v.)

The American Heritage Dictionary of the English Language [page]

From Gaurru in Guugu Yimidhirr (Aboriginal Language Of Northeast Australia). A widely-held belief has it that the word kangaroo comes from an Australian Aboriginal word meaning "I don't know." This is in fact untrue. The word was first recorded in 1770 by Captain James Cook, when he landed to make repairs along the northeast coast of Australia. In 1820, one Captain Phillip K. King recorded a different word for the animal, written "mee-nuah". As a result, it was assumed that Captain Cook had been mistaken, and the myth grew up that what he had heard was a word meaning "I don't know" (presumably as the answer to a question in English that had not been understood). Recent linguistic fieldwork, however, has confirmed the existence of a word gangurru in the northeast Aboriginal language of Guugu Yimidhirr, referring to a species of kangaroo. What Captain King heard may have been their word minha, meaning "edible animal."

R.M.W. Dixon (1980) The Languages of Australia, Cambridge

In 1898 the pioneer ethnologist W.E. Roth wrote a letter to the Australasian pointing out that gang-oo-roo did mean 'kangaroo' in Guugu Yimidhirr, but this newspaper correspondence went unnoticed by lexicographers. Finally the observations of Cook and Roth were confirmed when in 1972 the anthropologist John Haviland began intensive study of Guugu Yimidhirr and again recorded gaNurru [large black kangaroo].