Rohan Silva, who was David Cameron’s key aide on the tech economy, claims that the “offensive” incident took place in 2012 during a discussion about trade policy.
Mr Silva said he asked the Queen’s second son whether the government department responsible for trade “could be doing a better job”. He said the Duke of York responded: “Well, If you’ll pardon the expression, that really is the n***** in the woodpile.”
Palace sources categorically denied that Andrew ever used the phrase, insisting he would never use such language.
But Mr Silva, 38, now a co-working space entrepreneur and an Evening Standard columnist, said he clearly recalls “how I walked blinking into the sunshine outside Buckingham Palace, reeling at the prince’s use of language” after the meeting attended only by himself, Andrew and a palace aide.
He added: “For a long time afterwards I kicked myself for not confronting the prince on his choice of words — and it’s something I still regret today. After all, he clearly wasn’t taken to task very often by the people around him, which meant offensive language went unchallenged.”
The claim will raise new questions about Prince Andrew’s judgment in the midst of the firestorm over his “car crash” interview with BBC Newsnight about his links to paedophile Jeffrey Epstein. The N-word is among the most insulting of racist slurs because of its association with slavery and oppression of black people in America.
The “n***** in the woodpile” originated as a figure of speech in the Deep South in the 19th Century and was used to describe fugitive slaves who hid in piles of logs as they fled north to Canada.
It was once widely used to refer to a hidden problem or threat, but is now regarded as wholly unacceptable.
In 2017 the Tory MP Anne Marie Morris was suspended for using the same phrase in a meeting about Brexit. Theresa May, who was prime minister at the time, condemned her words as “shocking and unacceptable”.
In 2008, Conservative peer Lord Dixon-Smith used the phrase in the House of Lords but escaped censure after a series of apologies.
Mr Silva, whose parents are both Sri Lankan but who was born and brought up in Wakefield, said it was the second time that the prince had used unacceptable language in his presence.
At a Buckingham Palace meeting the previous year, Andrew and Mr Silva had been discussing European Union reform. Mr Silva claimed Andrew said: “What you have got to remember is that you’ll never get anywhere by playing the white man.”
Mr Silva added: “I genuinely didn’t know what he meant, and the discussion moved on. But the phrase ‘playing the white man’ stuck in my head, as I’d never heard it before. So when I got back to my desk, I immediately googled it.
“The definition flashed up on my screen: an old-fashioned saying, used during colonial times, meaning that only white people can be trusted to follow the rules, unlike dark-skinned natives.”