Footage from the event shows the Prince of Wales talking to one dignitary before moving straight past Mr Pence and his wife Karen.
He appears to blank the couple as he stops to talk to the Israeli Prime Minister Benjamin Netanyahu standing on their other side.
Social media users were quick to comment on the footage with some applauding Prince Charles for snubbing Mr Pence and others branding it "disgraceful".
However, both Prince Charles and Mr Pence's teams were quick to deny that the future king was displaying animosity towards the vice president.
Pence got snubbed by Prince Charles!! https://t.co/w6IUfAp63y— Paul Aningat (@PaulAningat)January 23, 2020
Clarence House said Charles did not ignore President Donald Trump's deputy but had a "warm and friendly" chat earlier.
It said: "Shortly before the Yad Vashem memorial event began the prince and Vice President Pence and his wife had a warm and friendly chat, which is why they did not greet each other again in the room."
Mr Pence's press secretary Katie Waldman shared a photo and a video of the two meeting earlier.
— Katie Waldman (@VPPressSec)January 23, 2020
She wrote: "This is not true. VP and Prince Charles spoke prior to entering the event floor and after his remarks as well."
During his speech at the event, the Prince of Wales urged nations to learn the "lessons" of the Holocaust as he joined world leaders in condemning the scourge of anti-Semitism.
Charles described the extermination of six million Jewish people during the Second World War as a "universal human tragedy" affecting all, not just the families of those killed by the Nazi regime.
— The Reality Report (@WeAreTRR)January 23, 2020
The heir to the throne told guests, who included Russian President Vladimir Putin and France's President Emmanuel Macron, the story of "incomprehensible humanity" must not be forgotten.
And he warned that "hatred and intolerance still lurk in the human heart" but society must remain "resolute in resisting words and acts of violence".