The renowned choreographer sat on the judging panel of the hit celebrity dance show when it launched in 2004.
She was replaced in 2009 by former winner Alesha Dixon, who is 35 years her junior, in a move that sparked a public outcry over alleged BBC ageism.
Now Phillips, 78, has revealed to the Celebrity Search Engine podcast that she was initially hesitant to take part in the show at all.
Watch: Strictly's Anton Du Beke opens up about having children in his 50s
She told host Abbie McCarty: "I said no three times.
"So I went to do this pilot, and Len Goodman was there, there was another kind of judge stand-in person, and there were some of the dancers we know, Anton [du Beke] was there, Brendan [Cole] was there… it felt like chaos!
"Then Len turned to me and he said, 'I don't know about this, are you going to do it?', and I said, 'I don't know, are you?' and he said, 'I'll do it if you do it!'
"And then we both agreed to do it!"
Goodman, 77, departed in 2016, and original judges Craig Revel Horwood and Bruno Tonioli currently sit alongside choreographer dance teacher Shirley Ballas, 60, and dancer Motsi Mabuse, 40.
Phillips said the show "was one of the most exciting things I've done in my life".
But the choreographer – who has worked on West End shows and films including the 1982 musical movie production of Annie – last year called the BBC's handling of her replacement in 2009 "shocking".
She revealed she had obtained the internal BBC report into her being dropped from Strictly, under the Freedom of Information Act.
Phillips said: "I’ve got the Freedom of Information and looking at the chaotic way that led up to it was shocking.
"Most is redacted. There’s some information there and even that’s chaotic, let alone what was redacted from the report.
"Nowadays, they could have given me the chance to resign or build a story around it.
"I didn’t have a contract that automatically rolled on to the next year, but I think it was handled so poorly.
"You then never forget those moments of turmoil."
Watch: Arlene Phillips discuss getting vaccinated against COVID-19