People sang Amazing Grace as they lit candles for Ms Clarke and her children – Laianah, Aaliyah, and Trey – who were murdered by her former partner Rowan Baxter.
Prime Minister Scott Morrison said it was important to remember their victims and who they were.
“All of that was taken from them in a murderous act of violence which none of us here can comprehend,” he said.
Labor leader Anthony Albanese hoped the brutal murder marked a turning point.
“We have all failed, particularly men have failed, the women and children of this country,” he said.
Greens co-deputy leader Larissa Waters said words were not enough and parliament needed to take action.
“We here collectively can fix the system and make sure that it doesn’t fail anybody else,” Ms Waters said.
The vigil comes as some MPs stress the need for domestic violence to be on the agenda constantly, not just in the aftermath of major tragedies.
Earlier on Wednesday, Labor MP Anne Aly, said it was crucial the issue continues being tackled long after Ms Clarke’s death.
“I want to make sure that this stays on the agenda – that we don’t just talk about this at that critical point where we are mourning lives lost,” Dr Aly told reporters on Wednesday.
She said there are a lot of women who will be wondering if they’re “going to be beaten black and blue tonight”.
“I say to those women, we see you. And we know you,” she said.
On Tuesday, Dr Aly spoke publicly for the first time about her experience of domestic violence at the hands of her former partner.
Fellow Labor MP Linda Burney, who led Wednesday’s ceremony, said what women needed when they were escaping a domestic violence situation was certainty and practical outcomes.
“They need a place that is safe to go to. They need to be financially secure. They need to understand that there is support out there,” she said earlier on Wednesday.