The 59-year-old Leicester East MP’s resignation follows newspaper reports that he met the escorts at a flat he owns near his family home in north London in August.
Mr Vaz also offered to pay for a Class A drug and discussed using the party drug “poppers”, according to the Sunday Mirror.
The cross-party committee is considering a review of prostitution laws. Mr Vaz also opposed government attempts to criminalise poppers.
Mr Vaz, who has publicly apologised to his wife and children for the “hurt and distress” caused, was under pressure to quit. Committee members had threatened a vote of no confidence in him if he did not resign.
The Labour MP, who emerged smiling as he left his London home earlier, said in a statement: “Those who hold others to account must themselves be accountable.”
He said it was in the “best interest” of the committee for it to conduct its business “without any distractions”.
“This is my decision, and mine alone, and my first consideration has been the effect of recent events on my family,” he said.
On his way to meet senior committee members behind closed doors, he was asked by Sky’s Robert Nisbet whether he would be prepared to talk the media afterwards.
Nisbet said Mr Vaz replied: “Why would I want to speak to you when you say such nice things about me?” and seemed relaxed.
Senior Conservative MP Tim Loughton has been appointed interim chairman until a successor is elected, probably in October.
He said Mr Vaz gave a “very frank” account of what had happened and would no longer be on the committee.
Speaking to reporters at Portcullis House, the Conservative MP said: “The committee listened, I think, in sadness to what Keith had to say and with a good deal of respect.
“Keith has clearly acted in the best interests of the Home Affairs Select Committee and the important work that we do.
“With sadness we all accepted that that was the appropriate course of action that he has taken and we also appreciate the many challenges facing him personally and his family.”
Members applauded Mr Vaz as he left the room afterwards.
Mr Vaz still faces a possible investigation by the Commons’ sleaze watchdog over the allegations.
Labour leader Jeremy Corbyn has indicated there is no need for Mr Vaz to stand down from the party.
A decision on his future on the party’s ruling National Executive will be discussed and “partly made” by the MP himself, Mr Corbyn said.
Former shadow business secretary Chuka Umunna has been tipped to take over as chairman of the Home Affairs Select committee, given that he is the ranking Labour member.
Speaking to Sky News’ Adam Boulton, the MP said: “I don’t think today is the right day to be putting myself out there as a future chair of the committee.
“I will make my intentions known in due course.
“But it would be incredibly disrespectful towards Keith on the day of his resignation to start manoeuvring.”
Along with the sex allegations made by the Sunday Mirror against Mr Vaz, the paper also claimed money was paid into an account used by one of the escorts by a man linked to a charity set up by the MP.
There is no suggestion the payment was made by the charity or that its money was involved.
[Source:-Sky news]