The Gunners saw off West Ham 1-0 on Tuesday, while Chelsea needed a late Alvaro Morata goal to defeat Bournemouth 2-1 at Stamford Bridge.
Manchester City will face Bristol City, who dumped out Manchester United.
At the semi-final stage, the two ties are played over two legs.
The first games will be played on the week beginning January 8, with the reverse fixtures taking place two weeks later.
Arsenal and Bristol City will have the home advantage in the second legs.