The Swans hold a 2-0 lead going into the second leg thanks to goals from Michu and Danny Graham.
Given their excellent home record since securing promotion to the top flight, Swansea have arguably done the hard part when it comes to reaching the club's first major domestic cup final.
But Chelsea have shown they have the ability to score the goals that could yet turn the tie on its head. The Blues put eight past Aston Villa over Christmas and earlier this month racked up five against Southampton and four against Stoke.
"It's such a big game for us as individuals and as a club. We went up there, did really well and we have set ourselves up nicely," said Williams.
"But we know the job is not done and it will be as tough here as it was up there. They will come at us and we will be ready for that."
Despite grabbing the second at Stamford Bridge as part of a run of four goals in six appearances, Graham could be on his way out of Swansea before the transfer window closes.
Swansea, who are themselves keen to secure the services of Stoke's Kenwyne Jones, have received bids from Sunderland and Norwich for Graham, and he is thought to prefer the chance to return to his native north-east.
The home crowd pleaded with Graham to stay during his second-half cameo appearance in the 3-1 win over the Potters on Saturday, and Williams has revealed his teammates are doing the same.
"We've just been saying to him that we want him to stay, saying 'don't go', said the centre-half.
"We all want him to stay, he is a top player and has done well here. We definitely don't want to see him leave."
Follow us @SprtNationalUAE