Britain has "lost control" of its borders over the last five years, the defence secretary told Sky News after hundreds of migrants crossed the Channel.