Brown on impact losing Bye can have on Patriots