Arrange a rack in center of oven; preheat to 350°. Grease a 2.5-qt. (or a 9" x 9" square) casserole dish.
In a large saucepan over medium heat, melt butter. Cook shallots and garlic, stirring occasionally, until softened, 3 to 5 minutes.
Add flour and cook, stirring, until vegetables are coated, 1 to 2 minutes. Add milk, half-and-half, salt, and pepper and bring to a simmer over low heat.
Cook, stirring frequently, until slightly reduced, 3 to 5 minutes. Add 3 c. cheese and stir until melted and combined.
Place broccoli florets in prepared dish. Pour milk mixture over broccoli.
In a small bowl, combine panko and remaining 1 c. cheese. Top with panko mixture.
Cover dish with foil or a lid and bake 20 minutes. Uncover and continue to bake until top is golden brown and crispy, 10 to 15 minutes more.