To me it’s like eating your least favorite candy in a mixed bag first - you’re left with the best part last. You have to leave just a bit of it so you can still grab it.
E: Also, if you eat all the crusts last they usually go dry and hard instead of nice and warm when you eat them first or as you go along. Depends on whether the dough is good really.