This gets complicated. The small heater is almost certain to be resistive electric heat. Your central heat could be anything. In my area, the gas furnace is usually cheaper to run for the entire house than it is for a space heater in a single room.
OTOH, if you have a resistive electric furnace, your advice is spot on