Is the ground for the voltage rail and input signal the same?
Yep.
What exactly is wrong with the circuit I built? I want the LED to only turn on when 5V is supplied at the input, right now the LED can turn on if I connect the ground to the voltage rail supply even without an input voltage.
Schematic of exactly what you did… not what was on paper, how it is on the breadboard.
I’ve seen the post on Adafruit with the feedback resistors connected to the same ground as the rail supply, but the circuit diagram does not show where the input voltage ground is? Link: blog.adafruit.com/…/ask-an-educator-making-a-non-…