it depends. Those foreign to the US are likely to say “yanks” to mean all Americans, but Americans may find it slightly odd to refer to Southerners as Yankees.
In the same way as calling a Catholic from Belfast “British” - it’s true but it could feel a little awkward.
“Yankees” was the unofficial term of the people of the Union (who won the war and became the united states) vs the Confederates (“Rebs”) who supported slavery