It was assumed (though not, unfortunately, set down in writing) that when the individual colonies were voting on whether to ratify the Constitution, they were voting on joining an entirely VOLUNTARY association--NOT starting down a "one-way" street from which there was no exit! If it had been understood that "once in, you can't leave," the larger colonies, esp. VA, would NEVER have bound themselves in a "indissoluble" union!... all the southern states were trying to do was to leave the union.
During the War of 1812, the New England states (which hated "Madison's War") planned a convention to organize for secession, but the war ended before it could be carried out; however, it was never at the time argued that they had no legal right to leave the union they had joined voluntarily.