A 4 inch barrel develops less muzzle velocity than a 5 inch barrel does. That changes the ballistics of the round as it flies.
The longer barrel also impacts your sight picture, and it's stability in hand.
The different muzzle velocities, impact of barrel length on bullet weight and shape, powder burn rate and other details all combine in a complex way.
Sometimes experiment is the best way to determine this kind of thing. Note that reloading manuals often publish the most accurate load for a given bullet, but it's generally shot from a five inch instrumented test barrel.
Will it matter in a practical sense? Consider the typical self defense distance. If you're a bullseye shooter, it's another thing completely.