To be honest if I was in a battle against the Navy and they were all wearing pink triangles and rainbows I wouldn't feel at all threatened and would have no intent to turn myself in. I also think that people wouldn't want to be in the Navy if those were the uniform.
Do you know what symbolism is? Symbols - any symbol - means what it means to the present culture. Sure, you can talk about the origins of the swastika and what it used to mean.
Simply put, in Western Culture, it no longer represents well being, good fortune, or whatever else you might care to look up on the big wide internet.
Show anyone in the Western World a swastika, and what does it represent?
Vioence. Opression. War. The worst genocide in world history. Nazis.
And, as Beff pointed out, White Supremacists, and Neo-Nazis.
Why do you think Prince Henry met with such a strong reaction when he wore a German military uniform, complete with swastika, to a costume ball?
Do not think for one moment that casual display of a swastika is "not that big of a deal".
Now, on to the deeper paranoia; the question begged is "Why?".
Throughout the colonial United States there are hundreds if not thousands of buildings which have, by intent or design, encompassed some very varied archetecture. Europe is rife with elaborate design formulas...
Google makes it all available with a birds eye view.
Perhaps an addition to an existing structure, without an overhead view beforehand? Poorly planned from an aethstitic point of view, but probably functional. And the armed forces are all about functional.
even if it was just an addition, the whole structure would have to have revised blueprints to tell the work crews how to build it in relation to the older buildings. and those include an overhead view. and there is no way in hell it was designed to be functional, theres to much landscaping for it to be functional
And I'm sure it hardly looks like a Swastika from the ground, but that's not the point is it? From the underside of a pachyderm... you'd hardly guess an Elephant.