I have seen so much BS about Ninjas and Ninjutsu that I decided it was time to put my two cents in. Actually, I think I have a great deal more than two cents to add, given the fact that I studied Ninjutsu intensively for 7 years (tested 2nd dan), have read most of the commercially available materials from the major authors in this field in the martial arts community, and have spent the last 25 years living in Japan, the birthplace of ninja culture.
At the risk of controversy from martial artists who may believe whatever they want to in the face of actual evidence, here's my take on a few of the common discussion topics. These views are my own and I take sole responsibility for them.
Who were Ninja?
We all want to believe the fantasies about black-clad assassins jumping from rooftop to rooftop using their superhuman skills to achieve the impossible. Perhaps the most impossible thing they achieved was an over-inflated sense of grandeur about the whole thing. While it is difficult to dispute the historical evidence that mercenary groups existed who fulfilled some aspects of the roles ascribed to ninjas (assassin, spy, informant, bodyguard) there is not much evidence to suspect that this was an orderly, controlled affair. The historical documents of "Yamabushi' or mountain warriors blended with Shinto mysticism and martial arts are very likely to be highly exaggerated and a majority of so-called "ninja" were nothing more than villains/thugs for hire to the highest bidder, without the counterculture anti-samurai bushido that is accorded to them in movies.
Yet another strong possibility is that Ninja were forerunners of organized crime groups (yakuza), who were used to help keep social order during times of unrest, as Tokugawa Ieyasu used them during his reign. It was useful to have groups from outside the capital who would not be subject to the influence of the politics surrounding the shogun, but who could blend in when needed and provide valuable intelligence on the ground.
Socially, such groups helped to maintain the social fabric in Japan (and still do), allowing justice to be done and/or grievances settled when the legal system is unable to do so properly or to the satisfaction of those involved. While gambling and other gray acts were the hallmark of Japanese organized crime syndicates, there is nothing to say that these groups were not "ninjas", or worked in collaboration with other mercenary groups who might be called "ninjas".
To confuse matters more, some traditional "samurai arts" such as Yagyu Shinkage Ryu include "ninjutsu" as a sub-system in their study of "heiho" (strategy), much in the same way that clandestine operations and subterfuge are part of our modern military hierarchy.
The 1980's vision of "black ninja versus white ninja" and all the various Sho Kosugi/Franco Nero/Lee Van Cleef entertainment stemming from it added popularity and mystique to the world of the Ninja - my first experience being in Chuck Norris' "Octagon" (1980) which to be fair was actually better than a lot of other movies which came later.
Dr. Masaaki Hatsumi and the Bujinkan
At least as of this writing, Dr. Hatsumi still lives in Noda City, Chiba Prefecture, and continues to have his bone-setter practice in addition to teaching Togakure-Ryu Ninjutsu as Head of The Bujinkan, his global organization. There is no evidence to suggest he is anything other than authentic, and he continues to appear on Japanese TV from time to time demonstrating Ninjutsu for various information programs. The boom seems to have come during the 1980s when Stephen K. Hayes, an American karate practitioner from Ohio went to Japan and asked to be his live-in disciple. Hayes became the foremost Western authority on Ninjutsu and went on to publish many books on Ninjutsu during the 1990s, which I read extensively when I was still training with my teacher. He also served as Dr. Hatsumi's translator and I believe he was a fundamental part of the globalization of the Bujinkan. Later he would go on to study Tibetan Buddhism as well as advise for TV and movie programs and various government agencies.
There were a number of other Bujinkan luminaries (Shoto Tanemura, Doron Navon, Jack Hoban, etc.) who came and went from Noda City, and later founded Bujinkan chapters around the world. These seemed to be especially popular in the US, Germany, Israel and Australia.
Although the most famous lineage, Bujinkan was not the only school or system promoted in the 1990s during the Ninja Boom. Ron Duncan popularized Koga Ryu Ninjutsu in the 1970s and 1980s and his work seems at least as credible as Dr. Hatsumi's, since it has proven difficult to verify the claims of any of the schools to a lengthy lineage beyond the current generation.
Ashida Kim was also well-known by researchers (although his work seems a bit more fantasy than reality) and published a number of books on Citadel Press.
In summary, esoteric and exotic sells. Ninja have been made out to be everything from secretly trained mercenary assassins to deeply spiritual warrior monks. There seem to be many versions of the truth, depending on who is telling the story.
Ninjutsu Fighting Methods
Fighting techniques covered in Ninjutsu include both traditional Japanese empty hand and weapon arts. The empty hand arts might most closely resemble Japanese Kempo, including fluid striking/kicking and locking/throwing systems. Weapon arts include traditional Japanese weapons such as jo and sword (although the katana is uncommon), and some schools teach spear (yari) and halberd (naginata) as well. Kobudo weapons such as bo, kama, sai and nunchaku also appear, although these are of Okinawan rather than Japanese origin. Despite the dominance of Japanese archery (Kyudo) in samurai culture, there doesn't seem to be a precedent for such training in Ninjutsu.
The Ninja Star or throwing star (shuriken) is probably the most symbolic of all ninjutsu weapons and ironically probably the least practical of any of them. Use of a straight throwing spike has a traditional precedent in old sword schools where the kozuka was sometimes thrown as a distraction when combatants entered fighting range. Many traditional Ninjutsu schools still teach throwing this straight spike rather than the commonly assumed flat, spiked disk.
Another favorite in the media was the Kumade or bear claw, which is a set of claws attached to the hands or fingers and used to scale walls (supposedly) as well as for hand-to-hand combat. Aside from sales of such items to teenage fanboys, it is unlikely that such tools were a major component of the Ninjutsu practitioner's arsenal and I dare anyone to go free climbing in them. I have seen and heard speculation of a wide variety of exotic "Ninja" weapons, from blowguns to crossbows and everything in between. I don't personally consider them more than curiosities.
In swordsmanship, since this was not the primary art of Ninjutsu practitioners but one of many other training disciplines, face-to-face combat with trained swordsman was generally avoided in favor of angled attacks to the wrists/arms/legs of opponents and group attacks on single opponents were certainly preferred where possible. Some schools would mount short swords with two-handed katana handles to deliver more cutting power at close ranges. Ninjutsu sword techniques also include stabbing attacks far more than traditional sword styles, which emphasize cutting.
While movies portray Ninja as masters of disguise and deception, with skills like invisibility, water-walking, poison, and the like, the reality is that this was highly unlikely. Furthermore, there is little evidence to suggest the black suits and back-mounted scabbards have any basis in historical fact either. Imagine my surprise when I discovered that tabi, the black goat-footed shoes that ninja wear in movies, are worn by most construction workers and handymen and sold at the DIY store in the mall (so are kama, by the way, look in the gardening aisle near the shovels).
While there is plenty of controversy as to whether or not Ninjas actually existed, if they did they are unlikely to have self-identified themselves as such. Moreover, there would have been many different interpretations of what constituted Ninjutsu practice. I am extremely skeptical of the media portrayal of Ninjutsu, which bore little resemblance to what my teacher taught me.
Perhaps it is all best left in the shadows after all.