Pass Factor

I had a good question come in about my references to lower tier schools, and the question asked me to reflect on what were the top tier schools. You can find in Nagayama’s The Connoisseur’s Book of Japanese Swords good listings of the Leading Schools for each time period. I think every collector should have this book. It was out of print for a while and prices went way up, but it is back in print now and you can buy it following that Amazon link (which does not make me money, just get this book and use it).

Trying to get a handle on which schools are the best actually seems easy at first but it gets a little bit complicated the deeper you dig.

Anecdotally

I will riff this off the top of my head for what I personally think are the first rank schools. This is mostly personal opinion so don’t be offended if I didn’t include something you hold dear or an author holds dear.

Yamashiro: Sanjo, Gojo, Awataguchi, Rai, Ayanokoji.

Bizen: Ko-Bizen, Ichimonji, Aoe, Osafune (Kamakura/Nanbokucho).

Soshu: Shintogo mon, Masamune mon (includes Kaneuji). 

Yamato: Hosho.

Mino: none.

Shinto is more difficult to interpret by means of schools. There are outstanding smiths in many schools and the skill may peak with that smith and fall off before and after him. I would include Hizen, Umetada, Horikawa, as the first rank of schools. I am not so qualified though to even be talking about Shinto in this regard as I haven’t studied it enough.

Shinshinto: none.

Maybe it is a bit harsh but it’s supposed to separate the ultimate from the extremely good.  

Some might give me hell for not including something like Senjuin in the first rank. The reason I would exclude Senjuin is that it is extremely broad over a very long time with very uneven quality of work. Top quality Senjuin can hold its own with any of the other top ranked schools, but most Senjuin is not of this quality and a lot is a lot further down the hill. So to me it would be Senjuin as a whole in the second tier. Trying to nail down the exact nature of the second tier, going through everyone, is going to be more of an exercise. I have found a lot of Yamato to kind of follow this model of some standouts and some lackadaisical followups. But Hosho as a school, to me they have always been exquisitely and carefully made beauties. So, those are my personal feelings on that.

I am riffing this off the top of my head as well and I may have missed stuff that I will add later.

Analysis

The reason for writing this is more about the analysis than anything else. Because “how good” is an interesting question. There is good enough to be superior to most other things or be on an even footing with excellent work, and then there are the true tentpole heroic types.

Examples of this are Naoe Shizu which is maybe straddling 2nd and 3rd ranks and Shizu which is first rank, and Sue-Sa, in which the immediate followers of Samonji are 2nd rank while Samonji himself is 1st rank. 

When we look at the record of Naoe Shizu in the Juyo items we see that the talent level is certainly more than enough to qualify for this ranking. There are 157 swords from the Naoe Shizu school which are Juyo and that puts this school 21st overall in the list.

However, not a single one of them has passed to Tokuju, with one semi-exception. That blade got re-attributed to Shizu Kaneuji when it passed Tokuju. In this way we need to understand then that Naoe Shizu is associated with Shizu-like blades that don’t have enough outstanding qualities to be considered by the Mino founder Kaneuji (himself really a Soshu tradition smith). 

If we want to just examine flat out, pedal to the metal counts of school work passing Juyo, then these are the top 10 schools:

  1. Rai
  2. Soden-Bizen (mid-to-late Nanbokucho Osafune)
  3. Soshu (core group around Shintogo and Masamune)
  4. Ko-Osafune (pre-Nanbokucho Osafune)
  5. Hizen (Shinto Tadayoshi school)
  6. Osaka (Shinto)
  7. Aoe
  8. Horikawa (Shinto)
  9. Ko-Bizen
  10. Sa (Samonji and his followers)

One of the surprises is that the Hizen school outranks so many koto schools. If we look at individual swordsmiths we see this:

  1. Kanemitsu
  2. Rai Kunimitsu
  3. Rai Kunitoshi (not including Niji)
  4. Nagamitsu
  5. Tadahiro (Omi Daijo)
  6. Shizu
  7. Motoshige
  8. Soshu Yukimitsu
  9. Horikawa Kunihiro
  10. Unji

Again for me, two big surprises in Omi Daijo Tadahiro ranking so highly and also Unji. I don’t consider either to be a first rank swordsmith, nor does Fujishiro in his rankings.

The problem with just looking at simple counts is that there are relatively many works that exist for Tadahiro and Unji as well. Unji in fact may accumulate some swords that are a bit difficult to place in other bins.

If we are to add the Niji Kunitoshi attributed blades to Rai Kunitoshi then then this smith would move into first position for Juyo count.

Another thing which distorts this list can be seen when looking at the top five smiths. All of these smiths are heads during their time of very successful schools. Each of them had very talented master or grand-master smiths working for them. In the case of Nagamitsu his students have Kagemitsu, and Sanenaga, both of whom are Sai-jo saku and both of whom have made Kokuho blades. But there is an extended list of highly ranked smiths who worked directly under Nagamitsu and one starts to get an impression of Nagamitsu overseeing many sub-groups each of which was centered around one of these student grand-masters. These blades all came out with similar styles and when made mumei, if the quality is sufficient they likely took attribution to Nagamitsu. Not until Nagamitsu dies do the various smiths end up with their own signed work and some migration away from the shop style under Nagamitsu.

The same thing is seen with Rai Kunitoshi, and in fact Rai Kunitoshi and Nagamitsu even share one student in common: Rai Mitsukane. 

This pattern definitely held true for Omi Daijo Tadahiro, and in his case the volume of blades is also compounded by his very long production lifetime (working from age 18 to about 80). The fact that he is relatively young on the timeline compared to these other smiths has also contributed to a high survival rate of his blades.

We can include Shizu as another smith who had a lot of followers, and it’s not until we get to Motoshige and Yukimitsu that we can start feeling stronger about individualism in the manufacture of these blades.

So we definitely see some issues right away in trying to simply use counts of Juyo or Tokuju to try to measure skill. Furthermore the older things get, the harder it is for a blade to survive. Though Omi Daijo made many Juyo, these, percentagewise among his surviving works, are very low down on the list. So what we see in the Juyo works is maybe only the top 1% of his blades (guessing).

The ideas I am kicking around are trying to find a solid and objective metric for measuring the strength of a school or a smith. This metric needs to discount survival rate as much as possible as well as sheer volume of manufacture.

One thing I have turned over in my head in the past is the idea of Tokubetsu Juyo to Juyo ratios. This eliminates for the most part survivial rate by directly comparing the body of work to itself. Also, if a smith is simply getting a lot of Juyo through by having high production this should naturally deal with that. The one thing that it doesn’t handle is that for super rare makers, of which maybe only a couple of blades exist, this rarity will cause the relative importance of the few remaining to skyrocket. It’s not a perfect metric overall then, but it is an attempt to get a handle on things.

Some numbers

For this metric we concern ourselves then with counting the percentage of Juyo that go on to pass Tokuju. I’ll call that a pass factor. And will express it as a percent.

Once we look at the pass factors of the top 10 smiths by Juyo count, they look like this:

  1. Kanemitsu: 18.2%
  2. Rai Kunimitsu 15.6%
  3. Rai Kunitoshi (not including Niji): 11.3%
  4. Nagamitsu: 15.4%
  5. Tadahiro (Omi Daijo): 1.5%
  6. Shizu: 12.1%
  7. Motoshige: 13.8%
  8. Soshu Yukimitsu: 19.8%
  9. Horikawa Kunihiro: 6.8%
  10. Unji: 4.3%

So immediately I would draw your attention to Tadahiro and Unji, two smiths I singled out as not being first rank smiths. Their pass rate to Tokuju is substantially lower than other smiths on this list. If we look at the pass factor, it indicates that the strongest work on average on this list of the top 10 is at position #8: Soshu Yukimitsu.

Since there are about 1000 Tokubetsu Juyo swords and there are about 11,500 Juyo swords, this would mean that an average pass factor would be 8.6%. The better number that I’d want to dig at is the average pass factor for smiths who have achieved Tokubetsu Juyo. Unburying that number from the data is a little bit more work.

There are by my count, 293 different attributions for swords that have taken place at Tokubetsu Juyo. An interesting thing to note is that these 293 attributions (to smiths and individuals) make up 7,717 of the existing Juyo swords (67%). This tells us that the absolute best smiths are making up the lion’s share of the Juyo attributions. 

So anyway this gives us a better number for average pass factor among elite smiths, it is (1053/7717) = 13.6.

The average pass factor among non-elite smiths (but still superior smiths), is 0 by definition. 

We can then try to make some assignments with these numbers.

  1. If your work has passed Juyo, you are by definition a superior smith
  2. If your work has passed Tokubetsu Juyo, you are by definition an elite smith (pass factor > 0)
  3. If your pass factor is higher than 13.6, you are in the upper half of elite smiths (elite among elites).

Now we have some interesting modern metrics to apply. Let’s go back and look at the list of smiths again.

Our top ten partitioned into two groups looks like this now using a pass factor of 13.6 to partition them. We get two groups of 5.

  1. Soshu Yukimitsu: 19.8%
  2. Kanemitsu: 18.2%
  3. Rai Kunimitsu 15.6%
  4. Nagamitsu: 15.4%
  5. Motoshige: 13.8%

And the bottom half of the top 10, which are elite smiths from the bottom half.

  1. Rai Kunitoshi (not including Niji): 11.3%
  2. Shizu: 12.1%
  3. Horikawa Kunihiro: 6.8%
  4. Unji: 4.3%
  5. Tadahiro (Omi Daijo): 1.5%

This is telling us as well that there are likely a lot more pigeons to put into these holes, since we have 10 smiths from the top 10 count that are split evenly around the median pass factor.

Before hand-wringing too much about throwing Rai Kunitoshi under the bus, I will put emphasis on two points: there is one smith responsible for the work of Niji Kunitoshi and Rai Kunitoshi and the NBTHK acknowledges this currently as the best theory (I have no doubts on it myself). Even so, they follow tradition and attribute to two different smiths based on the style of the work. Niji Kunitoshi works are fewer in volume but higher in status, they have a pass ratio of 27.5. When you unify the two smiths it works out as 17.7 which puts him solidly in the upper half. Later works also have the hands of students in them most likely as daimei are quite prevalent though not indicated in the paperwork. Earlier work is all his own hand and this also shows why there may be a bias toward Niji Kunitoshi over Rai Kunitoshi in passing to Tokuju.

So, let’s have a look at pass factor itself and try to pull some top smiths from this information.

Pass factor analysis

I will trim very low Juyo count smiths from the results, because with only a handful of attributed items they distort the results. The top 10 smiths with a minimum of 5 known Juyo works are:

  1. Fukuoka-Ichimonji Norikane: 80.0%
  2. Awataguchi Kunitsuna: 66.7%
  3. Aoe Naotsugu: 66.7%
  4. Fukuoka-Ichimonji Sukemori: 60%
  5. Ko-Ichimonji Sadazane: 50%
  6. Hosho Sadaoki: 50%
  7. Ko-Bizen Kanehira: 47%
  8. Awataguchi Kuniyoshi: 45%
  9. Rai Mitsukane: 42%
  10. Awataguchi Norikuni: 42%

Now we have a list that is looking a lot more like the classically important schools I riffed off the top of my head at the top of this article. There are of course some surprises like Rai Mitsukane. 

There is a distortion involved here because some of these smiths get to dump their unsigned work into other buckets. So we we have left is only 5 Norikane, 3 of which are signed and one with Honami Kochu kinzogan mie and all of those four passed Tokuju due to this plus the reputation of the smith. His unsigned work will simply get classified as Fukuoka Ichimonji or Ichimonji and as such is not counting against him when we calculate pass factors. 

Rai Mitsukane has something similar, where his signed work is very rare and important but his unsigned work might get lumped in with Nakajima Rai or mumei Rai Kuninaga / Kunimitsu. 

We can try to control for this a bit better by increasing the minimum number of Juyo required, and so we eliminate some of these unicorn type smiths that almost do not exist. Let’s increase it to 8 for a new cutoff to be included in the list. 

  1. Ko-Ichimonji Sadazane: 50%
  2. Ko-Bizen Kanehira: 47%
  3. Awataguchi Kuniyoshi: 45%
  4. Kamakura Ichimonji Sukezane: 42%
  5. Ko-Osafune Mitsutada: 40%
  6. Soshu Akihiro: 40%
  7. Ko-Bizen Masatsune: 39%
  8. Samonji: 37.5%
  9. Umetada Myoju: 37.5%
  10. Masamune: 36.5%
  11. Awataguchi Yoshimitsu: 35%
  12. Fukuoka-Ichimonji Norifusa: 34.8%

The list starts looking a bit more reasonable once we eliminate some of the unicorns. Sadazane probably has some dump into Ko-Ichimonji attributions and Kanehira to Ko-Bizen so I added 2 more items to this list. But once we get past there, those smiths that remain on this list appear to correlate well to lists of highly superior smiths handed down from posterity.

This leads me to believe that this is likely a good metric and represents meaningful data. It is not perfect as you need always to understand the concept of a mumei dump and a unicorn smith. Smiths of the Ko-Bizen, and Ichimonji schools, along with others you can imagine such as Mihara, Uda, and so forth, usually send their mumei blades to the school attribution. This pass factor is then affected by this since the signed pieces carry more merit that allows them to get to Tokuju, while the smith is not carrying any mumei with him that will pump up the Juyo count. The tendency then is to exaggerate their numbers when associated with one of these schools.

And of course the unicorn smiths with their two or three known works can cause a very high ratio due to their rarity. Small numbers for sampling is always a problem in statistics.

However with this groundwork laid down, we can look at some schools, where I think it applies better due to larger sample sizes.

School pass factor

Unlike smiths we can also measure schools for weakness as well as strength. A relatively weak school can penetrate Juyo but will not often crack Tokuju. Remember: this is relative only to superior works since we’re talking about things that can pass Juyo.

I’ll get to reporting weakness another time, as I need to clarify some relatively obscure schools… and for now there are a lot that are coming up very low and won’t be interesting to write about.

Strength on the other hand is fairly clear.

  1. *Senoo: 55.6%
  2. *Umetada: 35.4%
  3. Awataguchi: 36.4%
  4. Kamakura-Ichimonji: 32.4%
  5. *Hikozan: 27.3%
  6. Fukuoka-Ichimonji: 26.4%
  7. Ko-Senjuin: 25.0%
  8. Soshu: 20.5%
  9. Ko-Bizen: 20.4%
  10. Kanro: 20.0%
  11. Ko-Ichimonji: 18.9%
  12. Ko-Aoe: 18.2%
  13. Katayama-Ichimonji: 16.7%
  14. Ko-Osafune: 15.5%
  15. Rai: 14.6%

Some notes… Senoo school blades are unusually strong because of the mumei-dump issue above. If unsigned they are likely to go to Ko-Aoe or Ko-Bizen attributions. Umetada blades passing Juyo are almost exclusively Umetada Myoju so this more represents his individual output rather than the school itself. Hikozan as well is almost exclusively Bungo Yukihira. 

So if we take those items with some grain of salt to understand that they have some bonuses, we can see the schools lining up with expectations from the anecdotal riff at the time.

Soshu in this case, as a school, is what I use to cover the core Soshu smiths: Shintogo Kunimitsu and Kunihiro, Yukimitsu, Norishige, Masamune, Go, Sadamune, Hiromitsu and Akihiro. Shizu is always hard to quantify, either as a Soshu “school” smith or as his own school. That is something I am rattling around, but in the context of this report above he is not included though I mentally include him anecdotally.

We can note in the above data that no Shinto schools are included. This is partially due to the difficulty of assigning Shinto schools with makers moving around in some of them, and some fairly opaque (like Kotetsu) but highly regarded. 

What I have for my own assignments has Noda (another essentially one-man school for Hankei) coming in as the first Shinto school in 17th place for strength on this list, and the next being Shimosaka at 21. 

An interesting thing with Shimosaka is that Echizen Yasutsugu always seems to catch some flak for being relatively unskilled compared to some of his peers. 

Yet his own pass factor (23.1) puts him in 29th place out of all individuals (including koto) and only Ippei Yasuyo (23.5) is ranked higher as a Shinto smith when we use this metric. He is higher than Hankei (15.4) and Shinkai (9.5) and Kotetsu (9.2). 

Some of this can be attributed to the fact that he made some striking copies of koto work and the NBTHK likes these for Tokubetsu Juyo. Overall, if he was truly of such lower skill as some authors would put him (Yamanaka regularly rags on him and Fujishiro places him at Jo-jo saku, below contemporary greats), then I don’t think he would place so highly in this metric. 

I think what we see in the data is that you can’t just take Fujishiro as the beginning and end of rating a swordsmith. Fujishiro is surprisingly good, all things considered. He was one man at one time period and didn’t have access to the large numbers of blades that current experts can see. In spite of this he did extremely well, but I think the data shows some departures from many of his conclusions. Not usually extremely significant, but I have had clients dismiss Jo-jo saku work as being not good enough for their considerations. And this is definitely the wrong way to go about it, it even misunderstands how Fujishiro built his rating system.

Conclusion

I think this is an interesting way of using a results-driven approach to ranking swordsmiths. It is not based on handed down reputation but on tangible data. However, I can’t say that tangible data is not affected by handed-down reputation! It surely is. Other items also affect these numbers I’m sure, such as the fact that the great old schools will have found their way into prominent collections and this has some influence in them passing to higher papers. 

As well, some other smiths like Nagamitsu and Masamune, many of the best works of these smiths have been “raided out” to levels of Juyo Bunkazai, Juyo Bijutsuhin and Kokuho. 

As such to improve this metric I should probably fold in all those above-Juyo rankings together. The purpose so far is to just bring this concept to light and give it a test run. So far it looks promising as an additional metric beside the traditional Fujishiro ratings and Toko Taikan ratings and as well, what Hawley did with his ratings system.

Another nice thing about this system is that it is dynamic. It will update every year as more information comes in. That in itself is very important because it allows the ratings to continually represent new realities when new realiities become available. 

This isn’t intended as a replacement for any system currently out there, they are all useful. At the moment it is mostly food for thought.