Now, I am wondering if the torque converter is a different size, too…

I recall swapping the bell housings around in my old 77 mustang II but I don't recall the years.

I think you already have your answer - the small and large bells are different and look different, but you can bolt a small bell to a large case C4. Here is more information. Being an engineer, I'm thinking you like a ton of it. 
This is from a post I made years ago on FordMuscle as "R Code". I asked all the C4 guys who were frequent posters to gin up their C4 knowledge and then I spent hours and hours and coalesced it all. It's got a lot of good info, but unfortunately a ton of dead links. 191,000 views and 82 replies.  

C4 Monster Thread - 
Bellhousings of course come in several versions. Case fills come with bells that bolt to the pump while pan fills come with bells that bolt to the case. C5 bells are deeper than either of the C4 versions. Most of the time, the case fills use the 157 tooth flywheel and the pan fills use the 164 tooth, but there are exceptions. The best tip I've seen to ID them is to measure the depth. If the depth is 5 7/8" it is a small bell 157t, if it is 6 1/4 is regular bell 164t and if it is 7" it is a c5 bell 164t.


https://www.fordmuscleforums.com/threads/monster-c4-thread-text-edited-way-long-version2.416584/