Dear Tally (I'm wordy so this will end-up truncated, click the link at the end or below your view that says "view entire message):
First of all, you state "Not sure how this spot rotted so badly. I had a leak in the tub drain."
Are you sure you have corrected the only leak? If you repair this now and caulk well and you still have a leak problem, you're only going to hide and seal up this water to cause even further damage that won't evidence itself until its too late. At the end of this I'll throw a list of things to investigate for leak sources (sneaky leaks that only leak when you're not looking), but there's always the obvious thought that someone isn't protecting the area adequately with a shower curtain.
Secondly, regarding deck screws, I wouldn't use them. I suggest for such a wet area, where removal/repair in such a wet environment of a bathroom being so likely, and errosive/corrosive effects on screws at their most extreme
to use STAINLESS 18/8 Metal screws with flat topped phillips heads. At least a #10, possibly #12, but I think #10 head will be perfect for your application. Since you have to screw into the floor joists and blocking material, I'd suggest at least a 1-1/2 screw, but you might want to go with 1-3/4 or 2" if you can find them. They are just like "brass wood screws" except they are threaded all the way up to the head. Be sure to drill pilot holes as recommended on the screw box, usually a #10 requires a 9/16th pilot hole. Mark your drill bit with some masking tape at the length of the screw so you don't "over drill" this is quite important when doing this on structural members, and as you never want to penetrate beyond the one-third mark from above on these floor joists as that will weaken their structural integrity, and you are kind of weak already with 2x8's at 16" OC. Do not try to counter sink these beyond flush or just a scant below flush, as to do so will bust the fibers in the outer ply of the plywood thereby reducing its strength. Use the same screws I mentioned or longer if necessary to secure the flange ring for the toliet to the replaced sub-floor in that area. If you can find these screws for a No. 1 bit or a regular screw head that would be even better, but usually the "box stores" only carry #2 phillips head for these screws.
Third, the outer edge of your tub is one of the heaviest load points on the drain/service side. Generally when tubs are installed they are not level, but pitch towards the drain and away (high) at the walls that are NOT the drain sides. your seams in the sub floor should be centered on joists but NOT the joist that suppports this tub edge, (in fact there SHOULD be a double joist that runs the length of this tub on the long outside side. You really want those joist edge seams 2 joists away from this major stress edge if at all possible if in the traffic area of the room, or you'll have flex problems in the traffic area messing up your floor finish and squeeks.
Most likely your partition "plumbing wall" is applied ontop of this subfloor and the rot may have broadcast under that wall plate. You really should consider opening up that plumbing wall a bit near the floor from the oposite side of the wall from the tub side and investigate the area there for damage, and/or leak sources.
for blocking the perpendicular sub-floor seams you can use 2x4"s toe-nailed to the floor joists from below, layed with the 4" side flat against the lower face of the subfloor, then stagger your screws about 3/4" OC in from the edge of the subfloor on either side of the seam and about 2-1/2" OC apart. This way you'll not have perpendicular flex between old and new.
I'm guessing your home is mid 60's to mid 70's construction as you mentioned 2x8"s and plywood subfloor, as in older homes you usually see dimentional lumber sub floors layed with staggered butt ends 45-degrees from the floor joists, and later homes beefed up the codes to 2x10 joists 16" OC. You might find that you actually have 2 layers of 1/2" plywood for your sub-floor/floor under the tile (cheaters method) or you may find that the lower (closer to the floor joists) layer is 5/8" plywood then dressed with 1/2" layer, as mid-60s required minimum sub-floor 5/8", or if transitioning to like tounge and grove hardwood elsewhere, you might see 2 layers of 1/2" ply or a 1" ply so that with the addition of tile, it is the same height of like hardwood in the hall outside the bathroom.
Third: possible hidden sources of leaks:
1) Assuming this is a tub/shower combination situation, the tile grout or surround seams may have failed, allowing water behind it, running down the inside to the flange/mounting edge lip-edge of the tub, that then travels down to one of the corners, then drips off and runs along the sub floor under the tub to just inside/under the tub at the area you see evidence.
2) weakened area that allows the structure and/or tub to flex when in use that allows a leak situation only when stressed (like when someone standing in it for shower, or the tub filled for bath and/or with addition of person
3) the caulk seal between the tub lip and the wall/tile/surround, allowing water behind, traveling to the lower corner, dripping on the subfloor, and running along the surface of the subfloor to the point you find the damamge.
4) the overflow/drain assembly, often the overflow escutchen gasket (rubber) fails overtime, allowing water behind it, running on the outside of the overflow and onto the subfloor.
5) mixer valve/water handle escucheon plates not being sealed/caulked or the caulking failing that's behind it
6) leaks in the mixer valve or plumbing in the wall servicing the tub/shower
7) someone who shall remain nameless not securing /overlapping the shower curtain properly, and water running out the tub and onto the floor while in shower use or excessive splashing/playing while in the tub
8) As a tub is not set "level" but with a pitch toward the drain, so are all the exposed rim of the tub, so the site of the drip damage is not always the location of the LEAK/invasion from the wall zone above the contained tub.
9) if the floor is pitched from the toliet down to this point at the tub could be a leak from the toliet? that runs on top of the tile and terminates there?
I'd open the oposite wall so you can see there view with a mirror and flashlight, view again with the tub filled (heavier each gallon of water adds about 7 pounds), then get a volunteer to stand in the full tub, using a level on the floor, see if you're getting flexing/additional pitch when the tub is loaded. Do the same test with a volunteer sitting on the toliet seat AND flushing both with dry tub and full loaded tub, then investigate the overflow/drain assembly's connection to drain/vent see if that flexes and shows a leak when "weighted". Next have someone direct the shower spray at the plumbing wall while standing IN the shower loaded weight while you investigate see if you have a flexing issue there regards to the overflow and/or drain and escutcheons.
most likely the water rot has broadcasted outside of the tub now, but started under/inside the tub, and that area will need to be repaired as well, or you'll just have mold/rot issues continue just hidden until the problem is worse. You need to identify the source of this water and correct THAT first. Also I'm concerned that your plumbing wall is constructed ON TOP of this subfloor and that the rot may be under that plate as well, please check as if this wall is effected you have more work to do, as if its resting on rotten wood, that wall will flex and you'll always have a leak situation between the tub and this tile on this line or where the back wall meets this wall on the inside of the tub.
I hope I was clear with what I wrote. Please advise if I wasn't and also let us know what you found.
Good luck and best regards,