For the steps you need to build a header over the landing. The bottom and insides of this header and the ceiling will be drywalled. I did this and I'm glad I did! The landing at the bottom is not a real landing, just an imaginary area about 2 feet out from the last step.
You can do the same thing around the windows.
BTW - 3 inches for the drop isn't much. That precludes using fluoresence fixtures. Recessed cans would still work.