I live in a townhouse where the shared walls from the foundation up are concrete –> brick masonry –> some other unit type masonry ( but different than the bricks right above foundation. 

The joists of the first floor are imbedded into the masonry units . My first question would be what is the best way to insulate around those joists -or if it is necessary at all. The wall is not an exterior wall in the same as a normal one since it connects to another house with a conditioned basement.

Secondly, the plywood subfloor is no imbedded into the masonry so there is effectively a gap that goes right into the 2nd floor. Is it worth sealing that with great stuff type foam to seal it? While the basement is conditioned it is typically cooler than the rest of the house.