1. Rashbam uses the standard Hebrew word for grandson, נכד, to mean nephew here. (A number of biblical texts state unambiguously that Lot was Abraham’s nephew; see e.g. Genesis 12:5.) Rashbam uses נכד to mean “nephew” a number of times in his Talmud commentary, too. See e.g. his commentary to BB 109a, s.v. ומשני אחי האב, and his commentary to BB 114b, s.v. האשה את. From Ben Yehudah (7:3655) it appears that Rashbam was the first to use the word נכד in this unusual way.
Perhaps interference from French caused Rashbam to do this. The common French word for nephew is neveu (or, as it is generally spelled in medieval French, nepveu). That word is actually derived from the Latin nepos, which means grandchild. According to Le Robert: Dictionnaire historique de la langue française (Paris, 1992), p. 1320, the French word neveu or nepveu originally meant grandchild. But already in the very early days of French it took on the additional meaning of nephew. Since Rashbam knew that the word nepveu could be used for either grandchild or nephew, I assume he felt that one Hebrew word could also mean both grandchild and nephew.
Cf. Qeren Shemuel here who emends נכדו to read קרובו, and see Rosin, RSBM, p. 44, note 1, who notes correctly that the emendation is uncalled for.

2. Rashbam reiterates the second point that he made in his comment to vs. 5. See note 40 above.