In semiparametric models estimation methods interest is often in the finite dimensional parameter, with the nonparametric component a nuisance function. In many examples, including Robinson’s partial linear model and the estimation of average treatment effects, the nuisance function is a conditional expectation. For the large sample properties of the estimators of the parameters of interest it is typically important that the estimators for these nuisance functions satisfy certain bias and variance properties. Estimators that have been used in these settings include series estimators and higher order kernel methods. In both cases the smoothing parameters have to be choosen in a sample-size dependent manner. On the other hand, nearest neighbor methods with a fixed number of neighbours do not rely on sample size dependent smoothing parameters, but they often violate the conditions on the rate of the bias unless the covariates in the regression are of very low dimension. In many cases only scalar covariates are allowed. In this paper we develop an alternative method for estimating the unknown regression functions that, like nearest neighbor methods, does not rely on sample-size dependent smoothing parameters, but that, like the series and higher order kernel methods, does not suffer from bias-rate problems. We do so by combining nearest neighbor methods with local polynomial regression using a fixed number of neighbors.