Gamma-ray bursts (GRBs) are the most energetic explosions in theUniverse, thus providing a unique laboratory for the study of extremeastrophysical processes. In parallel, their large luminosity makes GRBsa premier probe of the early Universe. My thesis has explored andexploited both aspects of GRB science by addressing the followingfundamental open questions: 1) what is the nature of the GRB ejecta?, 2)how does the GRB progenitor population evolve with redshift, and 3) howcan GRBs be used to probe the high-redshift Universe? To answer thesequestions, I present the first multi-wavelength detection and modelingof a GRB reverse shock, a comprehensive analysis of the plateau phase ofGRB light curves, studies of the evolution of the progenitor populationto redshifts, z~9, and demonstrate the use of GRBs as probes of galaxyformation and evolution through the first galaxy mass-metallicityrelation at z~3-5. I find support for baryonic ejecta in GRB 130427A,evidence that GRB jets contain a large amount of energy in slow-movingejecta, and proof that the GRB progenitor population does not evolve tothe highest redshifts at which it has yet been observed. Building on thedecade of observations by the Swift GRB mission, future observations andmodeling of GRBs and their host galaxies will provide clues to these andother open questions in GRB science, allowing for the first statisticalstudies of their progenitors and host environments to the epoch ofreionization and beyond.