Rose Hill Cemetery served the Rose Hill community, originally known as Nelleville. The settlement was founded by German immigrant George Heinrich "Henry" Nelle and his wife Dora (Lohmann), who donated this property for cemetery use in 1887, although the earliest known burials here date to 1883. The burial ground also contains a number of unmarked graves. Those interred include a Civil War veteran, farmers, and other early settlers. The cemetery was associated with Rose Hill Christian Union Church, which later became Rose Hill German Lutheran Church. Today, it remains a vestige of the Rose Hill community, chronicling the pioneers who lived here in the late 19th and early 20th centuries.