To make a DataGridView have deletes that work when setting
RowHeadersVisible = false;
it is necessary to set the SelectionMode for the DataGridView to be DataGridViewSelectionMode.FullRowSelect
dataGridView1.SelectionMode = DataGridViewSelectionMode.FullRowSelect;
This will then allow the Deletes to work as expected