There is growing concern that there aren't enough doctors to properly staff the South Okanagan General Hospital emergency department.