Prime Minister Anthony Albanese and Opposition leader Peter Dutton have had little to say on First Nations issues.