The word comes from Sanskrit, mantrinah (advisor, minister). And mantrinah comes from another more well-known Sanskrit word: mantrah (advice, prayer, hymn).
Like orange, mandarin originates in China. In the West, the younger sisters of the oranges started naming for a simple reason the same name as the imperial bureaucrats: the costumes and the fruits of the officials were of the same color.