Ayurveda suggests that you shouldn't put anything on your skin that isn't good enough to eat! I love this idea of the skin as a digestive organ and it narrows down your massage oil options immensely. It also explains why we should never, never use mineral oil for massage. Mineral Oil is a highly processed by-product of petroleum and aside from being something you definitely would NOT eat, it blocks natural skin secretions and stops the skin from regulating temperature.
Any oil used on a baby is best organic, extra virgin and cold pressed. To keep it simple I'll only suggest three oils and you can choose what works for your baby. As with everything in the world there is a small risk of allergy.
Here is an Ayurvedic description of three massage oils.
It is the most classical Ayurvedic massage oil, particularly black sesame if you have access to it. It is the only oil to penetrate all seven tissues. It's hot and heavy qualities are very grounding and nourishing, but maybe imbalancing if Kapha or Pitta is high.
It is one of my favourites because very high-quality olive oil is so readily available. It is not as heavy as sesame oil and is cooling, so very useful for babies with any kind of hot red rash. Allergy is very rare.
It is a light, cool oil that suits all Ayurvedic constitutions. It is not as rich and nourishing but it is high in essential fatty acids and Vit E. It is most useful for a Kapha baby.
Store oil out of direct sunlight, in a dark glass bottle in a cool place.