The Kitáb-i-Aqdas - The Most Holy Book
Index term: warm recommended, under for washing; Water
n167.
Wash your feet
The believers are exhorted in the Kitáb-i-Aqdas to bathe regularly, to wear clean clothes and generally to be the essence of cleanliness and refinement. The Synopsis and Codification, section IV.D.3.y.i.-vii., summarizes the relevant provisions. In relation to the washing of the feet, Bahá'u'lláh states that it is preferable to use warm water; however, washing in cold water is also permissible (Q&A 97).
q51.
 
Question: With reference to ablutions, it hath been revealed, "Let him that findeth no water for ablution repeat five times the words 'In the Name of God, the Most Pure, the Most Pure'": is it permissible to recite this verse in times of bitter cold, or if the hands or face be wounded?
Answer: Warm water may be used in times of bitter cold. If there are wounds on the face or hands, or there be other reasons such as aches and pains for which the use of water would be harmful, one may recite the appointed verse in place of the ablution.
q97.
 
Question: Concerning the washing of the feet in winter and summer.
Answer: It is the same in both cases; warm water is preferable, but there can be no objection to cold.