4 months no need to drink any water actually, breast milk and formula will provide enough water.
Nurse say below 6 months try to not to give water unless is recommended by doctor ( sometimes constipation). put some ice on baby area where the tooth is coming out.