Kittens 3 weeks and under can't regulate heat on their own. When cold their health can decline rapidly, so it's important to keep them warm at all times. Even after three weeks they enjoy being cozy, so providing bedding and blankets is very important. For younger kittens under four weeks of age, a heating pad or microwaveable heat disk is also needed.
Young kittens will require a milk replacement such as KMR, Breeder's Edge, and Fox Valley. Once they're a little older you can start mixing the formula with baby food or wet food. This mixture is called gruel. Eventually you'll wean them off gruel and on to wet food. I like to also offer dry food at all times once they reach the gruel stage.
These items are essential for keeping your foster clean and healthy. I use baby wipes after feedings and to help stimulate the young ones to pee or poop. I also use kleenex for stimulation. Pee pads keep your crates less messy and make for easier clean up! Pee pads are essential until they are using the litter box fully but they can still be helpful for messes even after that. I use vet grade disinfectant wipes and cleaners to keep my foster areas clean and germ-free. This is so important to preventing the spread of infections to other foster, or even the next set of fosters.
I like to keep 1ml-5ml syringes on hand for administering medicines and even formula to tiny kittens. Different medical supplies I like to keep on hand are flea meds, Metronidazole (flagyl), Strongid, Albon, Panacur, subcutaneous fluids, a fresh line and needles.