Whisk together sugar, cornstarch, cocoa and dry milk. Add water. Whisk until smooth. cook over low heat until thickened, whisking often. When thickened, remove from heat and add vanilla. Let cool a few minutes and then pour into popsicle molds or small paper cups. Freeze until partially frozen and then insert plastic spoons, if using paper cups.
Recipe from Homeschoolforms.com, Site Maintained by Donna Young, of DonnaYoung.org