It is interesting that you asked me, because I also do not enjoy working with chemicals very much! Yes, I think there is a pretty substantial difference between materials science and chemical engineering. Back when I was studying materials, we primarily studied metals (metallurgy) and also ceramics. Lab’s included things like: looking at the microstructure of polished metals under an optical microscope (which you might need to etch with an acid, but that is easy enough); doing tensile testing (where you get to pull apart a “dog-bone” shaped sample until it breaks); doing impact testing (where you swing a heavy lever at a sample & it breaks in half in a dramatic way); measuring the hardness of a material, etc. My favorite lab work is using a scanning electron microscope (SEM) to take very high magnification images of materials, which can sometimes looks like little forests or mountains. Some of my space exposed polymer SEM images actually remind me of outer space scenes.
Modern materials science will include other types of materials like composites (two types of materials together) and possibly polymers (plastics).
If your school has a materials science department, I suggest you go talk to the Chairman and learn a little bit about their program. Maybe you can sit in on a class or two. Or, try to shadow a materials engineer for a day.
Good luck, and send back any other questions!