I am scared to move away from my family?

I am going to school right now for nursing and I have awhile before I am done, but my husband wants us to move out of our state when I get my RN. This makes me nervous because he wants to go where we have no family it would only be my husband and I and are 3 children. I feel like a wimp because I am scared to live away from my family but he really wants to leave. Should I just go with his plan or what? I don't know what to do. Is it hard to be without your family? I also think my kids would miss their grandparents.