If You Are Black And Living In America, But Not Of African Descent, How Do You Feel About Being Referred To As An "African" American?

I have always wondered about this. I, myself, am white...and I have never really felt 100% right in referring to all black American citizens as "African" Americans if their ancestry is not from Africa... But I also don't know if it's okay to refer to them as "black" for fear of sounding offensive in some way. Thoughts? Advice?