In the U.S. It used to be bad etiquette indeed, because it could get your second coach kicked out of the PSA and USFS, so they couldn't coach anymore, if they weren't teaching you with the first coach's permission.
Very recently, I believe the organizations were told that was an illegal anti-competitive practice, and PSA rules have changed.
But most coaches and skaters still feel it is just good manners to ask your current coach before teaching you too - and if they don't know about each other, they might keep stepping on each other's toes, giving you contradictory info, and be confused that you are doing something opposite from what they told you.
So, yes, talk to both of them about it. Some coaches have styles that are more consistent with each other than others, and your current coach might be able to advise you on who might be compatible.
I used to have two coaches, one Dance, one Freestyle. It was more confusing than anything else. I have also changed coaches too often, and am rather confused about many things. Unless you have a good reason to have two, it will be easier on you to stay with one while you can.
Of course, a lot of the top end skaters have multiple coaches - because it's possible no one can completely master all the intricacies of skating - I don't think anyone even knows all the USFS and ISU rules that you have to follow, which change frequently. They might have one coach who specializes in spins, one in jumps, one in choreography, one in off-ice training, etc. Some coaches are very good at keeping track of what specific test and competitions judges like, or maybe at playing politics when it comes to who the national skating organization allows to compete at International competitions. But they usually have one coach who takes the lead, and helps them make decisions where conflicts arise.
I've never been a top end skater, so that never made sense for me.
But it's your decision. Figure out what makes sense to YOU.
Sometimes a great way to try out coaches is to take a few group lessons from them. But again, it's good to tell your current coach that you will, and ask if it is a problem. If you are above the level of the standard group lessons offered in your area, some coaches offer specialized classes or clinics of their own creation that don't follow a standard ISI/USFS content.