My Living Room Walls are Yellow

Two summers ago I painted my living room.  The color is butter.  There is a wide selection of colors at the paint store with food names.  I might do the whole house in foods.  Salsa in the kitchen.  Latte in the bedroom perhaps.  In the morning the yellow is pale, but in the afternoon the light makes the color much brighter, and it is like living in a cartoon.  I like the color of my living room, but from time to time I remember how the guy I was in love with at the time said that no man likes yellow walls.  This I did not know.  There is a color wheel for men?  What other colors don't they approve of?  And is it only walls, or does this apply to furniture and clothing as well?  Logically I know that it doesn't matter what he thought, and certainly his issues were legion, fear of wall color only one amoung them.  But when it gets cartoon bright in here, I wonder if he was right.