If you intend to run rhythmbox outside of GNOME and don't have a
default icon theme setup, you need to configure one. Add the following
line in ~/.gtkrc-2.0:
    gtk-icon-theme-name="gnome"