Hi wushuliu and
Every system being used here will have slight variants, so there is no "correct" buffer settings per se. However, as a general feeling, the larger the buffer setting, the "warmer" the sound is perceived. Conversely, as the buffers are lowered, the sound will appear to exhibit quicker transients, which some perceive to be more "accurate". Personal preference will dictate as to what you finally end up with. You will notice from my signature I use very low buffers, and that is attributable to the Juli@ card to a large extent in being able to support those low buffers.
It is also a good idea to monitor your CPU usage initially while lowering the buffer size. A very low setting will increase the CPU requirements considerably. If you experience background clicks and glitches, there is every likelihood the CPU is not able to keep up, or, your "schemes" need adjustment.
In helping to arrive at your ideal setting, I suggest you only change one parameter at a time, and then spend a day or two, before altering another. It can be very confusing if you change too many items at once !
When you have a spare moment, a signature ( go to your profile ) outlining your source path and XXHE settings will be useful for others to see what you are using.
Have fun
Russ