We all fell in the W7 pitfall

[PeterSt]

Peter, is it possible for XXHE to release some memory while playing? Does it have to consume more and more memory during playback? I mean the memory that was needed for previous track.

Marcin,

I already optimized this (better) for 0.9z-3, and you can use significantly more memory now. But it is not really about "more and more", but about the largest track you coincidentally run into. I know, this doesn't show much in 0.9z-2, but anyway, freeing that is of no use, because you still need that "largest" track, and you need it at the same time a next small track comes by.
Or drop gapless.

Peter

[PeterSt]

But I agree - the SFS still plays a big factor in sound. Surely it shouldn't, now that XX sits in RAM (and 'Copy to XX-drive' is active). Someone, anyone, please. please offer at least an explanation of what might be causing this!

Mani.

Ok, just an idea ...

Can a few of you (if not all) SFS Expericnced tell me whether you have your Virtual Memory On or Off ?

Peter

[Marcin_gps]

Off here.

PS
I lost my post about optimisations, wrrrrrrrrrrrrr 

[manisandher]

Haha, ON here! (Currently set to 4390MB by system.)

Mani.

[PeterSt]

Bad luck.

I thought having it Off might incur for "attempting" it constantly, therewith influencing.


Another thing is : I can hardly imagine that which such huge sizes you (Marcin) don't have problems with Gapless ...

[PeterSt]

I did some reading about 18 months ago about the Audio system and Multimedia Scheduling Service (MMSS) changes that MS implemented moving from XP to Vista. I found a good MS technical briefing paper and came across these registration keys. I have suggested values below but to be honest I have not tried many permutations as they seem to deliver the musical goods. Again I am trusting my ears but I think this is a big sound quality tweak.

TWEAK PART 1

Use REGEDIT on the following.
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile
There are sub key tasks for;

Audio
Capture
Distribution
Games
Playback
Pro Audio
Window Manager

These KEYs are how MMSS prioritises music replay processes.
For the “Audio”, “Playback” and “Pro Audio” keys try the following DWORD values
Affinity = 3 (dec)
Background Only = TRUE
BackgroundPriority = 8 (dec)
Clock Rate = 1000000 (dec)
GPU Priority = 4 (dec)
Priority = 8 (dec)
Scheduling Category = High
SFIO Priority = High

A Note on Windows 7 performance;
The “Clock Rate” value sets the granularity of processor scheduling for audio. This was used for Vista but is not used in windows 7 !! “The maximum guaranteed clock rate the system uses if a thread joins this task, in 100-nanosecond intervals. Starting with Windows 7 and Windows Server 2008 R2, this guarantee was removed to reduce system power consumption.”


TWEAK PART 1

Use REGEDIT to change;
In HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile there is a key “SystemResponsiveness” set this to “0”.
Don’t worry this value defaults to give all Low priority processes 10% of CPU time.

Here is a bit more information on what is going off. http://msdn.microsoft.com/en-us/library/ms684247.aspx

Again I hope this works well on other peoples systems, I am dreading Peter saying that they have nothing to do with how XX works, if so it’s quite a placebo  !

Enjoy.

Nick.

Ps there are some interesting variables that can only be set in code referenced by the article above and in other I have read that might be useful to Peter in the code for XX.

Nick, All,

I am afraid I was right in my earlier "suggestions" that this won't do a thing (I forgot where it was, but you will have remembered );

So, these Registry Entries are "profiles", like the Key suggests. These Profiles can be addressed from within a program and *then* the data in there comes alive. Without that addressing ? nothing. Not even for WASAPI (Engine#3).

The good side of this all is ... I didn't do this addressing. Well, not before today. Haha.
I do other things though, and at this moment I really can't tell if it will make a difference;
The largest part of the work has been done (getting it all to run), so now it is a matter (for me) to fill that data with some sense and see whether I perceive differences. If not, I'll leave that to you - which I will do anyway (0.9z-3, and both Engine#3 and #4 will use it).

The nice thing is : Profiles are Profiles, and a Key like "Audio" or "Pro Audio" can be made up yourself (read : created in the Registry). Next what it takes is a means to select such a key, and per change of a Setting in XX the used Profile can be changed.

Notice that for XP this doesn't work at all, and for Windows7 I can't estimate to what degree it works. But officially for the latter : less.
Vista and its W2008 counterpart (under R2) must work.

So, another good reason to postpone 0.9z-3 a little.
Peter

[Nick]

Peter hi,
I agree that there are program calls to access Multimedia registry profile settings. My understanding is that MCSS sets up prioritisation based on these profiles when asked to by any code that makes the calls. On my config there is defiantly an marked A/B change in sound quality with the registry exits so I still think something is requesting MCSS to use the registry profile data. Could the HiFace Driver be invoking the settings ? or windows still has an awareness of the HiFace as a music hardware device, could MCSS apply the settings anyway when a process (say XXhignEnd) registers with the sound device ?

I took a scatter gun approach when coming up with the suggested key values by applying them to many of the profile types (pro audio, audio etc). I will spend a little time trying to pinpoint the profiles and key values that have an effect.
Whatever is happening  for with current versions, I do hope this is exploitable for 0.9-z3.

Regards,

Nick.

[PeterSt]

Well Nick ... I don't know what you're hearing, but it is worse. That is, to me that seems, and I can't make anything better out of it. What ?

It just doesn't work at all. Or more accurately : it certainly doesn't do what it intends.
I have been trying and testing and everything, but to me it's a half baked hoax. Additionaly (or maybe that's why) MS of course admitted that half of the OS collapsed because of these jokes, which -thus- didn't do what they wanted in the first place. Add to that that WASAPI didn't work in the first place when Vista came out (I with a bunch of tweaks / XXHighEnd was the first one for more than a year), so all was just stupid theory. And now it appears to just not work.
It may do things to other processes, but a simple appointing Affinity already does not work.

Besides all, it wouldn't do anything extra regarding to what I do myself and also it will confuse if I'd let it in and if it would work.

Maybe with(in) WASAPI it will work afterall (today's MS examples (which weren't there before at all) are with WASAPI only), but I don't believe that. And what if so ? we're not using that anymore (ok, hardly anyone I think).

In the end ... trust me. I do all (and much more) than this whole thing would do if it worked. I/O and Driver priorities, that's another story. But as I explained earlier, that doesn't seem to be necessary (by coincidence ?).

Case closed I guess.
Peter

[Josef]

Case closed I guess.

Not so fast: case re-open!

I was finally able to try out Vista (thanks Peter for the key!) and, indeed, I must admit that overwhelming positive response here is fully justified! (This is so weird I still have trouble believing it so my next step is to install clean Win7 just to make sure this is not a placebo or a corrupted Win7 installation.)

I was also able to test a couple of tweaks mentioned here and so far I can verify the following:

1. MMCS registry settings: These have no effect in Kernel mode (Engine#4).
This is very easy to verify as MMCS service is clearly completely inactive  when XX is playing via Engine#4.
However, it can also be seen that MMCS does work when WASAPI (Engine#3) is used and it works as advertised: popping up every 10ms and doing it's thing (whatever that is). (note I have not tried MMCS registry tweaks as I have used only Engine#4 so far).

An interesting note: seems that Foobar (which, btw, also sounds better under Vista) does _not_ really use Kernel streaming even though it can be selected! The reason is that MMCS is clearly active when Foobar works (whether WASAPI or Kernel is selected) while it's dead as rock when XX is playing via Engine#4 - a proof of just how difficult it is to get KS 'right' and/or proof of Peter's audio-programming prowes? Kudos to Peter anyway   (note I used 'older' Foobar 0.9.6.9 but Kernel Streaming component hasn't changed since 2006 so it should not matter...)

2. Much more interesting (at least to me) was Nick's Interrupt Affinity discovery as I was frustrated not being able to get it to work under Win7 and yet I believed in (at least) a theoretical merit of being able to manually appoint interrupts to CPU cores. Short story: Nick was right!

Affinity tool works perfectly well under Vista SP1 64 (not tried 32 and Peter I believe you are on 32?)
I could clearly hear DAC 'ticking' when tool asked to restart device. Likewise, my mouse stoppped responding for a while and USB HD came up from sleep. But, I did have to reboot - even with all devices properly restarting affinities were NOT applied until after reboot.
Curiously, XX reverted to demo mode so I had to reactivate it every time I changed affinities (it worked OK every time though).
I first observed interrupt & DPC distribution without affinities, then with USB Host Controller (with Offramp attached) set to CPU1 first and then to CPU0. When there is no music distribution is about equal between cores, as would be expected. However, playing music will drastically increase #DPCs from 200-300 to 1,400-1,500 range.
It could be clearly seen that CPU1 handled _all_ these 'extra' DPCs while CPU0 was practically unchanged from 'no music' state.
And when affinity is switched to CPU0 then _it_ would serve Offramp's DPCs and CPU1 would show no change. There is no way around this - this works! Kudos to Nick!

Now back to listening tests

[PeterSt]

I/O and Driver priorities, that's another story. But as I explained earlier, that doesn't seem to be necessary (by coincidence ?).

Case closed I guess.

I want to emphasize the above, because after reading Josef's response it looked like I wanted to close down "the whole case", which isn't true. And then I thought "but I thought I made a remark about the I/O interrupts ?". Well, only after the 5th time of reading my last post I found this little (quoted) sentence,  so I must have hidden it very well in the remainder of that post. So, in case this happened to you (Nick) the same ... I only wanted to close this "Profile" thing, certainly not the I/O thing. Even

that doesn't seem to be necessary (by coincidence ?)

this shouldn't do a thing to "resting the case" because this counts for 4 cores, and not for 2 (as I explained about earlier).

So, I hope it is clear that I am as open to this subject (see Josef's post) as can be, only I couldn't get it running. And of course not to forget, I was even asking for it somewhere in between the lines - which Nick so superbly picked up.

Peter

[PeterSt]

Josef,

Curiously, XX reverted to demo mode so I had to reactivate it every time I changed affinities (it worked OK every time though).

Please look very carefully into your RAMDisk setup and possibly the AutoHotkey config (if you use that to startup XX), because this is totally impossible. But it is all so easy to start XX from somewhere else than you think, especially when both places are equal.

However, it can also be seen that MMCS does work when WASAPI (Engine#3) is used and it works as advertised: popping up every 10ms and doing it's thing (whatever that is).

As I already told more or less, MMCSS existed right from the beginning of Vista, but examples weren't there. Today's examples do contain the call to this particular function (which is really4 lines of code), and I certainly don't have that in. Now, it can be that somewhere under the hood the function is called by one of the hundreds of programs involved (which just come along with the SDK samples I ever used) BUT keep in mind it is a profile which is addressed. So if this is the case, it would be the most likely that this would be the "Audio" task, knowing that AmirM (head of audio development at MS back then) was a consumer audio guy. But if not that, then "Pro Audio". And only one of them. So, changing them all would just be good if you don't know which one is addressed (like Nick's example did), but it doesn't proove it works. So, if you think this does something (and I sure do belive you of course), try to find which one is involved, and proove it by something you can recognize. Affinity is the easiest one, but choose Scheme "Nothing" in XX.
Notice that for me it wasn't clear at all how to test it because I do all these kind of things as well and they mey interfere with the results, but since you "see" it working anyway, please proove it by means of manipulating the registry data of the task concerned (which is to be found out first).

I think you may come up with nothing, but with the conclusion there's something else again. Or just the difference between WASAPI and KS.

(note I have not tried MMCS registry tweaks as I have used only Engine#4 so far).

See. It *must* be something else.

And I hope this is clear too : if it can work afterall, no case is closed of course. 32bit/64bit (I'm indeed on 32) can make a difference when drivers are involved (and they are).

Don't feel obliged to do this all, but know I'm listening for sure !
Peter


PS:

(This is so weird I still have trouble believing it so my next step is to install clean Win7 just to make sure this is not a placebo or a corrupted Win7 installation.)

I did that (including the RAMDisk stuff). It didn't help (4 pairs of ears listening). Haha.

[Marcin_gps]

Josef, how many cores does your CPU have?

[Josef]

Please look very carefully into your RAMDisk setup and possibly the AutoHotkey config (if you use that to startup XX), because this is totally impossible.

I solved the problem late last night but was too tired to post: What happened was that I did activation only in XX run from RAMDisk. When I did a _proper_ activation i.e. when XX was started from SDD it didn't go to demo mode after reboot.

See. It *must* be something else.

I'm afraid I don't understand a word of what you just said (especially that thing about 'profiles').

We may have a misunderstanding here which is probably my fault for not more clearly explaining what I did and what I 'saw' so let's try it this way:

1. Open ProcessExplorer and add column 'CSwitch Delta' to display
2. Let XX play using Engine #4
3. You should _not_ see any increase in CSwitch Delta next to your svchost.exe processes (some go up & down at times but that is normal)
4. Now let XX play using Engine#3
5. You _should_ see one of those svchost.exe go to 100 CSwitch Delta: If you have a quiet system this will be obvious as previously it was probably not active at all (my case). And it is 100 because MMCS gets activated every 10ms per MS defaults. How do we know?
6. Right-click svchost.exe->Properties->Services: MMCS should be in the list.

That's it.

So this is purely an objective technique to prove when MMCS gets activated. As I said, what/if it does anything to sound and whether it is really influenced by registry tweaks is beyond this - I plan to test that later but it's not my priority (as Engine #4 simply sounds better).

And about testing IRQs: it's also easy:

1. Open ProcessMon and look at DPCs queued/sec & Interrupts/sec.
2. Do this with different Interrupt Affinity tool settings (e.g. affinity to CPU0, CPU1, none)
3. You should see a clear pattern of (especially DPCs!) following CPU you used for affinity

Again: keep in mind that reboot is required after changing affinities.
Also, and this is what I discovered later, if your sound device shares IRQs with other devices you MUST set all of those to same CPU! For example, my laptop has 2 USB ports but both share same IRQ. So, in effect, sound device (solo on port 1) was sharing IRQ with mouse & HDD on port 2. Device Manager showed 3 'Host Controllers' associated with this same IRQ. I had to set affinity for ALL of them to be the same, otherwise it didn't work.

So, again, no subjective (listening) impressions here just easily observable facts.

Please do try these 2 simple experiments and post your findings!
You never know, maybe there is something weird about my laptop or I have made a mistake (and I can certainly make some, like 'rogue' Q2 setting from last week Well, at least we all learned about what Q2 really does and it is quite amazing in it's own way but that should go into a different topic )

[Josef]

Marcin: 2 cores (plain vanilla Intel Core Duo).

[PeterSt]

I'm afraid I don't understand a word of what you just said  (especially that thing about 'profiles').

Josef, no problem. I at least understand you.

If you look a few posts back : http://www.phasure.com/index.php?topic=1398.msg13655#msg13655
... you see what I was talking about, including the Profiles. I went on on that one in my post following the one I referred to (doesn't work blahblah), and next you tried to lay out it all *does* work. But what ? I think it is clear now that you didn't much refer to my post(s), although you nicely started with the "case re-open".

So, no problem on this side, but I'm still talking about the same, while you're a bit on your own route. Mind you, as valuable !

Ok ?
Peter