DAT files?

[Josef]

Peter,

If I drag/drop lots of WAV files into playlist window (say, 1000) and if 'unattended' is ON, then when I click Play it seems that XX is creating a small dat file for _every_ WAV file.
I also have 'start Engine #3 while conversion is in progress' (or whatever that option is called) turned on so music starts relatively quickly - so far so good.

However, creation of those 1000 dat files takes a long time (tens of minutes) and I'm pretty sure it impacts SQ (random pops/clicks) as it has to do lots of disk reading & writing...

Now, I have no clue why XX needs all those small pesky dat files but I assume you have a good reason for it so np:  I _could_ live with that provided it's done only once but it seems that even if I 'save' playlist and then exit XX it will delete all dat files and the re-create them next time I play that same list? (or am I doing something wrong and there is a setting to turn this off?)

Wouldn't it be better if XX created a single dat file for whole playlist? (less disk activity)

Wouldn't it also be better if dat files would respect XX setting for 'Data folder' instead of being created in same folder where XX exe is? 

And wouldn't it be better if dat files would not have to be created every time but rather saved together with playlist?

I realize this only applies to WAV files as other types need converting but since XX only wants WAV in the end anyway, then at least I am OK with creating a separate folder where all music has already been converted from FLAC etc to WAV - it sure is extra work and it would be nice if XX would respect that and start playing with minimum of fuss

[Ava12]

Do you have "copy to XX-drive by standard" checked?

[PeterSt]

I will come back on this tomorrow, with hopefully some reasonable explanations. But the questions are good ...

[Josef]

>Do you have "copy to XX-drive by standard" checked?

No.
I guess if i did it would be even worse i.e. XX copying 1000 big WAV files as opposed to 1000 small (~300 bytes) dat files...right?

[PeterSt]

Ol, let's try ...

First off, an album would copy in 7-8 seconds, assumed optimal speed (which would be (e)SATAII). Converting from FLAC does not take all that longer.
Creating 1000s of those dat files ... a few seconds at most.

There are many reasons complete files will need a copy, and some are beyond what you'll expect. One of them will be the Unicode thing (diacritical marks etc.), so if they are in the path or file name, a copy already takes place. Another is one type of header format (IIRC WMP can create that), and the whole file needs to be recreated in order to play well.
Whichever reason there is, if the file is copied, converted or recreated, it will turn into a "UnicodeTrack" file. They too show up in the XX folder, but are hidden.
If it takes tens of minutes to let XXHighEnd finish the preparations concerned, you must assume that at least many files will be copied etc. But you can check it, by looking for those UnicodeTrack files (UnicodeTrack0001.wav, UnicodeTrack0002.wav, etc.).

A next subject might be why to load so many tracks in the Playlist in the first place;

Well, you might say "because I have saved Playlists that contain so many". So yes, a good reason. But now you must be careful with the further approach, which is just subject to the good sound thing, and the preparation and time that takes. This a.o. means that those conversions should not take place during playback, and indeed when those preparations are going on, SQ should be less. This is what the checkbox (Start playback during conversion) is for. You could untick it for better SQ in the beginning.

All implies that one should just not hand a 1000 etc. tracks for playack, also "knowing" that usually these files must be converted, assuming that we all store the files in a compressed format (FLAC). Not necessarily of course, but I must assume it, because you just might (and I myself do to btw). This implies for myself :

I never use saved Playlists. Note that the saved Playlists where there right from the start, and later the Galleries emerged;
A Gallery does just the same, but in a more convenient way, and it resides at a, say, higher level. You can randomize them, pick albums (or tracks, whatever you saved in them) by means of coverart, use them for physically copy (or convert from - to) and much more. Also, they challenge you more to play some kind of selection from it.

The latter is important when you use the saved Playlists anyway. So, they are there, and they are there to be used of course, but, they tend to be large(r), and there you have the perceived problem. However, in the Playlist area you can select too. This is just a matter of windows ctrl-click etc. Thus, when you have this 1000 (or more) tracks in there, just don't play them all. Select the first 100 or whatever you think is suitable for the session and press Play. Or a second batch of 100, or the last etc.

Now you will see where the Gallery is more convenient. I mean, you can select tracks in the Playlist Area, but you can't see what you're actually doing. In a Gallery you can, because you will be selecting by means of coverart. And note that if 1000 individual tracks from different albums are in a Gallery, they all carry the original coverart.

Now, I have no clue why XX needs all those small pesky dat files but I assume you have a good reason for it so np:  I _could_ live with that provided it's done only once but it seems that even if I 'save' playlist and then exit XX it will delete all dat files and the re-create them next time I play that same list?

So yes, those dat files are recreated always. Even when you higher the volume by means of Alt-u during Unattended Playback, they are. But trust me, this is not what takes the time, and it will go unnoticed, even with a 1000 of them. It will be the UnicodeTrack files that bother you. However ...
*Those* files are not recreated at all. This is a rather complex mechanism, and although you might see their names change at changing volume or press Next (Alt-n), they are not recreated as long as you go down the list. So for example, press Prev, and all indeed has to be recreated.

The dat files contain the information for XXEngine3 to work with. If you change the volume or whatever it is, it is in there, and XXEngine3 takes that information for playback. So again the example : change the volume, and the volume data in the dat files will be changed for all the remaining dat files to play.

All that data indeed could be in one file, but then the problem (of resources needed) would be bigger. For each track to play the right spot has to be found in those files, and instead of picking the right file by ID, the file with tens of thousand of records has to be scanned. And no, this is not a sequential process because of the things you might do and change during playback (I don't think it is necessary to explain that, but trust me on it).
The most important is and remains : maintaining those dat files does not take the time. Unless something is wrong in your system of course, and I think it can. If you look at the latest posts around JohanZ you'll see it can.

Wouldn't it also be better if dat files would respect XX setting for 'Data folder' instead of being created in same folder where XX exe is?

Yes, but merely for "IT" reasons. And for those who try to follow what's happening of course, and made the hidden files visible.
In the beginning I did not anticipate on this (and the Data folder wasn't there yet), and right now many things have to be changed in the code to let it work. The decision to have those files just in the middle of the install was explicit, but wrong (as how I look at it now).

And wouldn't it be better if dat files would not have to be created every time but rather saved together with playlist?

As said, they contain real time data (like volume), so no. But again, they shouldn't bother.

I realize this only applies to WAV files as other types need converting but since XX only wants WAV in the end anyway, then at least I am OK with creating a separate folder where all music has already been converted from FLAC etc to WAV - it sure is extra work and it would be nice if XX would respect that and start playing with minimum of fuss

I understand what you mean, but if this requires your own control, this is not what you want. In the very beginning I had to work like this myself, and you'll get crazy of it.
Thinking of the "Copy to XX-Drive" feature, indeed there is a possibility to keep those files like a kind of cache. However, besides it will take time to create it and I don't want to put the priorities to that right now, it occurs to me this just isn't necessary. However, this takes the experience mostly related to the beginning of this post. So it can be done, it will be beneficial to some, but it won't be in soon anyway.

Well, I hope this gives you some insight. Don't hesitate to ask or suggest more.
Peter

[Josef]

Thanks for extensive explanations - no offense, but it  seems to me that some not too great design decisions have been made early on and now it’s impossible to reverse them, short of re-writing the whole thing

I’m not sure I buy the whole Unicode story: why in the world do I care what is filename like if it is a perfectly valid WAV? I just wanna listen to it for Christ’s sakes…
And how come no other player has these problems? Is that really then a Unicode problem or, rather, an XX problem that ought to be fixed?

Anyway, I am getting a bit frustrated with XX: everybody in the world understands the concept of playlists, yet, you seem to suggest that XX playlists are different and should not be used at all or only for really small amount of files – Instead, you say to use ‘galleries’ which I totally don’t dig why (again past design decisions, i guess) but OK, if that solves my problem I’m all for it.

So, how do I create a playlist/gallery thingy that contains all music so I can just click Play and have XX randomly pick a track?
Is that possible at all with XX without 1000’s of file copies or somesuch?

Sorry for sounding a bit agitated (I am): I understand you are trying to be helpful and it’s appreciated – it’s just that many seemingly simple things seem to be so difficult or non-intuitive to do with XX: like this task – it is something that any cr*ppy player can do easily – XX sounds better then all but I am increasingly getting the impression that it was meant to be used only as we did in analog/cd days: look in the box, pick the record/cd, put it on/in, wait till it starts, sit & listen, when finished repeat procedure…..

Hopefully you can point me where I err and if I can use XX like it’s 21st century….

[PeterSt]

Thanks for extensive explanations - no offense, but it  seems to me that some not too great design decisions have been made early on and now it’s impossible to reverse them, short of re-writing the whole thing

No, of course not. But think about it ... only because you set your system to Show Hidden Files you see it, and because of that you want to know, and with some answers you want to change it. But there's no reason really. It just takes a couple of hours to change it, but even a couple of hours are a waste of my current time, if only nothing is really wrong with it. And there isn't.

Also note that these kind of things spring from not wanting to have an official install (it is just a copy to a folder - done), that by itself allowing for having more versions running next top eachother. Maybe it doesn't do too much to you, but to many it does.

I’m not sure I buy the whole Unicode story: why in the world do I care what is filename like if it is a perfectly valid WAV? I just wanna listen to it for Christ’s sakes…

It is not as simple as that. Think about it ...
We're dealing here with the strange combination of a person sitting in a country (with Country Code and all) working with data from all over the world. Normally these things solve themselves by binding the Unicode set used to the Country Code. Here that cannot be, or otherwise you'd be changing your country code per album.
The point is, this data is flowing over data files (which by itself is unavoidable because of the loose running player (XXEngine3)) and files are bound to a country code, or Unicode version if you want. The insight is : Unicode can contain the "languages" for all over the world, but the individual characters (with double/triple ascii value) need to be interpreted differently per country. So there is no way one denoted set can suffice for all.
This problem turns into a real problem when you'd see that file names are identifiers, and when passed over a data file, the other side (the other program) needs to know that interpretenation again, and it doesn't exist. Oh, it does, but not by means of ONE means. So, pass a track name via a file, and the other program can't find it. Not when (visual, that's how you can look at it) characters are in there, not belonging to your country code.

To get the idea, get some data of an album from Amazon (via XXHE I mean). Get the reviews, and get the from all Amazon sites. Now look at the German reviews. Unreadable. This is the same problem. Amazon can't solve it, I can't solve it. But please go ahead; take such an HTML and try to convert it to the proper "code pages". May you succeed, now make a general means for that one HTML page you see from the different countries. Chinese we won't understand anyway, but dutch/german/frensh we will. If you succeed I will be the most happy to implement that means. Ok ?

Anyway, I am getting a bit frustrated with XX: everybody in the world understands the concept of playlists, yet, you seem to suggest that XX playlists are different and should not be used at all or only for really small amount of files – Instead, you say to use ‘galleries’ which I totally don’t dig why (again past design decisions, i guess) but OK, if that solves my problem I’m all for it.

Josef, it is not clear to me why to get frustrated, shout about it and suggest I don't know what I am doing, before even understanding the problem, while I explained about it anyway. So, you just didn't try yet, clearly.
Maybe you are just a bit fast on the trigger, and did not take the time to combine your "everybody in the world" with the "about nowhere in the world" and thinking of sound quality. Remember, it is all about that. Go look in thius forum, search for it, I don't mind, but you will see exactly nobody complaining about these things, or the user interface for that matter (which would be a justified first to come up with all over). On the other hand, you may be the first who honestly talk about this in here openly, while usually this does not happen in here. So, it is appreciated in the mean time, BUT, what's always on top is sound quality.
Of course, if you desire to ignore the Unicode problem, many arguments spring from that "ignorance", but that does not mean they are justified. The problem just is there and it had to be solved. Well, this is the result, and with some good hints there's no problem.
Following responses like not being from the 21st century are beyond mee, then.

Btw, it would have been more fair and useful to justify your apparent slow dat file manipulation, or for that matter, why you think a 1000 of them take tens of minutes to create. IOW, who knows what here ? I do ... (test me).

So, how do I create a playlist/gallery thingy that contains all music so I can just click Play and have XX randomly pick a track?
Is that possible at all with XX without 1000’s of file copies or somesuch?

It doesn't work like you think. Not here. And might you find the procedure awkward instead of beneficial, that's ok with me. However, it would indicate that you don't understand or did not take the effort to try (yes, I am talking like you now, agitated ...). But by all means, once you tried and don't like things, put forward suggestions for improvement. That is what I'm open to. Quite less to shouting.

Allright. A "Gallery" is a subtract from any means of physical music data, or from a Gallery itself. It is meta data.
The most easy example would be getting a random selection of albums in the Library Area, select a bunch by means of normal Windows selecting means, rightclick, and choose "Add to Gallery ...". From there you can create Galleries, or choose an existing one. It's just folders.
This example works at the Album level.
Once you have created a Gallery, you can pick that as a main "folder" in the Library Area again. Normally you'd do that by choosing from the Embedded Explorer at the left (the Library Area being active). Don't forget to press the Search button at the bottom.

The albums (remember, we were working at the album level) you see appear can be subject to many means of other selections, and one of them (the most used) is just typing next to the Search button. It will narrow down the list.
Now, from the albums eventually remaining, you might pick one or two by means of Windows selecting again, and click Load. This causes them to go into the Playlist Area, and now they are ready to play.
Keep in mind : in the Playlist Area you again can select by Windows selecting means. Only the selected will play, or if none (or only one) is selected, they will play all.

Now, for working at the track level, the most easy example would be rightclicking a selection (or one only) in the Playlist Area, rightclick (the row must be completely blue for it) and choose "Add to Gallery ..." again. So, now you are adding individual tracks to a Gallery, or when you selected all tracks from an album it will be the same as a complete album (or leave one track out if it, etc.).

A more "difficult" way of explicitly creating playlists at the track level, is going to the Library Area again, but replace the "A" in the  box next to the textbox where you can enter stuff for narrowing, with a "T". Now, whatever means you use for Search, individual tracks will be shown. And, after they are there, you can narrow them down via the usual way, and you can select again for adding to (other) Galleries.
Note that this little box can also be filled with time boundaries, to e.g. select tracks of e.g. 6 minutes at least, or 3 minutes at most. See the ToolTip on it.
And once you have created your Playlists by this means, you can again ask for them, and load selections of it into the Playlist Area.

Note that more options exist at rightclick on a selection in the Library Area, and some Randomize functions are in there as well.
Btw, a most convenient "randomize" function is the small "R" button, which just puts forward one of the items in the list, no matter how the list itself got there. The little "D" button might give you your newly ripped albums (or tracks) from the last 20 days, ... whatever.

Well, I never find myself with a list of 1000 items in the Playlist Area, just because I am working in a kind of more dynamical way perhaps. With this I mean that I never use any same (saved) Playlist, just because it would always be the same. However, I do have "playlists" saved as a Gallery, and from there I select again, always. Now, whether 1 or 2 or 5 albums remain from that whatever selection, it won't be a 100. Why ? because I am not going to listen to those 100. This is obvious. Well, for me it is. I try to sleep once in a while.

Keep in mind that the Galleries can be treated recursively. Thus, you can make Galleries of Galleries endlessly.
But also note that the exact same principle allows to copy albums. Or copy to FLAC and maintain the FLACs where the FLACs already were there. And the other way around.
Then, try to see through the mechanism of this being your opportunity to make one logical drive of all your TB disks. Galleries do that. Copying, hence backups, use that too (obviously, once you got the grasp of it).

And oh, before you come up with it yourself, if you go around the XX means of maintaining etc. the Galleries by means of Explorer (also the Embedded one), that won't work. Not yet. XX will not know you did it, and things get mixed up. Just not finished here ...

Lastly, don't forget to drag the right side of XX to the right;
When you ticked "Show data", visible at the bottom with the Playlist Area active, more coverart is visible there, if you have it. There too you can select and rightclick. It's self explanatory.

Well, I hope this gets you started a bit, and don't forget : I am open to everything, and the wishes I did not fulfill can be counted on one hand for just not being done, and possibly two fingers because it was really rejected. But always with good reason, although in that one occasion it could my my own reasons. Some of it is future strategy which I know only. If people here disagree, I expect them to come forward now. We'll see what comes from that.

But Josef, please be constructive. Ok ?

Peter

(not checked for typos)

[Josef]

Hi Peter,

I was travelling and did not have time to respond - I'll try to address your points in next couple messages.

About the Unicode thing:

>This problem turns into a real problem when you'd see that file names are identifiers, and when passed over a data file, the other side (the other program) needs to know that interpretenation again, and it doesn't exist. Oh, it does, but not by means of ONE means. So, pass a track name via a file, and the other program can't find it..........If you succeed I will be the most happy to implement that means. Ok ?

If I understand you correctly you're saying that because you put filenames in .dat files this is the reason for requiring old 8.3 filenames? (because if you used normal filenames you run into Unicode issues?)

And do I understand correctly that was also the reason why you have to copy a perfectly valid WAV file to XX folder and name it UnicodeTrackxxx.etc? (because if there is no 8.3 filename other process potentially can't find it?)
 
If so, have you looked at symbolic links in Vista?
I am not a programmer (though I used to do that long time ago so i have some idea how things work) but it seems they would allow you to create a small shortcut which you can name anything you want (= 8.3 compatible) and so you would not need to copy large WAV files around?

Alternatively, perhaps an even better way would be to use inter-process communication between GUI & Engine process? Then neither WAV copying nor dat files would be needed anyway as GUI process could track data from *.dat files in memory and either send messages to Engine or reply when asked (whatever works best for less CPU use and better sound)?

You mentioned that tracking *.dat files in memory would take too much resources - it seems they are always around 350 bytes per file which even for 10,000 tracks would result in only 3.5MB RAM which, if I look at task manager, is, at this moment in time, less then 1% of RAM XX uses. As those apparently need to be updated if user changes volume it would seem to be much faster doing it in memory than updating 10,000 files on disk. This is an exaggerated example, of course, but the principles should still apply in general case and would make XX more responsive and far less I/O hungry which, if I understand correctly is always an enemy of SQ right? 

BTW I have personally went the all-WAV way as I find it causes XX to behave best (=minimal I/O) so I'm not so much affected by this but  'regular' FLAC,mp3 users might benefit?.... 


 

[Josef]

About Galleries:

You were wondering why I got so frustrated with XX - It was mainly because of the whole Library thing: it simply did not work. XX would seemingly randomly completely freeze - very, very frustrating....
Anyway, I spent some time debugging and was able to find the cause: it seems XX is a bit too sensitive to cover art files. It so happens that tool for getting artwork from internet which I used some time ago would put an 'empty' jpeg file if it could not find the real artwork. It's not really empty (see attachment) and I can open it OK if I click on it (so it appears to be a perfectly legitimate jpeg) but for some reason this small jpeg causes XX to completely freeze and killing it from task manager is the only option....

Anyway, after getting rid of this jpeg I thought lets' try this Gallery thing: I created a test gallery with just 3 albums. This went ok although I find the procedure slightly involved for the average user but never mind (first open Library tab, click & select 3 albums, select 'add to gallery', go back to Settings, change Music Root to newly defined gallery folder, go back to Library tab). However, when I click 'R' button, as you suggested, nothing happens... And if I click on root Gallery folder I don't get cover arts from subfolders? (see attachment) This seems to be different from 'normal' music folders where I _do_ get all cover art as expected...

So I guess I am doing something wrong as I can't Galleries to work and I guess you can explain what it is I am doing wrong.

However, I can't help noticing that if I use e.g. Foobar or even iTunes all I have to do is drag & drop the files from Windows Explorer and I have a playlist which I can either randomize manually or just click on 'random play' and let Foobar/iTunes randomize on-the-fly - the fact that I have to do so much clicking around and figuring stuff out is IMHO a serious usability disadvantage of XX which is really a shame as I really would like to be able to use only XX as it sounds waaay better....

So, what I do right now is using Foobar to quickly create a playlist, randomize it and then drag & drop it onto XX playlist window - this does work but for one small annoyance: it seems the order gets messed up - this is most likely a problem with Foobar as I get same behavior if same playlist is dropped on iTunes so it has nothing to do with XX but question remains: is it possible to have 'random play' in XX and, if not, I, for one, would like to cast a vote for it....   



   

[PeterSt]

Well, let's start with the Galleries.

And let's begin with your end : randomizing the Playlist itself -> that's ok with me. Not difficult either.

About the empty jpeg, hmm ... I will try it with you provided example. Note though that it was you who saved that file in the first place. Not XX or Amazon etc. So in the end I don't get what happened and/or what you did about it. Note that the data obtained from Amazon must be explicitly saved (the yellow bordered items).
But thanks for finding the cause. I will make it proof to that anyway.

I created a test gallery with just 3 albums. This went ok although I find the procedure slightly involved for the average user but never mind (first open Library tab, click & select 3 albums, select 'add to gallery', go back to Settings, change Music Root to newly defined gallery folder, go back to Library tab).

Yes that looks tedious. I'd be the first to agree. But ... somehow your approach to, say, "playback" is different from others. I'm not sure where it comes from, and probably it is justified (why not), but it is not how XX has been setup. I mean (in general that is) : you seem to want to create playlists explicitly and physically, but this is not what is being done, and at least I never do it. Ok, I did, when the Galleries weren't there yet ...

In fact I think I explained it in my previous post, but probably you have it in your mind somewhat different and now it doesn't come through. What if I say this : Playlists as such indeed are being made, but not saved. Does that help ?

So, what you got from my post is creating Galleries in order to act as playlists, or something like that. And yes, when done like that, it's a tedious operation. Of course you *can* go about like that, and most probably it was implied by me as an answer to wanting to have playlists, but still the rule is : "... but not saved". Remember what I told earlier about myself : I wouldn't like to listen to the same tracks over and over. So why save a playlist ?

Maybe I'd better explain it from another angle, and tell what the Galleries are for;
The Galleries are there to group your albums to whatever angle you want, and one album can be in as many Galleries you like. For instance, a Miles Davis album could be in Jazz as well as in Male Singers at the same time.

Looking at your example structure it looks wrongish to do everything you want to do with it. It should be

XXHighEnd (as the main root for music data)
--Pop
----Abba
------Best of
------Gold
----Beatles
------Nr1
------Yellow Submarine
----Steve Miller Band
------The Joker
--Ambient
----Fripp & Eno
------KaZeMa
--Rock
----Steve Miller Band
------The Joker

Note that the only entries you made explicitly are XXHighend, Pop, Ambient and Rock. The others just emerge by appointing the the albums to an entry like Rock. Thus, appoint The Joker (which is somewhere on disk) to Rock, and the remainder of the structure emerges automatically.

What I pointed out in the above is not new, and otherwise you will say "yea, I could make that up myself". True. But the merits of this structure are much more rich than your example. You now can :

Select the main XXHighEnd folder. And indeed, this is not allowed by clicking on it, which you can do for everything else. So, since XXHighEnd is the root, it will be shown in the textbox at the bottom, and it is a matter of removing the right part and click search. Note that after this it's in the Favorites (F button) and you can select it from there. Btw, this is what I personally almost always use for a source (and hardly the Rock etc. folders).

When the structure is like this, you can click Pop, Ambient etc., and there is your list of Pop etc. albums.
Might you have a pile of Pink Floyd under Rock, then you'd see that at traversing down from Rock to Pink Floyd, you can click the Pink Floyd folder, and there are all your PF albums. Or click one. Or click one track (this latter is always dangerous because it takes explicit attention by me because of some redundancy there).

Now it is the whole point, that whatever emerges in the Library area, it can easily be narrowed down by typing in the textbox next to the Search button. And try to keep in mind, when I ask for your "XXHighEnd" folder, 12000 albums pop up, or some 150,000 tracks for that matter. So, no matter I selected unaccording the type of music (hence I selected nothing), at typing Pink F ... there they are instantly. Including a few classical covers this time.
Note that I also could type ABBA in advance, and *then* start to select from the embedded explorer. Only things containing ABBA will show up.

There are many more things for narrowing down (like date creation), and they are not difficult to find.
But now back to the subject : with these facilities at hand, why would one ever need to create a *saved* playlist ?? I just don't get that, never mind it will be justified to want it. But since it is (as perceived by me !) not needed, why go through the trouble.
But you could try this too :

Whatever you have in the Playlist at a certain moment, and no matter how it emerged there, if you like it as a playlist, you can select all the tracks, and rightclick - save it to a playlist you create on the fly. This is less than 10 seconds work. This playlist as I see it (and wich kind of was the angle I talked from in the previous post) is a Gallery with all its faciities, and once asked for via embedded explorer, you again can select and narrow down as usual.

Josef, it seems you know a bit more from IT stuff than average, and it should not be difficult to get the merits of this. But you forgot one thing : setting up your music library in the first place. You just don't have it, although you have albums on disk somewhere of course.
Keep this in mind, because I think it is important.

Another thing is, I'm afraid you never got to listing your physical albums in the Library Area. Just -again- select a root folder to it by means of that same textbox at the bottom, and they will all show up. This is your base for creating Galleries. You most probably already will have a Pop and Jazz etc. structure. You can just use it from there, select that Jazz, next select all the albums showing up and create a gallery Jazz from it under that XXHighEnd Gallery folder. And when you're at it, press the R there. It will work. But what it does is bringing forward albums (or tracks) in the visible area. That won't work with three only, because all three are already visible ...

Now, if you look at the pile of text above, I can very easily type a 10 fold of this to explain all about the Galleries. Easily. And it has to be done for the manual the upcoming time. I only want to say, there is much much more in there than you'd expect, but look a bit through IT eyes. Not everybody can do that, you can.

But randomizing the Playlist itself ... it can be done. Why not.
Peter

(not checked for typos)

[PeterSt]

The DAT file thing.

And do I understand correctly that was also the reason why you have to copy a perfectly valid WAV file to XX folder and name it UnicodeTrackxxx.etc? (because if there is no 8.3 filename other process potentially can't find it?)

It seems a long way around, but basically that is what it comes down to, yes. This is not all though, and part of the problem is displaying it correctly. So, it is unlike you said in the beginning that this is about playing the music only. It is also nice to see what you're playing.
Remember, the trick is that from an 8.3 name the original can be obtained, and while (code page) conversions don't work, this works.

If so, have you looked at symbolic links in Vista?

Yes, I have. But I can't recall anymore why they couldn't do the job. Maybe something with rights, maybe something I could solve today because of more knowledge ... but back then, it was useless.

Alternatively, perhaps an even better way would be to use inter-process communication between GUI & Engine process?

In theory yes, but I think you didn't get that there is no GUI at all. There's just nothing. Also under the hood ... nothing.

You mentioned that tracking *.dat files in memory would take too much resources - it seems they are always around 350 bytes per file which even for 10,000 tracks would result in only 3.5MB RAM which, if I look at task manager, is, at this moment in time, less then 1% of RAM XX uses. As those apparently need to be updated if user changes volume it would seem to be much faster doing it in memory than updating 10,000 files on disk.

I don't recall I said something like that, although I will have said something that makes you think this. Anyway :
Same problem as before : there is nothing in memory to change, because ... well, there is nothing in memory. I'm afraid you try to understand something which isn't even there. Besides that, it is irrelevant. Also the context of 10,000 files is not "justified", as talked about before.

but the principles should still apply in general case and would make XX more responsive and far less I/O hungry which, if I understand correctly is always an enemy of SQ right?

Ehhm, wrong ?
I think if you want to talk in terms of "I/O hungry" it is time to tell, well, don't talk in suggestive terms while you actually don't know. Strange though, because you were able to explain something about 350 bytes. So you do know, which leaves "suggestive";
But ok, here are your numbers :

A coverart picture is some 100KB. Better throw those out, because already one of them is 286 times larger than those 350 bytes.
Next is the track to play, which 1:1 goes along with that 350 bytes. I advice you to make a 64 mp3 of everything, because such a (5 minute) 50MB track really is 142857 times larger than those 350 bytes. Forget about hires files at all.

Now, shall we stop this subject so I can continue with useful things ? (btw, go ahead if you like; I really don't mind. What I do mind is justifying about things which really don't make sense).
One of them might be examining those symbolic links again. Note though that this won't help me a bit for those who have FLACed everything. It also doesn't help when you denote "Copy to XX-Drive" which for sure you will do once you have an SS disk.

Peter

[Josef]

>Note though that this won't help me a bit for those who have FLACed everything. It also doesn't help when you denote "Copy to XX-Drive" which for sure you will do once you have an SS disk.

I _had_ everything FLACed. And, yes, I _do_ have an SS disk.
And what I got was XX creating 100+ of UnicodeTrackxxx.WAV files which used all capacity on SS disk.... (and without 'Copy to XX-drive' being checked) 

That is the whole reason why I am trying to give you suggestions on how to avoid this and minimize disk usage (SS disks are still fairly low in capacity - mine is 64G and it's not used solely for XX...)

But nevermind, seems we just don't understand each other: you have an idea of how XX should be used which to me is way too complicated but that's ok. What I was looking for is ability to have XX always play randomly from my complete collection with minimum fuss (and minimum disk usage) - just like 'Play random' in Foobar or Shuffle in iTunes. Nothing more, nothing less. But if you think such feature is not useful then I guess I'll just have to leave it at that...

[JohanZ]

...What I was looking for is ability to have XX always play randomly from my complete collection....   

Yes please!
Selecting Gallery then random play for all, selecting Jazz then only random play for jazz.

[PeterSt]

Josef, I am sorry, but you don't get it. What you quoted from me is somehow interpreted by you 180 degrees the other way around;
When you have a FLAC album, it has to be converted and the result is again such a Unicode file.
When you copy your files to the XX-Drive, well, they have to be copied, right ?

And if you want to do that with 10,000 files, you can be as persistent as you want, it is not going to work. Not with a player like this. This is not about me but about the principles of the player. And its purpose of course. Well, you know ...

I don't understand what you don't understand about this. I am sorry.
Peter

[PeterSt]

Josef,

I may have sound harsh, which I don't intend to. So, to be clear on things :

Everything which has to be converted, or copied for that matter, has to be done in advance in order not to disturb SQ. However, since it might be an annoying thing to wait some longer before the first notes sound, the check box "Start Engine3 during conversion" has been brought up. Now, have too many tracks in the Playlist to convert or copy, and SQ degrades for a longer time or doesn't commence for that same time. Nothing else to do if you don't want the influence at each track. This is just how it's designed ...
Btw with the most explicit part of conversions not being done on the fly (like normal players do). It really matters.

Peter