XXHighEnd

This Tutorial describes how Albums from different "Libraries" can be compared. Your objective : Find albums obtained in the one Library not present in the other and how to add them to the other.
Library : A logical set of Albums as contained on a disk, a folder in it, or grouped by means of a Gallery.
Gallery : Meta data or pointers to locations where the albums really are.
Don't worry too much as of yet ...

---

Our objective for this Tutorial is how to de-dupe (deduplicate, remove doublures) albums from our collection. However, because it is more easy to explain what needs to happen by means of a virtual "new disk" with albums of which we wonder whether we already have processed all what's on it in our regular collection, we'll assume such a situation. Thus :
- New disk;
- Our collection.
And the question : What on this new disk do we already have in our existing collection.

In order to be able to let XXHighEnd "see" what's all in our collection, all our albums must be accessible from one logical location. This means that no matter on how many physical disks (etc.) our albums are stored, one list with all of them should be available. This is where Galleries come into play. Look in this Tutorial how to create a main Gallery of all your albums : Galleries and how to create them (http://www.phasure.com/index.php?topic=2867.msg30119#msg30119).
It is assumed that this is arranged for; Logically you will now have denoted that Main Gallery as "Music Root" in Settings, but this is not a requirement (it is convenient though).

(http://www.stordiau.nl/xxhighend/AlbumCompare01.png)
This shows the first part of our Main Gallery, or IOW the root of our Galleries. You see it in this case as C:\Galleries\ in the bottom of the screen. It also shows that 36007 albums should be in there, or better : 36007 Gallery Entries.

Now let's grab our "new disk" and say this is a disk I maintained on a different system, in my case a Mac; I copied my albums as maintained on the Mac to a USB disk and I connect it to my main system (where XXHighEnd is used).

(http://www.stordiau.nl/xxhighend/AlbumCompare02.png)
I used the button under the mouse arrow (near the bottom) and you see the "new disk" under drive D: (named GA).
Because I can't play DSD anyway at this moment, I select the AIF folder in there (not visble in above picture), press OK and what comes from it is this :

(http://www.stordiau.nl/xxhighend/AlbumCompare03.png)
Now I want the music contents of D:\GA\AID\ to show up in the Library Area and press the Search button. The result :

(http://www.stordiau.nl/xxhighend/AlbumCompare04.png)
Apparently there are 140 (AIFF) albums on that disk (in that folder I selected). Not all contain Coverart but that's another matter.

Now I want to know which of these 140 albums I already have in my main collection (hence which are obtained in C:\Galleries\). Therefore I do this :

(http://www.stordiau.nl/xxhighend/AlbumCompare05.png)
I selected all the albums in there (click the first and apply ctrl-shift-end), did rightclick and will choose "Compare Albums ...".
This screen shows up :

(http://www.stordiau.nl/xxhighend/AlbumCompare06.png)
The "Albums to find (newly obtained)" field is grayed out because it is that "disk" which currently is in the Library Area.
The "Look in existing source" field defaults to C:\Galleries\ because that is our Music Root in Settings.

So, our "existing source" is functionally all we have for albums, but notice that any other sub structure of the Galleries can be appointed (like C:\Galleries\Jazz\) or that any Original Location (where the albums really are stored) can be entered. So it is just want you want to do, and in our case we want to know which of these 140 albums we already have.

Notice the "Compare method" which is set to "Find not present". This means that once we press the Compare button in the bottom right, the result showing in the Library Area later will be the albums "not present" in our main Gallery.

Notice :
Each album which is processed by the Album Compare function for the first time, needs to be generated a unique "ID" which relates to the music content (and not to the name). The generation of this ID takes time and therefore the very first time the Album Compare is applied to your whole collection can easily take an hour or more, of course depending on the size of your collection (but roughly think half an hour per 10000 albums of course still depending on many more things like disk(s) and processor speed).
Once the IDs have been assigned, all blasts through (few minutes for the shown 36000).

(http://www.stordiau.nl/xxhighend/AlbumCompare07.png)
When the Compare button has been clicked, the process begins and while the progress bar shows that things are being processed ...

(http://www.stordiau.nl/xxhighend/AlbumCompare08.png)
The "Album counter" in the bottom of the XXHighEnd screen also runs.

You will see three passes :
1. The "newly obtained" is processed / prepared for comparison;
2. The "existing source" is processed / prepared for comparison;
3. The both are compared against eachother.

When done, this message appears :

(http://www.stordiau.nl/xxhighend/AlbumCompare09.png)
This message tells you that the result of the comparison is stored as a Result List and while the result currently is shown in the Library Area (seepicture below) it can happily be retrieved latet for further processing (we'll talk about that in a minute). For Result Lists, see here : Save Library Result Lists (http://www.phasure.com/index.php?topic=2057.msg21094#msg21094).

(http://www.stordiau.nl/xxhighend/AlbumCompare10.png)
So here we see (the end of) the Result List of the Comparison and apparently 119 of the 140 albums I do not have in my XXHighEnd environment.

In the mean time when the [ F ] button is rightclicked (see mouse arrow in above picture) we'll see this :

(http://www.stordiau.nl/xxhighend/AlbumCompare11.png)
And when the list is opened (see mouse arrow above) we can see our result under the mouse arrow below :

(http://www.stordiau.nl/xxhighend/AlbumCompare12.png)
... and we can see that we can happily close XXHighEnd with the obtained Result List now showing in the Library Area because we can retrieve it later (actually per means of above two pictures).

All right. Let's assume that all we see in the Result List is indeed correctly showing the albums we don't have in our main Gallery. So what's next ?
Next we want to copy these albums to our normal environment so that the disk we obtained from like in my case the Mac can go back to there.

(http://www.stordiau.nl/xxhighend/AlbumCompare13.png)
We can again select the items in the Library Area (I selected all of them) and next choose one of the Copy To functions. You will be provided with the possibility to select a target location (not shown here) and when this is all chosen the selected albums will be physically copied.

If we do it like this, and now referring to not much hassle and just copy all to one folder (which implies sub folders for the albums as usual), all will be copied to that main folder indeed but it will not be grouped (into Jazz etc.); this can be done later by means of Galleries and it is not the subject here. But notice that you can also take your time to album per album denote a grouped location in the first place (so, copy the Jazz albums physically to your storage location with Jazz albums). Anyway :
Once the albums have been copied, one way or the other they now should be obtained in your main Gallery, waiting for the next time of an Album Compare. Thus, when we'd omit that our "newly obtained" albums will not be seen as "already there" and they will not be included in the Comparison. Moral : do not forget to obtain these new albums in your usual Gallery structure (like Jazz in there and all) and no matter where you put them, they will be under your main Gallery structure so the next time they will be included in the Comparison.
Not shown here, but when I had done this all my number of albums shown from the Mian Gallery (C:\Galleries\) would show 36126 (36007 + 119).

---

The procedures all other ways around

Try to wrap your head around this :

(http://www.stordiau.nl/xxhighend/AlbumCompare14.png)
Here I denoted "Find Present" and the remainder of the fields are left untouched;
What will be the result of this ? 29 albums (140 - 119) which I already have on the main system. It can serve as a kind of double check; further not much useful. But now this one :

(http://www.stordiau.nl/xxhighend/AlbumCompare15.png)
Here I started out with asking for my main Gallery in the Library Area, I selected all again and next denoted the 2nd field to that same "new disk" from the Mac. What are my intentions now ?
I now want to apply the procedure the other way around as described in the large outlay above and I want all of my albums on the main system to be copied to that other (Mac) system. So now the "GA" disk represents all on the Mac as how it came from there, and on the main system I want to compare all what is not on that GA disk and the result (all what's not on it) I want to copy to there.
What will come from this is a Result List of 36007 - 29 = 35978 albums and if I select all of them for "Copy to" I could virtually copy them to the GA disk and next the Mac system will be appended will all the albums from the Windows system.

Notice : I said "virtually copy to the GA disk" because it won't fit. However, what will happen is that all is copied until the target disk is full and there it stops on the boundaries of full albums; You will easily see what was copied last (look at the target disk) and when a new empty disk is connected you will start the copy from where the other stopped (matter of selecting the proper first album (top to bottom) in the Library Area before selecting "Copy to".

---

Off-line procedure

This is quite something else;
Imagine that on the Mac I have running XXHighEnd just the same. I want to achieve the exact same as the main procedure but now I can't detach the disk from the Mac to connect to the XXHighEnd PC;

(http://www.stordiau.nl/xxhighend/AlbumCompare16.png)
What you see here is that I activated the "Or Export this Path" field. Notice the "Or" which tells that we are not going to apply any Comparison at all; we are only going to export the existing albums on the Mac via a small footprint file;
I'll press Compare in the bottom right anyway to create the file;
Within a fraction of a second this pops up :

(http://www.stordiau.nl/xxhighend/AlbumCompare17.png)
Aha.

(http://www.stordiau.nl/xxhighend/AlbumCompare18.png)
And indeed, there is is ... (5KB in size).
I move over this file to the XXHighEnd PC, put in in my \XXData\ folder over there, rename it to Import .txt and now do this :

(http://www.stordiau.nl/xxhighend/AlbumCompare19.png)
So now I activated "Or Import comparing to this Path" with again the emphasis on the "Or" now implying that the upper part of the screen is actually not active at all.
And I'll press Compare again ...

(http://www.stordiau.nl/xxhighend/AlbumCompare20.png)
This is now the result ...
Remark : The 35985 you see there is 3 off from 36007 - 19 because I found an erroneous album and had to mangle with two others which appeared to have a wrong structure. That left alone : What noa happened ?

I exported te "data" of 140 albums (but think 138 by now) from the Mac side of things and I imported that to compare with my present list in C:\Galleries\. I denoted "Find not present". And about trying to wrap your head about things, think about what the purpose here can be. So :

The export on the other machine only presents some small footprint data (I could email it to an office PC elsewhere). The *purpose* of this activity can only be to receive back physical album data from that PC elsewhere. And this is now the reason that the Result List shows the not present albums compared to that export from elsewhere. Further intention :
Select this whole list, "Copy to" another disk and bring over that other disk to the place where the export happened (my Mac).

The reason that it works this way is because no Library Result list (with Coverart and all) can be shown from the small footprint data; the data (like Coverart) is just not in there. And so the purpose of the activity has to be taken into account.
So notice : When I would have activated the "Not Present" it would imply that the list coming from the Mac would show in the Result List (in the Library Area). And this is NOT possible ...
Thus :

The PC which receives the exported list and which imports it on that PC (Import.txt in \XXData\) will show what is NOT present in the exported list at the other end, to next enable copying all that. Literal sequency (now incorporating an office PC elsewhere) :

1. Export list on Mac.
2. Email the Export.txt to Office PC elsewhere;
3. Put the file as Import.txt in \XXData\ in Office PC;
4. Run the Compare through "Import Comparing this Path" while ordering for "Find no Present".
5. Physically copy the Result List (hence copy the albums) to a disk;
5. Bring over the disk to the Mac and connect it;
6. Run the Compare like the main description if you like (but all should be "not present");
7. Use the disk as it is or copy it to the storage locations as you want.

Why this seemingly difficult way of working ? Because
a. physically copying ALL of the Mac albums is thus (!) not necessary
b. while physically copying the albums from the Office PC to the Mac was the idea in the first place. So *this* step of transferring a disk can not be avoided to begin with.

It just is the leanest procedure possible.

---

Errors occurring

When the Album Compare procedure is run for the first time, it can easily happen that errors occur. Most likely they will be related to a "wrong structure" which isn't detected by other XXHighEnd means. For example, an album folder (with thus music data in there) which is a sub folder of another album folder. This is just not allowed;

If you look at the Album Compare screen you see an "XXCP-" mentioned in the bottom right; Log Files starting with this prefix will always emerge regardless the Log On/Off settings in XXHighEnd. This log file will show you how the comparison went along and more importantly, what ID's were used for the albums. Now notice that it can happen for all kind of reasons that you want to learn how or why an album was rejected as being equal; Just use the ID to find further lines in the log file containing that ID.
Explaining all is way being this tutorial, but it may help you.

When a real error message occurs, you should see that same error message in the normal XX (prefixed) Log File. BUT, this requires Logging to be On. It is always good to have Logging On because in the XX log file also interesting entries will show.

Lastly, how you often won't trust albums to be NOT the same, while you were sure they are. Example :

(http://www.stordiau.nl/xxhighend/AlbumCompare21.png)

You may have seen this album in one of th first pictures in this Tutorial. Well, it came out as "not present". Of course I don't trust it. So I load both in the Playlist Area (the one I already had on my XXHighEnd PC and the one coming from the Mac) and the second one starts at the highlighted track.
Notice that I can also use two XXHighEnd instances for this and put them next to eachother for more easy comparison. Now :

It is obvious that the two are not the same at all because the second one (this one came from the Mac) has 12 tracks while the original 9 (as I already had it in the XXHighEnd PC) has 9 tracks. But also look at both track's 02. That difference in time can easily be dedicated to ripping (CDRom anomalies and ripping settings); this is as well as possible eliminated from the equation. But now also look at track 03. This 5-6 second different for sure is not the same "mix" or whatever happened. So, a thing like this already makes the album different; the difference of track 02 will not.
But of course the different number of tracks is leading here.